JP2021097727A - Image processing device, image processing method and program - Google Patents

Abstract

To provide an image processing device that detects an anomaly from an optical image to support an operator who detects a foreign matter.SOLUTION: An image processing device includes: an image acquisition unit for acquiring an optical image in which a subject is optically imaged and a radiation image in which the subject is imaged by radiation; an anomaly detection unit for detecting an anomaly relating to the subject from the optical image acquired by the image acquisition unit; and a display control unit for displaying an object on the basis of anomaly information relating to the detected anomaly together with a radiation image when the anomaly detection unit detects an anomaly from the optical image.SELECTED DRAWING: Figure 2

Description

The present invention relates to an image processing apparatus, an image processing method and a program for detecting an abnormality from an optical image.

The radiographic image can acquire image information in the body of the subject that cannot be acquired by the optical image. By using a radiographic image, the position of a foreign substance such as a broken piece of glass that has entered the body can be specified, so that the operation can be performed based on the specified position and the foreign substance can be removed.

Depending on the radiographic image, foreign matter in the subject's body may overlap with hard tissue such as bone. When a foreign substance overlaps with a hard tissue on a radiographic image, it becomes difficult to detect the foreign substance from the radiographic image, and the risk of oversight increases. If the foreign body in the body is not found on the radiographic image and the foreign body is not removed by surgery, the symptoms of the subject may become severe.

Patent Document 1 discloses a technique of determining the body position of a subject and changing the display direction of a radiographic image with respect to an optical image acquired by using an optical camera.

JP-A-2018-68822

However, in the invention described in Patent Document 1, although the body position of the subject can be determined from the optical image, the configuration that assists the operator in finding a foreign substance by detecting an abnormality from the optical image is provided. Not listed. For example, when bleeding is detected from an optical image, the operator can observe a radiographic image based on the bleeding information, and it becomes easier to find a foreign substance.

The present invention has been made in view of the above problems, and an image that assists an operator in finding a foreign substance by detecting an abnormality from an optical image and displaying anomaly information about the detected abnormality together with a radiographic image. It is an object of the present invention to provide a processing apparatus.

In order to solve the above problems, the image processing apparatus according to the present invention is
An abnormality detection that detects an abnormality related to the subject from an image acquisition unit that acquires an optical image obtained by optically photographing the subject and a radiation image obtained by radiographing the subject, and the optical image acquired by the image acquisition unit. It includes a unit and a display control unit that displays an object based on the detected abnormality information regarding the abnormality together with a radiation image when the abnormality is detected from the optical image by the abnormality detection unit.

According to the present invention, it is possible to assist the operator in finding a foreign substance by detecting an abnormality from an optical image and displaying an object based on the detected abnormality information together with a radiographic image.

System configuration diagram of the radiography system according to the first embodiment Configuration diagram of the image processing device according to the first embodiment Display screen view of the radiography control device according to the first embodiment Flow chart of the image processing apparatus according to the first embodiment Configuration diagram of the image processing device according to the second embodiment Flow chart of the image treatment apparatus according to the second embodiment An example of an abnormality input screen according to the second embodiment An example of a setting screen according to a third embodiment Flow chart of the image processing apparatus according to the third embodiment Modification example of system configuration in radiography system

Hereinafter, embodiments will be described with reference to the drawings. The configuration shown in the following embodiments is only an example, and the present invention is not limited to the illustrated configuration.

(First Embodiment)
<Configuration of radiography system>
FIG. 1 is a diagram showing an example of a configuration of a radiography system according to an embodiment of the present invention.

This system has, for example, an image processing device 100, a radiography control device 110, a radiation generator 120, and an optical image acquisition via a network 140 composed of a LAN (Local Area Network), a WAN (Wide Area Network), or the like. It is composed of the device 130. The network 140 may be a wired network or a wireless network.

The image processing device 100 acquires an optical image optically photographed by the optical image acquisition device 130 and a radiographic image radiographed by the radiographic imaging device 115. Then, the image processing device 100 performs an abnormality detection process on the acquired optical image, and displays the detected abnormality information together with the acquired radiation image. The image processing device 100 is a device composed of an information processing device such as a computer. The computer is provided with, for example, a main control unit such as a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. As long as the image processing device 100 is connected to the radiography system via the network 140, for example, the configuration of the image processing device 100 or a part of the configuration may be on the cloud. Further, the configuration of the image processing device 100 described later may be realized as a part of the devices constituting the system.

The radiography control device 110 is a device constructed by an information processing device such as a computer that communicates with the radiography imaging device 115 to control radiography. The computer is provided with, for example, a main control unit such as a CPU, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The radiography control device 110 received from a network device connected to the network 140, an operation unit 111 that accepts operations by an operator such as a keyboard and a mouse, a control unit 112 that controls the entire device, various control programs, and a radiography imaging device 115. It is composed of a storage unit 113 that stores a radiation image, an optical image acquisition device 130, received image information, and a display unit 114 that displays a radiation image such as a liquid crystal display. Although the operation unit 111 and the display unit 114 are described as separate devices, an operation unit in which these devices are integrated may be used. The radiography control device 110 also communicates with the radiation generator 120 to acquire information when radiation is emitted from the radiation generator 120. The radiography control device 110 communicates with the optical image acquisition device 130 to control the optical image acquisition device 130 and acquire the optical image optically photographed by the optical image acquisition device 130. PACS150 (Picture Archiving and Communication Systems) is an image server. The radiography control device 110 adds inspection information to the radiographic image taken and transfers the radiographic image to the PACS 150.

The radiography apparatus 115 transitions to a state in which imaging is possible according to an instruction from the radiography control device 110, performs radiography while synchronizing with the radiation generator 120, and emits radiation based on the radiation emitted from the radiation generator 120. It is a device that generates an image. The number of radiographic imaging devices 115 is not limited to one, and a plurality of radiographic imaging devices may be used.

The radiation generator 120 detects a radiation irradiation instruction by the operator and generates radiation from the tube 121 based on the irradiation conditions set by the operator via an operation unit such as an operation panel that accepts the operation by the operator. It is a device.

The optical image acquisition device 130 is a device that acquires an optical image by performing optical photography according to an instruction from the radiography control device 110. In the present embodiment, an optical camera is used in the optical image acquisition device 130 to acquire an optical image. In the present embodiment, the image acquisition device 130 is attached to the tube 121, photographs the radiation generation direction of the tube 121, and has an imaging range equivalent to that of the radiation image.

The above is a description of an example of the configuration of the radiography system according to the embodiment of the present invention. The configuration shown in FIG. 1 is merely an example and can be changed as appropriate. For example, in FIG. 1, various devices are connected to the image processing device 100 and the radiography control device 110 via the network 140, but the radiography system does not necessarily have to have the device configuration as described above. .. Further, each device may exist in a plurality of devices on the network.

<Abnormality detection processing>
Here, with reference to FIG. 2, each configuration and each function of the image processing apparatus 100 constituting the radiography system will be described.

The image processing device 100 constituting the radiography system is for the optical image from the storage unit 113 in the radiography control device 110, the image acquisition unit 102 for acquiring the radiation image, and the optical image acquired by the image acquisition unit 102. It is composed of an abnormality detection unit 103 for detecting an abnormality, and a display control unit 104 for displaying an object based on the abnormality information related to the detected abnormality on the display unit 114 of the radiography control device 110 together with the acquired radiographic image.

The image acquisition unit 102 acquires an optical image and a radiographic image stored in the storage unit 113. Then, the acquired image is transmitted to the abnormality detection unit 103.

The abnormality detection unit 103 performs an abnormality detection process on an optical image among the images transmitted from the image acquisition unit 103. In the abnormality detection process, for example, an abnormality is detected using CNN (Convolutional Neural Network), which is one of machine learning. That is, the abnormality detection unit 103 is characterized in that it detects an abnormality based on machine learning. For abnormality detection, even if other deep learning technologies are applied, other machine learning technologies are applied, or conventional rule-based image processing technologies are applied, each technology is combined. May be done. When the abnormality detection unit 103 detects an abnormality related to the subject, the abnormality detection unit 103 displays the abnormality information indicating the type of the detected abnormality and the position of the abnormality on the image, and the acquired radiation image. Send to 104. That is, the abnormality information is characterized by the type of abnormality and the coordinates on the optical image of the abnormality.

The display control unit 104 displays an object based on the abnormality information transmitted by the abnormality detection unit 103 and a radiographic image on the display unit 114 of the radiography control device 110.

That is, the image processing device 100 detects an abnormality from the image acquisition unit 102 that acquires the optical image obtained by optically photographing the subject and the radiation image obtained by radiographing the subject, and the optical image acquired by the image acquisition unit 102. An abnormality detection unit 103 is provided, and when an abnormality is detected from an optical image by the abnormality detection unit 103, a display control unit that displays an object based on the abnormality information regarding the detected abnormality together with a radiation image is provided.

<Shooting process>
Here, a procedure for performing anomaly detection processing on an optical image according to the flow of inspection by the radiography system shown in FIG. 1 and displaying anomaly information generated based on the detected anomaly together with the radiographic image. Will be described with reference to FIG.

FIG. 3 is a diagram showing an example of the display screen 200 displayed on the display unit 114. First, the operator operates the operation unit 111 of the radiography control device 110, inputs the patient information and the examination information of the subject, and starts the examination. The patient information includes a patient name, a patient ID, and the like, and the test information includes imaging information that defines the content of imaging to be performed on the subject. According to the inspection start operation, the radiography control device 110 displays the display screen 200 as shown in FIG. 3 on the display unit 114.

In the imaging information display area 205 on the display screen 200 displayed on the display unit 114, the patient information display unit 204 representing the patient information input before the start of the examination and the imaging method button 206 representing the examination information are displayed. The imaging method button 206 includes, for example, information on the imaging region and the radiography apparatus 110 used for imaging. According to the input inspection information, the control unit 112 of the radiography control device 110 radiates the radiography conditions (tube voltage, tube current, irradiation time, etc.) set corresponding to the radiography method button 206 (imaging method). It transmits to the generator 120 and controls the radiographing device 115 to prepare for taking a radiographic image.

When the preparation for taking a radiographic image is ready, the radiological imaging control device 110 transitions to an imagenable state. When the radiological imaging control device 110 changes to the imaging ready state, a "Ready message" indicating that the radiography imaging control device 110 is in the imaging ready state is displayed in the message area 203.

Subsequently, the operator confirms the method of photographing the radiographic image from the display screen 200 displayed on the display unit 114, sets the photographing, and positions the subject. When a series of shooting preparations is completed, the operator refers to the message area 203, confirms that shooting is possible, and then presses the irradiation switch. When the radiography irradiation switch is pressed, the radiography control device 110 irradiates the subject with radiation by the radiation generator 120, and causes the radiography device 115 to detect the radiation transmitted through the subject. As a result, a radiographic image is taken. Along with the radiography, the optical image acquisition device 130 also takes an optical image.

When the imaging of the radiographic image is completed, the control unit 112 of the radiographic imaging control device 110 acquires the radiographic image from the radiographic imaging device 110 and the optical image from the optical image acquisition device 130 and stores them in the storage unit 113. The radiography control device 110 performs image processing on the acquired radiographic image based on predetermined image processing conditions. Predetermined image processing conditions are predetermined according to the photographing method. When the image processing is completed, the radiography control device 110 displays the image-processed radiographic image in the radiographic image display area 201. The radiation image display area 201 is a part of the display content displayed on the display unit 114.

When the radiographic image and the optical image are acquired, the image processing apparatus 100 acquires the radiographic image and the optical image from the storage unit 113 by the image acquisition unit 102. Next, the abnormality detection unit 103 detects an abnormality in the acquired optical image. Here, the abnormality refers to trauma such as discoloration, swelling, and bleeding of the skin. When the abnormality detection unit 103 detects an abnormality from the optical image, the display control unit 104 causes the display unit 114 to display an object based on the abnormality information regarding the detected abnormality and the acquired radiation image. The operator can search the radiographic image for a foreign substance related to the abnormality based on the object based on the abnormality information. The display control unit 104 in the image processing device 100 causes the display unit 114 to display the object 209 for displaying the abnormality notification at the position of the radiation image corresponding to the location where the abnormality is determined in the optical image, for example. When the operator wants to change the contrast or the like of the radiation image, the operator operates a button such as the contrast or the brightness provided in the image processing setting area 207.

Similarly, when it is desired to change the cutout area of the output image, the adjustment button provided in the image operation area 202 is operated. For example, when a character string to be image diagnosis information is given, the annotation button 202d or the like is operated to display the character string as shown in the annotation 208 superimposed on the image. If the orientation of the image is not suitable for diagnosis, geometric transformation is performed using the rotation button 202a, the inversion button 202b, and the like. As described above, the operator can perform additional image editing on the image displayed in the radiation image display area 201.

The operator repeats the above-mentioned procedure to capture all the radiographic images according to the imaging method of the imaging information display area 205. When all the shooting is completed, the operator presses the inspection end button 208. This completes a series of inspections. When the series of inspections is completed, the radiography control device 110 adds the inspection information, the imaging conditions, and the like to the captured image as incidental information, and then outputs the captured image to, for example, the PACS 150. At the end of the inspection, the optical image acquired from the optical image acquisition device 130 may be discarded, or may be output to the PACS 150 and stored in association with the radiographic image.

<Flow of adding abnormality detection from optical image in image processing device 100>
FIG. 4 is a flow chart of the image processing device 100.

First, in step S401, the image acquisition unit 102 acquires a radiation image and an optical image from the storage unit 113.

Next, in step S402, the abnormality detection unit 103 performs abnormality detection on the optical image acquired by the image acquisition unit 102. Abnormalities are, for example, trauma such as discoloration, swelling, bleeding, bruises, and fractures of the skin, and detection of abnormalities uses, for example, CNN, which is one of the techniques based on machine learning.

CNN is classified as supervised learning of machine learning. Supervised learning can be used for anomaly detection tasks by learning CNN using teacher data that is a pair of learning data and correct answer data with correct answers to the learning data. .. Here, teacher data is configured as correct answer data corresponding to each abnormality, and learning is performed on the CNN.

When the optical image of the subject is input, the learned CNN outputs the type of abnormality and the coordinates of the abnormality. Although the process of detecting the detection of the abnormality detection unit 103 by machine learning has been described, the detection process may be performed by a conventional rule-based image processing technique.

In step S403, the display control unit 104 generates association information for associating the coordinates of the radiation image acquired by the image acquisition unit 102 with the optical image. The association information is generated based on, for example, irradiation field information. The irradiation field information is information indicating a range to be captured as a radiographic image. In the acquired optical image, the region corresponding to the irradiation field region is displayed brighter than the other regions due to the light indicating the irradiation field. Therefore, by trimming the irradiation field region of the optical image based on the brightness, the association information with the coordinate correspondence with the radiation image is acquired. The method of generating the association information is not determined only by the irradiation field information, but for example, the positional deviation of the optical image acquisition device 130 and the tube 121 for photographing radiation, and the position information of each device are used. May be good. In addition, enlargement / reduction, rotation, and angle correction processing may be performed according to both acquired images. In addition, features may be extracted from the image by a technique such as machine learning, and association may be performed based on the extracted features. The association information may be realized by any known technique as long as the coordinates of both images can be associated. When the coordinates are associated, it is possible to convert the specific coordinates of one image to the coordinates corresponding to the image to the other based on the association information.

In step S404, the display control unit 104 converts the object into a radiation image based on the abnormality information including the abnormality type and the coordinates of the abnormality detected by the abnormality detection unit 103 and the calculated coordinate association information between the images. Give. The object attached to the radiographic image may be in any form as long as the display unit 114 can display the coordinates corresponding to the detected abnormality on the radiographic image. That is, the display control unit 104 is characterized in that an object based on the abnormality information detected from the optical image based on the association information is displayed on the radiographic image. For example, the area corresponding to the coordinates of the detected abnormality may be highlighted on the radiographic image, and the detected abnormality information may be superimposed in the vicinity of the display area. When the abnormality information is given to the radiation image, the display control unit 104 transmits the radiation image and the abnormality information to the display unit 114. That is, the display control unit 104 is characterized in that the abnormality detected from the optical image is displayed on the coordinates of the radiation image corresponding to the coordinates of the optical image based on the detected abnormality information and the association information. To do.

The display unit 114 displays the information transmitted from the display control unit 104 in the radiation image display area 201. The operator can detect the foreign matter while reducing the load on the operator by searching for the foreign matter related to the abnormality detected by the abnormality detection unit 103 based on the coordinates corresponding to the abnormality.

(Second embodiment)
In the first embodiment, an example has been described in which the abnormality detection unit 103 detects the abnormality and the coordinates of the abnormality from the optical image, and the display control unit 104 reflects the abnormality information regarding the detected abnormality in the radiographic image. In the second embodiment, an example in which the operator inputs the abnormality information of the optical image will be described.

The description of the configuration, function, and operation similar to those of the above embodiment will be omitted, and the differences from the present embodiment will be mainly described.

FIG. 5 is a diagram showing an example of the configuration of the image processing device 500 in the radiography system according to the embodiment of the present invention. In the present embodiment, the image processing device 500 has an abnormality input unit 501 that operates an operation unit 103 such as a radiography control device 110 and receives an input of abnormality information of the operator. The operator operates the operation unit 103, and inputs a portion corresponding to the abnormality with respect to the optical image to the abnormality input unit 501 with a mouse or the like.

FIG. 6 is a flow chart of the image processing apparatus 500 of the present embodiment.

In step S602, the abnormality information such as the type of abnormality and the coordinates of the abnormality determined by the operator is input to the optical image acquired by the image acquisition unit 102. The abnormality information input by the operator is processed in the same manner as in the first embodiment, and is displayed in the radiation image display area 201 on the display unit 114. The image processing device 500 may have the abnormality detection unit 103 described in the first embodiment. By comparing the abnormality detected by the abnormality detection unit 103 with the abnormality input to the abnormality input unit 501 by the operator, the abnormality detection accuracy is improved. When the image processing device 100 has an abnormality detection unit 103 and an abnormality input unit 501, for example, the area where the abnormality detection unit 103 detects an abnormality is outside the abnormality coordinate area input by the abnormality input unit 501. May be. With this configuration, the abnormality detection unit 103 can detect the abnormality information input by the operator and the abnormality from the area where the operator's abnormality information is not input, and while reducing the load on the operator, the foreign matter can be overlooked. It can be suppressed.

FIG. 7 is a diagram showing an example of a GUI in the display unit 114 for the operator to input the abnormality information to the abnormality input unit 501 by, for example, the operation unit 111. When the radiation image capturing control device 110 acquires an optical image from the optical image acquisition device 130, the radiation image capturing control device 110 displays the abnormality input screen 600 on the display unit 114. The abnormal input screen 600 may be realized so that the abnormal input screen 600 and the above-mentioned display screen 200 can be displayed and switched on the same liquid crystal display, or another liquid crystal display is provided to provide the abnormal input screen 600 and the display screen 200. May be realized so as to be displayed at the same time.

The acquired optical image is displayed on the optical image display unit 601. The operator checks the displayed optical image, and if it determines that there is an abnormality, inputs the abnormality information. To input the abnormality information, the operator first specifies the coordinates of the abnormality. Specifically, the coordinate specification start button 610 is pressed to shift to the coordinate specification mode. After that, the coordinates of the abnormality on the optical image display unit 601 are specified via the operation unit 103 of a mouse, tablet, or the like. The coordinates corresponding to the designated abnormality are superimposed and displayed on the optical image display unit 601 as the abnormality location designation area 640. The position of the abnormal location designation area 640 may be adjusted by an operation such as dragging after the designation. If there is an error in the specified abnormal location, the input of the abnormal location may be deleted by pressing the abnormal location release button 611 with the abnormal location designation area 640 selected. After designating the abnormality location designation area 640 as the coordinates of the abnormality location, the abnormality type is input by the abnormality type selection unit 620 that selects the abnormality type. The abnormality type selection unit 620 is realized by a drop box for selecting an abnormality type from a preset list, a text box for inputting an arbitrary character string by an operator, and the like. After inputting the abnormal part on the optical image, the abnormality part designation completion button 630 is pressed. As a result, the input of the abnormal portion to the optical image is completed, and the abnormal input screen 600 ends. When the abnormality information is input by the operator, the image processing apparatus 500 performs coordinate conversion and information addition in the display control unit 104 based on the abnormality portion input on the abnormality input screen 600, and the display unit 114 displays the information. The radio image to be displayed and the abnormality information are generated.

According to the present embodiment, the displayability control unit 104 inputs abnormality information to the optical image, and the displayability control unit 104 receives the input via the abnormality input unit 501 by the image processing device 500. The coordinates of the radiographic image and the optical image are associated with each other, and the information is transmitted to the display unit 114 of the radiography control device 110. As a result, only the abnormalities that the operator deems necessary to be confirmed on the radiographic image are displayed on the radiological image, and even if an abnormality that does not affect the inspection is detected, the display is not performed, which reduces the burden on the operator. It becomes possible to do.

(Third Embodiment)
In the first and second embodiments, an example in which the image processing apparatus detects an abnormality or performs display control on the input abnormality information and causes the display unit 114 to display the abnormality has been described. In the third embodiment, a configuration in which the operator sets the type of foreign matter to be displayed on the display unit 114 will be described.

The description of the configuration, function, and operation similar to those of the above embodiment will be omitted, and the differences from the present embodiment will be mainly described.

FIG. 8 is a diagram showing an example of a GUI in which the operator of the display unit 114 sets the type of abnormality reflected in the radiation image. The abnormality type setting screen 700 on the display unit 114 includes an abnormality type 720 and a check box 710 for setting whether or not to reflect the abnormality information corresponding to each abnormality in the radiographic image. FIG. 8 shows an example when bleeding and fracture are effective as abnormalities reflected in the radiographic image and notification of bruises and rash is set as invalid. By pressing the confirmation button 730, the operator's settings are stored in the storage unit 113. The display control unit 104 in the image processing device 100 gives a notification based on the set information.

FIG. 9 is a data flow diagram of the abnormality notification display processing of the image processing device 100 in this embodiment. In step S900, the image acquisition unit 102 acquires the radiation image, the optical image, and the abnormality setting information to be reflected in the radiation image from the storage unit 113.

In step S901, based on the abnormality setting information acquired by the image acquisition unit 102, the abnormality detection unit 103 determines whether or not the abnormality detected by the abnormality detection unit 103 is an abnormality to be reflected in the radiographic image. Here, if the abnormality detected by the abnormality detection unit 103 is the set abnormality, the subsequent processing is executed for the corresponding abnormality, and if there is no abnormality corresponding to the set abnormality, the processing is terminated. .. For example, as shown in FIG. 8, the case where bleeding and fracture are effective and the notification of bruises and rash is invalid is described. In this case, it is determined whether or not the abnormality detected by the abnormality detection unit 103 includes an abnormality corresponding to bleeding or fracture. When there is bleeding or fracture in the detected abnormality, the display control unit 104 generates a radiation display image to be displayed on the display unit 114 based on the radiation image and the abnormality information determined to match the setting information. On the other hand, when the abnormality detected by the abnormality detection unit 103 includes an abnormality corresponding to a bruise or a rash, the radiation display image to be displayed on the display unit 114 is not generated for the corresponding abnormality. That is, the image processing device 100 acquires the abnormality setting information to be reflected in the radiation image set by the user, and among the abnormalities detected by the abnormality detection unit 103, the abnormality information satisfying the setting information is displayed together with the radiation image. It is characterized by.

According to the present embodiment, the abnormal portion displayed on the radiation image display area 201 is limited to the abnormality desired by the operator, so that only the necessary abnormality is displayed on the radiation image. Even if an abnormality that does not affect the inspection is detected, the display is not performed, so that the load on the operator can be reduced.

(Modification example)
In the first to third embodiments described above, the image processing device 100 that performs image processing has been described as an independent device. However, as shown in FIG. 10, the configuration of the image processing device 100 may be a part of the configuration in the radiography control device as the image processing unit 1001. Further, the image processing device 100 and the image processing unit 1001 may be part of the configuration in another device or may be realized on the cloud.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 Image processing device 102 Image acquisition unit 103 Abnormality detection unit 104 Display control unit 113 Storage unit 114 Display unit

Claims (11)

An abnormality detection that detects an abnormality related to the subject from an image acquisition unit that acquires an optical image obtained by optically photographing the subject and a radiation image obtained by radiographing the subject, and the optical image acquired by the image acquisition unit. An image processing device including a unit and a display control unit that displays an object based on the detected abnormality information regarding the abnormality together with a radiation image when an abnormality is detected from the optical image by the abnormality detection unit. The image processing apparatus according to claim 1, wherein the abnormality detection unit detects the abnormality based on machine learning. The image processing apparatus according to claim 1 or 2, wherein the abnormality information includes the type of the abnormality and the coordinates of the abnormality on the optical image. The image processing apparatus according to claim 3, wherein the display control unit generates associative information for associating the coordinates on the optical image with the coordinates on the radiographic image. The image processing apparatus according to claim 4, wherein the display control unit displays abnormal information detected from an optical image based on the association information on a radiographic image. The display control unit is characterized in that the abnormality detected from the optical image is displayed on the coordinates of the radiation image corresponding to the coordinates of the optical image based on the abnormality information and the association information. The image processing apparatus according to 5. The image processing device according to any one of claims 1 to 6, wherein the image processing device has an abnormality input unit that receives input of abnormality information about a foreign substance found by the operator from the operator. The image processing device acquires setting information of an abnormality to be reflected in the radiation image set by the operator, and among the abnormalities detected by the abnormality detection unit, the abnormality satisfying the setting information is displayed together with the radiation image. The image processing apparatus according to any one of claims 1 to 7, wherein the image processing apparatus is provided with abnormal information. The abnormality detection unit according to any one of claims 1 to 8, wherein the abnormality detection unit detects at least one of the discoloration, swelling, bleeding, bruising, and fracture of the skin of the subject as the abnormality. Image processing equipment. An image acquisition step for acquiring an optical image obtained by optically photographing a subject and a radiation image obtained by radiating the subject, an abnormality detection step for detecting an abnormality from the optical image acquired by the image acquisition step, and an abnormality detection step. An image processing method including a display control step of displaying an object based on the detected abnormality information together with a radiation image when the abnormality is detected from the optical image. A program for executing the image processing method according to claim 10 on a computer.

Images