JP2023029106A - Image processing device, image processing method, and program - Google Patents

Abstract

To classify a large amount of image data to use for diagnosis of a subject eye and make it easy for a user to determine whether deletion of the image data is easy or not.SOLUTION: An image processing device comprises: an acquisition unit which acquires one or more pieces of image data of a subject eye; a classification unit which classifies the one or more pieces of image data into one of a plurality of classes indicating the possibility of deletion; and a display unit which associates a result of classification by the classification unit and displays deletion propriety of each of the one or more pieces of image data so as to be acceptable.SELECTED DRAWING: Figure 4

Description

The present invention relates to an image processing device, an image processing method, and a program.

2. Description of the Related Art Conventionally, there is known a diagnostic apparatus that analyzes an image of an eye to be inspected and makes a diagnosis. Such a diagnostic device is used for diagnosing eye conditions and for early detection of abnormalities, for example. In a diagnostic apparatus, image data is captured and used, but the captured image data includes image data that is not suitable for use at the time of diagnosis. Image data that is not suitable for use in diagnosis includes, for example, image data of low quality and content that overlaps.

For example, Japanese Patent Application Laid-Open No. 2002-200001 discloses a configuration in which an exceptional frame is determined from a plurality of frames that constitute a moving image captured by an ophthalmic imaging apparatus, and image processing is applied to frames other than the exceptional frame. Such a configuration enables robust image analysis even when an exceptional frame that is not suitable for measurement is included.

JP 2017-56245 A

A large amount of image data captured by a diagnostic apparatus is expected. By sorting out such a large amount of image data, it is required to perform diagnosis more efficiently and to save only necessary image data.

Japanese Patent Laid-Open No. 2004-200000 describes that, among a plurality of frames that form a moving image, frames that are regarded as exceptional frames are excluded from processing targets. However, multiple classifications including deletion are not considered for multiple image data.

SUMMARY OF THE INVENTION In view of the above problems, an object of the present invention is to classify a large amount of image data used for diagnosis of an eye to be examined, and to enable a user to easily decide whether or not to delete image data.

In order to solve the above problems, one embodiment of the present invention has the following configuration. That is, an image processing device,
an acquisition unit that acquires one or more image data of an eye to be inspected;
a classification unit that classifies the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
a display unit that associates the results of classification by the classification unit and displays whether or not deletion of each of the one or more image data can be accepted;
have

Moreover, another form of this invention has the following structures. That is, an image processing method comprising:
an acquisition step of acquiring one or more image data of an eye to be examined;
a classification step of classifying the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
a display control step of associating the results of the classification by the classification step and displaying on a display unit in such a manner that whether or not deletion of each of the one or more image data can be accepted;
have

Moreover, another form of this invention has the following structures. That is, the program
to the computer,
an acquisition step of acquiring one or more image data of the eye to be examined;
a classification step of classifying the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
A display control step of associating the results of classification by the classification step and displaying on a display unit in a manner capable of accepting deletion of each of the one or more image data;
to run.

According to the present invention, a large amount of image data used for diagnosis of an eye to be examined can be classified, and the user can easily decide whether or not to delete image data.

1 is a block diagram showing an example of a system configuration according to one embodiment of the present invention; FIG. 1 is a block diagram showing an example of functional configuration according to a first embodiment of the present invention; FIG. 4 is a flowchart of classification processing according to the first embodiment of the present invention; FIG. 4 is a diagram showing an example of screen configuration according to an embodiment of the present invention; FIG. 4 is a diagram showing an example of screen configuration according to an embodiment of the present invention; FIG. 5 is a block diagram showing an example of functional configuration according to the second embodiment of the present invention; FIG. 10 is a table showing an example of a DB configuration according to the second embodiment of the present invention; FIG. 9 is a flowchart of processing according to the second embodiment of the present invention;

Exemplary embodiments of an image processing apparatus, an image processing method, and a program according to the present invention will be described in detail with reference to the drawings. It should be noted that the descriptions of the documents cited in this specification and any known techniques can be incorporated into the following embodiments. In addition, the embodiment described below is one embodiment for describing the present invention, and is not intended to be interpreted as limiting the present invention. Not all configurations are essential configurations for solving the problems of the present invention. Moreover, in each drawing, the same component is indicated by the same reference number to indicate the correspondence.

<First Embodiment>
[System configuration]
FIG. 1 is a diagram showing a schematic configuration of an ophthalmologic system according to one embodiment of the present invention. The ophthalmologic system includes an imaging device 100 and an information processing device 200, which are communicably connected via a network 300. FIG.

The imaging device 100 is an imaging device for capturing an image of an eye to be examined, and is configured by, for example, an OCT (Optical Coherence Tomography) device and a camera. A tomographic image of the subject's eye is obtained by performing OCT measurement on the subject's eye using an OCT apparatus. In addition, a front image of the subject's eye can be captured by a camera. Note that the type of image captured by the imaging device 100 is not particularly limited, and a plurality of types of images may be captured according to the user's operation or the like. Hereinafter, various images captured and acquired by the imaging device 100 are also collectively referred to as image data.

The imaging apparatus 100 includes an imaging unit 101 , an image processing unit 102 , an imaging control unit 103 , a storage unit 104 and a communication unit 105 . The imaging unit 101 photographs an eye to be inspected based on a predetermined imaging principle. Here, it is assumed that a camera and an OCT apparatus for capturing a front image of the subject's eye are used. In the case of an OCT apparatus, an OCT image is acquired by applying an OCT scan to the subject's eye upon receiving a control instruction from the imaging control unit 103 .

The OCT technique is typically Fourier-domain OCT, and may be either spectral-domain OCT or swept-source OCT. The swept source OCT splits the light from the wavelength tunable light source into measurement light and reference light, and superimposes the return light of the measurement light projected onto the eye to be inspected on the reference light to generate interference light. Then, the interference light is detected by a photodetector, and the detection data (interference signal data) collected according to the wavelength sweep and measurement light scanning is subjected to Fourier transform or the like to form reflection intensity profile data. On the other hand, in spectral domain OCT, light from a low coherence light source (broadband light source) is split into measurement light and reference light, and the return light of the measurement light projected onto the subject's eye is superimposed on the reference light to generate interference light. do. Then, the spectrum distribution of this interference light is detected by a spectroscope, and the data detected by the spectroscope (interference signal data) is subjected to Fourier transform or the like to form reflection intensity profile data. That is, the swept-source OCT is an OCT technique that acquires the spectral distribution by time division, and the spectral domain OCT is an OCT technique that acquires the spectral distribution by spatial division.

The image processing unit 102 performs image processing on various image data acquired by the imaging unit 101 . The content of the image processing performed here is not particularly limited, and may be defined in advance according to the type of image data to be acquired, for example. The imaging control unit 103 controls the imaging operation of the imaging unit 101 . The image processing unit 102 and the imaging control unit 103 may each be configured by a dedicated circuit, or a processor such as a CPU (Central Processing Unit) (not shown) included in the imaging apparatus 100 may execute a program stored in the storage unit 104. It may be realized by reading and executing.

The storage unit 104 includes a RAM (Random Access Memory), ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various image data captured by the imaging unit 101. Remember. The storage unit 104 may also store shooting settings and various programs used by the image processing unit 102 and the shooting control unit 103 . The communication unit 105 manages communication with an external device (for example, the information processing device 200 ) via the network 300 . The communication method here may be wired or wireless, and the communication standard is not particularly limited.

The information processing apparatus 200 includes a processing unit 201 , a storage unit 202 , a communication unit 203 and a UI (User Interface) unit 204 . The processing unit 201 controls and processes the entire information processing apparatus 200, and is configured by a processor such as a CPU. The storage unit 202 includes a RAM, a ROM, an HDD, an SSD, and the like, and stores various settings used in processing by the processing unit 201, various processing programs, or various image data. A communication unit 203 manages communication with an external device (for example, the imaging device 100 ) via the network 300 . The communication method here may be wired or wireless, and the communication standard is not particularly limited. The UI unit 204 is a user interface used by the user of the information processing apparatus 200, and includes, for example, an operation unit for inputting user instructions and a display unit for displaying screens and the like. A mouse, a keyboard, or the like, for example, may be used as the operation unit. The display unit may be, for example, a liquid crystal display (LCD) or an organic light-emitting diode (OLED) display.

[Function configuration]
FIG. 2 is a block diagram showing an example of the functional configuration of the information processing device 200 according to this embodiment. The information processing apparatus 200 includes an image acquisition section 211 , a classification condition management section 212 , a classification condition setting section 213 , an image classification section 214 , a screen generation section 215 , an instruction reception section 216 and an image management section 217 . Note that the block configuration of the functions realized by the information processing apparatus 200 is an example, and may be further divided in detail corresponding to each process described later. In this embodiment, each function may be implemented by the processing unit 201 of the information processing apparatus 200 reading and executing a program stored in the storage unit 202 . Note that part of the functions shown in FIG. 2 may be realized by a server (not shown) provided on the imaging device 100 side or on the network 300 .

The image acquisition unit 211 acquires various image data from an external device (here, the imaging device 100). When acquiring image data, the image acquiring unit 211 may periodically inquire of the imaging device 100 about newly captured image data, or may acquire image data automatically transmitted from the external device each time. may be obtained.

The classification condition management unit 212 manages classification conditions for classifying the image data acquired by the image acquisition unit 211 . This embodiment will be described using an example of two classifications of deletion/not deletion for each image data. The classification condition setting section 213 sets various classification conditions managed by the classification condition management section 212 . Examples of classification conditions will be described later.

The image classification unit 214 classifies each image data acquired by the image acquisition unit 211 using classification conditions managed by the classification condition management unit 212 . In this embodiment, each image data is classified as either deleted or not deleted based on the classification conditions managed by the classification condition management unit 212 . The classification here may be managed using flags or the like, for example. The screen generation unit 215 generates a screen for displaying each image data classified by the image classification unit 214 together with the classification result. A configuration example of the screen here will be described later with reference to FIGS. 4 and 5. FIG. Instruction accepting portion 216 accepts an instruction for image data input via the screen generated by screen generating portion 215 . Specifically, the instruction receiving unit 216 receives a user instruction as to whether or not to actually delete each image data. The image management unit 217 manages the image data acquired by the image acquisition unit 211 and deletes the image data based on the instruction received by the instruction reception unit 216 . The image management unit 217 may manage the classification results in association with each image data.

[Classification condition]
Image data classification conditions according to the present embodiment will be described. Classification conditions are conditions for classifying images according to the characteristics of image data. The classification conditions may be set based on the user's instructions, or arbitrary conditions may be set by the system. Also, a plurality of classification conditions may be set. Further, for the classification conditions that can be set by the user, the range of parameters that can be set by the user and the options that can be selected may be defined.

As the classification condition, for example, threshold values for the following items may be set, and the classification may be determined according to the value.
- Brightness of image data - Size of image data - Resolution of image data - Dynamic range of image data - Shooting date and time - Data format (extension, etc.)

Further, the classification conditions may be set based on the feature points and objects of the image data extracted by analyzing the image data. For example, classification may be performed according to the following feature points and object detection results. In order to apply the classification conditions described below, the imaging device 100 and the information processing device 200 may perform image processing as appropriate.
・Shape and size of papilla in eye to be inspected ・Shape and size of blood vessel in eye to be inspected ・Edge indicating area of eye to be inspected ・Similarity of continuous image data

[Processing flow]
FIG. 3 is a flowchart of processing of the information processing apparatus 200 according to this embodiment. For example, the processing unit 201 provided in the information processing apparatus 200 reads and executes various programs stored in the storage unit 202, thereby operating as the functional units shown in FIG. Here, for the sake of simplification of explanation, the main body of processing is shown as the information processing apparatus 200, but each function as shown in FIG. 2 cooperates to realize the processing.

In step S<b>301 , the information processing apparatus 200 acquires each image data captured by the imaging apparatus 100 . The image data may be all unprocessed image data, or the user may designate image data to be acquired.

In step S302, the information processing apparatus 200 refers to managed classification conditions. The classification conditions referred to here may be switched according to, for example, the type of image data, or all set classification conditions may be referred to.

In step S303, the information processing apparatus 200 performs classification processing on each image data acquired in step S301 using the classification condition referred to in step S302. At this time, various image processing may be performed according to the contents of the classification conditions. Examples of image processing include tone conversion processing, image enhancement processing, threshold processing, contrast conversion processing, binarization processing, edge detection processing, image averaging processing, image smoothing processing, filtering processing, and region extraction processing. , alignment processing, and the like may be performed. Note that image processing is not limited to the above, and other processing may be performed in consideration of subsequent processing. As described above, in this embodiment, each image data is classified as either deleted or not deleted.

In step S304, the information processing apparatus 200 generates a display screen according to the result of the classification processing in step S303.

In step S<b>305 , the information processing apparatus 200 causes the display unit of the UI unit 204 to display the display screen generated in step S<b>304 . The display here may be displayed based on an instruction from the user of the information processing device 200 .

In step S306, the information processing apparatus 200 accepts a user instruction of image data to be actually deleted or not to be deleted (that is, to be saved) via the display screen generated in step S305.

In step S307, the information processing apparatus 200 manages each image data based on the user instruction received in step S306. More specifically, the information processing apparatus 200 deletes image data instructed to be deleted by a user instruction, and stores image data instructed not to be deleted. Note that the storage location of the image data may be a preset location or a location individually designated by the user. Then, this processing flow ends.

[Display screen]
A configuration example of the display screen generated in step S304 of FIG. 3 will be described with reference to FIGS. 4 and 5. FIG. FIG. 4 shows an example of the screen configuration when the image data classified as "delete" is focused on and displayed, and FIG. 5 shows the image data classified as "not deleted". An example of the screen configuration when

A display screen 400 shown in FIG. 4 is roughly divided into four areas 401 , 402 , 403 and 404 . A region 401 displays one image data 405 of interest. Information 406 of image data 405 of interest is shown in area 401 . Here, since the image data 405 is classified as “delete”, an icon 407 indicating this is superimposed on the image data 405 and displayed. Note that the display method of the icon 407 is an example, and other character strings or graphics may be used. By specifying the presence or absence of the icon 407 indicating "delete", the user can instruct whether or not to delete the image data of interest. The designation here may be given via a button (not shown) (for example, a key on hardware included in the UI unit 204). In this embodiment, the icon 407 is automatically displayed when classified as "delete", and the icon 407 is not displayed when classified as "not deleted". This eliminates the need for the user to individually input whether or not deletion is possible. Note that the automatic display of the icon 407 may be reversed.

An area 402 displays a list of a plurality of image data. In the list, a frame 408 is shown in the thumbnail corresponding to the image data 405 of interest in the area 401 . Also, an icon 409 corresponding to the icon 407 is displayed in association with the thumbnail of each image data included in the list.

A region 403 shows information about a plurality of image data displayed in the region 402 . Here, identification information (here, “ID5”) assigned to a plurality of image data groups, imaging mode information (here, “non-mydriatic fundus”), and imaging time are displayed. Note that items of information to be displayed in the area 403 are not particularly limited, and may be set by the user or may be switched according to the shooting mode or the like.

A region 404 shows information relating to classification of a plurality of image data. Here, the number of pieces of image data to be saved out of the number of pieces of image data is shown in association with the photographing mode information (here, “non-mydriatic fundus”) and the photographing time. That is, among the plurality of image data, the number of image data classified as "not deleted" is identifiable. In the example of FIG. (that is, do not delete). The items of information to be displayed in the area 404 are not particularly limited, and may be set by the user.

A display screen 500 shown in FIG. 5 has the same basic configuration as the display screen 400 shown in FIG. Here, the image data 505 classified as "do not delete" is focused on and displayed. Unlike the example shown in FIG. 4, the icon classified as "delete" is not displayed on the image data 505. FIG. The configurations of information 506 and cursor 507 are also the same as in FIG. Although not shown in the display screens shown in FIGS. 4 and 5, for example, by pressing a separately provided save button, the image data classified as deleted is deleted, and the image data classified as not deleted is deleted. is stored in the storage unit 202 .

As described above, according to the present embodiment, a large amount of image data used for diagnosis of an eye to be examined can be automatically classified, and the user can easily decide whether or not to delete image data. Therefore, it is possible for the user to suppress the task of individually confirming a large amount of image data, and it is possible to improve the work efficiency at the time of diagnosis.

<Second embodiment>
Next, a second embodiment of the invention will be described. In the above-described first embodiment, a mode in which a plurality of image data are classified using preset classification conditions has been described. In this embodiment, each image data is classified using a trained model obtained by performing learning processing using learning data. The description of the configuration that overlaps with that of the first embodiment will be omitted, and the description will focus on the differences.

In the following description, "learning" or "machine learning" refers to generating a "trained model" by performing learning using learning data and an arbitrary learning algorithm. A trained model is timely updated as learning progresses using a plurality of learning data, and even if the input is the same, the output may change. Therefore, the trained model does not limit the state at any point in time. Here, a model used in learning is described as a "learning model", and a learning model that has undergone a certain degree of learning is described as a "learned model". A specific example of the "learning data" will be described later, but its configuration may be changed according to the learning algorithm to be used. The learning data may include teacher data used for learning itself, verification data used for verifying the trained model, and test data used for testing the trained model. In the following description, the term "learning data" is used to refer to data related to learning in a comprehensive manner, and the term "teaching data" is used to refer to data for learning itself. It is not intended to clearly classify teacher data, validation data, and test data contained in learning data. can be.

[Function configuration]
FIG. 6 is a block diagram showing an example of the functional configuration of the information processing device 200 according to this embodiment. The information processing apparatus 200 includes an image acquisition unit 601, a learning processing unit 602, a trained model management unit 603, an image classification unit 604, a screen generation unit 605, an instruction reception unit 606, an image management unit 607, and a history management unit 608. consists of Note that the block configuration of the functions realized by the information processing apparatus 200 is an example, and may be further divided in detail corresponding to each process described later. In this embodiment, each function may be implemented by the processing unit 201 of the information processing apparatus 200 reading and executing a program stored in the storage unit 202 . Note that part of the functions shown in FIG. 2 may be realized by a server (not shown) provided on the imaging device 100 side or on the network 300 . In particular, since the learning process has a high processing load, it may be executed by a server provided for learning.

The image acquisition unit 601 acquires various image data from an external device (here, the imaging device 100). When acquiring image data, the image acquiring unit 601 may periodically inquire of the imaging apparatus 100 about newly captured image data, or may acquire image data automatically transmitted from the external apparatus each time. may be obtained.

A learning processing unit 602 performs learning processing for generating a trained model for classifying the image data acquired by the image acquisition unit 601 . A learning program for executing deep learning using predetermined learning data is held in the storage unit 202 . By performing the learning process, it is possible to update a trained model that has already been generated by performing the learning process. In the learning data used in this embodiment, image data is associated with classification information indicating classification results of the image data. This embodiment will be described using two classifications of deletion/not deletion as classification information. A trained model according to the present embodiment is a trained model generated by performing learning using learning data consisting of pairs of image data and classification information. By inputting the image data into the trained model, the classification information (here, delete or not delete) for the image data is output as the output. In the learning process according to the present embodiment, the classification information as the output is compared with the classification information indicated by the learning data, the parameters of the trained model are adjusted based on the difference, and the learning process is repeated. , the trained model is updated. For the adjustment here, for example, the back propagation method or the like may be used.

For the learning process in this embodiment, for example, a classification algorithm that uses a convolutional neural network (CNN) can be applied. A convolutional neural network includes multiple convolutional layers, multiple pooling layers, and multiple fully connected layers. Note that the configuration of the convolutional neural network is not limited to the above, and the number of layers and configuration may be different. Also, other learning algorithms may be used.

A trained model management unit 603 manages a trained model obtained as a result of learning processing by the learning processing unit 602 . The trained model management unit 603 may be configured to manage only the latest trained model, or may be configured to add and manage a new trained model each time it is updated. good. Also, the learned model management unit 603 may be configured to provide a learning model that serves as a base when the learning processing unit 602 performs learning processing.

The image classification unit 604 classifies each image data acquired by the image acquisition unit 601 by applying a trained model managed by the trained model management unit 603 as input. In this embodiment, each image data is classified into one of a plurality of classifications, and here, it is classified as either deleted or not deleted.

A screen generation unit 605 generates a screen for displaying each image data classified by the image classification unit 604 together with the classification result. The configuration example of the screen here may be the same as the configuration examples of FIGS. 4 and 5 described in the first embodiment. An instruction accepting unit 606 accepts an instruction for image data input via the screen generated by the screen generating unit 605 . Specifically, the instruction receiving unit 606 receives a user instruction as to whether or not to actually delete each image data. An image management unit 607 manages the image data acquired by the image acquisition unit 601 and deletes the image data based on the instruction received by the instruction reception unit 606 . The image management unit 607 may manage classification results in association with each image data.

A history management unit 608 manages the management status of image data by the image management unit 607 and the status of learning processing using image data by the learning processing unit 602 as history information. In this embodiment, after classifying the image data, the classification result is shown to the user. Furthermore, it receives instructions on how to actually handle each image data, that is, whether to delete or save it. These pieces of information can be used as learning data for learning processing in the learning processing unit 602. By updating the trained model using this learning data, a trained model capable of classifying more suitable for the user is generated. It becomes possible to

FIG. 7 shows an example of a history information database (DB) configuration managed by the history management unit 608 according to this embodiment. The history information DB 700 according to this embodiment includes items of identification information 701 , classification 702 , user instruction 703 , learning processing use 704 , and management status 705 . The identification information 701 is identification information for uniquely identifying image data. This identification information may be set on the information processing device 200 side, or may be set on the imaging device 100 side. The specification of the identification information of the image data (number of digits, character string used, etc.) is not particularly limited either. A classification 702 indicates the classification as a result of processing by the image classification unit 604 . In this example, either “delete” (classified as deleting image data) or “save” (classified as not deleting image data) is indicated.

A user instruction 703 indicates handling of image data based on a user instruction received by the instruction receiving unit 606 via the display screen. In this example, either “delete” (instruct to delete image data) or “save” (instruct not to delete image data) is displayed. The learning processing use 704 indicates whether or not learning processing is performed using pairs of image data and user instructions (that is, actual handling of image data based on the user indicated by the user instructions 703) as learning data. In this example, either "used" (learning using the learning data has been performed) or "unused" (learning using the learning data has not been performed) is indicated. A management state 705 indicates the management state of the image data. In this example, either "deleted" or "not deleted" is indicated. In this embodiment, when "delete" is designated by the user's instruction, the target image data is deleted after learning processing is performed on the image data. It should be noted that when a certain number of image data are held without being deleted due to waiting for learning processing, the image data may be deleted in the order of oldest within a certain upper limit.

As described above, in the present embodiment, the image data is divided into two categories, ie, deleted/not deleted. configuration can be changed.

[Processing flow]
FIG. 8 is a flowchart of processing of the information processing apparatus 200 according to this embodiment. For example, the processing unit 201 provided in the information processing apparatus 200 reads and executes various programs stored in the storage unit 202, thereby operating as each functional unit shown in FIG. Here, for the sake of simplification of explanation, the information processing apparatus 200 is shown as the subject of the processing, but each function as shown in FIG. 6 cooperates to realize the processing.

In step S<b>801 , the information processing apparatus 200 acquires each image data captured by the imaging apparatus 100 . The image data may be all unprocessed image data, or the user may designate image data to be acquired.

In step S802, the information processing apparatus 200 acquires the managed trained model. In the present embodiment, an example in which the information processing apparatus 200 performs the learning process in a timely manner to update the learned model has been described. may be configured to request a trained model of .

In step S803, the information processing apparatus 200 receives each image data acquired in step S801 as an input and performs classification processing by applying the trained model acquired in step S802. At this time, various image processing may be performed on the image data. Examples of image processing include tone conversion processing, image enhancement processing, threshold processing, contrast conversion processing, binarization processing, edge detection processing, image averaging processing, image smoothing processing, filtering processing, and region extraction processing. , alignment processing, and the like may be performed. Note that image processing is not limited to the above, and other processing may be performed in consideration of subsequent processing. As described above, in the present embodiment, each piece of image data is classified as either deleted or not deleted by the learned model.

In step S804, the information processing apparatus 200 generates a display screen according to the result of the classification processing in step S803. The display screen generated here may have the same configuration as that shown in FIGS. 4 and 5 shown in the first embodiment.

In step S805, the information processing apparatus 200 causes the display unit of the UI unit 204 to display the display screen generated in step S804. The display here may be displayed based on an instruction from the user of the information processing device 200 .

In step S806, the information processing apparatus 200 accepts a user instruction of image data to be actually deleted or not to be deleted (that is, to be saved) via the display screen generated in step S805.

In step S807, the information processing apparatus 200 manages each image data based on the user instruction received in step S806. More specifically, the information processing apparatus 200 saves the image data instructed not to be deleted. Image data instructed to be deleted by a user instruction is managed so as to be deleted in accordance with the results of subsequent learning processing. Note that the storage location of the image data may be a preset location or a location individually designated by the user.

In step S808, the information processing apparatus 200 updates the history information DB 700 based on the processing result of step S807. More specifically, the information processing apparatus 200 registers image data in the history information DB 700, and updates the values of items managed in the history information DB 700 according to processing results.

In step S809, the information processing apparatus 200 performs learning processing using the acquired image data as learning data, and determines whether or not to update the learned model. In this determination, for example, it may be determined to update based on a user instruction, or it may be determined to update when a certain amount of image data is accumulated as learning data. Alternatively, it may be determined to update according to a predetermined schedule. If it is determined to update the trained model (YES in step S809), the processing of information processing apparatus 200 proceeds to step S810. On the other hand, if it is determined not to update the trained model (NO in step S809), the processing of the information processing apparatus 200 ends.

In step S810, the information processing apparatus 200 performs learning processing using the accumulated learning data. When learning processing is performed by an external server (not shown) or the like, the information processing apparatus 200 provides learning data accumulated in the external server, executes learning processing based on the learning data, and performs the latest learned processing. It may be configured to require a model. Furthermore, the information processing apparatus 200 deletes the image data classified as deleted after the learning process is completed. Furthermore, the information processing apparatus 200 updates the contents of the history information DB 700 according to the result of the learning process. Then, this processing flow ends.

As described above, according to the present embodiment, by using a trained model instead of a classification condition, a large amount of image data used for diagnosis of an eye to be examined can be classified, and the user can easily decide whether or not to delete image data. . In addition, individual users do not need to make settings, and it is possible to easily classify image data according to user preferences.

<Other embodiments>
In the above embodiment, an example of dividing image data into two categories of deletion/not deletion has been shown. However, more classifications may be used. For example, three or more classifications may be used in descending order of probability of deletion. In addition, it may be divided into three or more classifications according to which of the plurality of classification conditions applies. Further, the number of classifications may be changed according to the type of target image data (for example, the imaged part or characteristics of the image) or the application of the image data (for example, the type of disease to be diagnosed).

In the present invention, a program or application for realizing the functions of one or more embodiments described above is supplied to a system or device using a network or a storage medium, and one or more processors in the computer of the system or device can also be implemented by reading and executing the program.

In this specification, "processor" includes, for example, CPU (Central Processing Unit), GPU (Graphics Processing Unit), ASIC (Application Specific Integrated Circuit), programmable logic device (e.g., SPLD (Simple Programmable Logic Device), CPLD Device (Complex Programmable Logic Device), FPGA (Field Programmable Gate Array)) or other general-purpose or dedicated circuit.

Also provided is a program for causing a computer to execute not only the method shown as an example above but also the control method for controlling the above apparatus. Such a program can be stored in any computer-readable storage medium. Examples of such storage media include semiconductor memories, optical disks, magneto-optical disks (CD-ROM/DVD-RAM/DVD-ROM/MO, etc.), magnetic storage media (hard disks/floppy (registered trademark) disks/ZIP, etc.), and the like. can be used. It is also possible to transmit and receive this program through a network such as the Internet or LAN. Similarly, the configuration may be such that the operation of the device is controlled based on the result of processing executed by a processing unit arranged on a network or on the basis of a control instruction.

As described above, the present invention is not limited to the above-described embodiments, but may be modified arbitrarily by those skilled in the art by combining each configuration of the embodiments with each other or based on the description of the specification and well-known technology. , adaptations, omissions, and additions are also contemplated by the present invention and fall within the scope of protection sought.

As described above, this specification discloses the following matters.
(1) an acquisition unit (for example, an image acquisition unit 211) that acquires one or more image data of an eye to be examined;
a classification unit (e.g., image classification unit 214) that classifies the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
a display unit (for example, a screen generation unit 215) that associates the results of classification by the classification unit and displays in a manner capable of accepting deletion of each of the one or more image data;
An image processing device comprising:
According to this configuration, a large amount of image data used for diagnosis of an eye to be examined is classified, and the user can easily decide whether or not to delete image data.

(2) The image processing apparatus according to (1), further comprising a setting unit (for example, the classification condition setting unit 213) that receives setting of classification conditions corresponding to each of the plurality of classifications from a user.
According to this configuration, the user can set the classification condition for classifying the image data.

(3) The image processing apparatus according to (2), wherein the classification condition is a threshold for luminance.
With this configuration, it is possible to classify the image data based on the brightness.

(4) The image processing apparatus according to (2) or (3), wherein the classification condition is the presence or absence of a predetermined object.
According to this configuration, it is possible to classify the image data based on the presence or absence of a predetermined object included in the image data.

(5) The classification unit converts the image data to the trained model using a trained model in which learning processing is performed using a combination of image data and one of the plurality of classifications as learning data. The image processing apparatus according to (1), characterized in that the classification obtained upon input is determined as the classification of the image data.
With this configuration, it is possible to easily classify image data using a trained model.

(6) The learned model is updated by performing the learning process using, as the learning data, the image data for which deletion permission/non-permission is accepted by the display unit (5) The image processing device according to .
According to this configuration, it is possible for the user to confirm the result of classification, and to update the learned model by performing further learning processing using the information specified as the result. This makes it possible to classify image data using a trained model that is more suitable for the user.

(7) The display unit displays a list of each of the one or more image data with an icon corresponding to the result of classification by the classification unit. The image processing device according to any one of .
According to this configuration, it is possible to improve the visibility of the classification result by the user.

(8) The image processing apparatus according to any one of (1) to (7), wherein the plurality of classifications are performed by at least two classifications of deletion and non-deletion.
According to this configuration, the user can easily grasp the image data that can be deleted.

(9) The image processing apparatus according to any one of (1) to (8), further comprising an image management unit that controls storage processing of image data in a storage unit based on whether or not deletion is permitted by the display unit. 1. The image processing device according to 1.
According to this configuration, the image data can be appropriately managed based on the user's instruction after confirming the classification result by the user.

(10) an acquisition step of acquiring one or more image data of the subject eye (for example, step S301);
a classification step of classifying the one or more image data into one of a plurality of classifications indicating the possibility of deletion (for example, steps S302 and S303);
a display control step (e.g., steps S304 and 305) of associating the results of classification by the classification step and displaying on a display unit an acceptability of deletion of each of the one or more image data;
An image processing method characterized by comprising:
According to this configuration, a large amount of image data used for diagnosis of an eye to be examined is classified, and the user can easily decide whether or not to delete image data.

(11) In a computer (for example, information processing device 200),
an acquisition step of acquiring one or more image data of the subject eye (for example, step S301);
a classification step of classifying the one or more image data into one of a plurality of classifications indicating the possibility of deletion (for example, steps S302 and S303);
a display control step (for example, steps S304 and S305) of causing a display unit to display, in association with the result of the classification by the classification step, the possibility of deletion of each of the one or more image data so as to be acceptable;
program to run the
According to this configuration, a large amount of image data used for diagnosis of an eye to be examined is classified, and the user can easily decide whether or not to delete image data.

DESCRIPTION OF SYMBOLS 100... Imaging device 101... Imaging part 102... Image processing part 103... Shooting control part 104... Storage part 105... Communication part 200... Information processing apparatus 201... Processing part 202... Storage part 203... Communication part 204... UI (User Interface) Unit 211 Image acquisition unit 212 Classification condition management unit 213 Classification condition setting unit 214 Image classification unit 215 Screen generation unit 216 Instruction reception unit 217 Image management unit 300 Network

Claims (11)

an acquisition unit that acquires one or more image data of an eye to be inspected;
a classification unit that classifies the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
a display unit that associates the results of classification by the classification unit and displays whether or not deletion of each of the one or more image data can be accepted;
An image processing device comprising: 2. The image processing apparatus according to claim 1, further comprising a setting unit that receives setting of classification conditions corresponding to each of the plurality of classifications from a user. 3. The image processing apparatus according to claim 2, wherein the classification condition is a threshold for luminance. 4. The image processing apparatus according to claim 2, wherein said classification condition is the presence or absence of a predetermined object. The classification unit uses a trained model in which learning processing is performed using a combination of image data and one of the plurality of classifications as learning data, and when image data is input to the trained model 2. The image processing apparatus according to claim 1, wherein the classification obtained in the above is determined as the classification of the image data. 6. The learned model according to claim 5, wherein the learned model is updated by performing the learning process using the image data for which deletion permission/non-permissibility is accepted by the display unit as the learning data. Image processing device. 7. The display unit displays a list of each of the one or more image data with an icon corresponding to the result of classification by the classification unit. The image processing device according to . 8. The image processing apparatus according to any one of claims 1 to 7, wherein said plurality of classifications are performed by at least two classifications of deletion and non-deletion. 9. The method according to any one of claims 1 to 8, further comprising an image management unit that controls storage processing of image data in a storage unit based on permission/prohibition of deletion received by the display unit. image processing device. an acquisition step of acquiring one or more image data of an eye to be examined;
a classification step of classifying the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
a display control step of associating the results of the classification by the classification step and displaying on a display unit in such a manner that whether or not deletion of each of the one or more image data can be accepted;
An image processing method characterized by comprising: to the computer,
an acquisition step of acquiring one or more image data of the eye to be examined;
a classification step of classifying the one or more image data into one of a plurality of classifications indicating the possibility of deletion;
A display control step of associating the results of classification by the classification step and displaying on a display unit in a manner capable of accepting deletion of each of the one or more image data;
program to run the

Images