WO2024057548A1 - 視野推定装置、ニューラルネットワークの製造方法、及びプログラム - Google Patents

視野推定装置、ニューラルネットワークの製造方法、及びプログラム Download PDF

Info

Publication number
WO2024057548A1
WO2024057548A1 PCT/JP2022/034831 JP2022034831W WO2024057548A1 WO 2024057548 A1 WO2024057548 A1 WO 2024057548A1 JP 2022034831 W JP2022034831 W JP 2022034831W WO 2024057548 A1 WO2024057548 A1 WO 2024057548A1
Authority
WO
WIPO (PCT)
Prior art keywords
visual field
information
eye
machine learning
examined
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/034831
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
誠 古山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to JP2024546678A priority Critical patent/JPWO2024057548A1/ja
Priority to PCT/JP2022/034831 priority patent/WO2024057548A1/ja
Publication of WO2024057548A1 publication Critical patent/WO2024057548A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B3/00—Apparatus for testing the eyes; Instruments for examining the eyes
    • A61B3/02—Subjective types, i.e. testing apparatus requiring the active assistance of the patient
    • A61B3/024—Subjective types, i.e. testing apparatus requiring the active assistance of the patient for determining the visual field, e.g. perimeter types

Definitions

  • the present invention relates to a visual field estimation device, a neural network manufacturing method, and a program.
  • Estimating the progression of visual field defects caused by conditions such as glaucoma is important for determining treatment methods to slow the progression.
  • the visual field was measured at least six times at intervals, and the MD value at each time point (Mean Deviation: the patient's mean deviation at all measurement points arranged throughout the visual field of a normal person) was calculated.
  • the weighted average deviation of the patient's vision from that of a normal person of the same age as the patient was calculated, and a regression line regarding the MD value was obtained in time series to estimate the rate of progression.
  • the conventional method takes at least two years, and visual field loss progresses during that time.
  • visual field loss can be effectively delayed if appropriate treatment is started early.
  • estimated visual field information of the eye to be examined is generated from at least one of the three-dimensional structure information and the front image using an acquisition unit that acquires at least one of three-dimensional structure information and a frontal image of the retina of the eye to be examined, and a trained model.
  • Patent Document 1 discloses an ophthalmologic visual field estimating device that includes a generation unit that generates an image. Although the technique disclosed in Patent Document 1 allows easier visual field measurement testing, it does not take into account early estimation of the progress speed of visual field loss.
  • the present invention has been made in view of the above circumstances, and its purpose is to provide a visual field estimation device, a method for manufacturing a neural network, and a program that enable early determination of the speed of progression of visual field defects. Make it one.
  • One aspect of the present invention that solves the above-mentioned problems of the conventional example is a visual field estimating device that acquires at least one of retinal image information or three-dimensional structure information for a plurality of eyes to be examined at a plurality of points in time, and , acquire visual field related information related to the visual field of the eye to be examined at the corresponding time, input retinal image information or three-dimensional structure information for each of the eyes to be examined as input information, and obtain multiple information about the corresponding eye to be examined.
  • Machine learning is in a state in which machine learning is performed to output visual field change information estimated based on the input information, using visual field change information representing a change in the visual field obtained based on the visual field related information at the time as teacher information.
  • the apparatus includes an output means for outputting for predetermined processing.
  • FIG. 1 is a block diagram illustrating a configuration example of a visual field estimating device according to an embodiment of the present invention.
  • FIG. 2 is a block diagram illustrating a configuration example of a machine learning model used by a visual field estimation device according to an embodiment of the present invention.
  • FIG. 1 is a functional block diagram illustrating an example of a visual field estimating device according to an embodiment of the present invention.
  • FIG. 2 is an explanatory diagram showing an example of a screen output by the visual field estimating device according to the embodiment of the present invention.
  • FIG. 2 is a block diagram illustrating a configuration example of ⁇ -VAE used by the visual field estimating device according to the embodiment of the present invention.
  • the visual field estimating device 1 can be realized using a general computer, and as illustrated in FIG. It consists of:
  • the control unit 11 is a control device such as a processor that operates according to a program, and operates according to a program stored in the storage unit 12.
  • the storage unit 12 uses at least one of retinal image information and three-dimensional structure information as input information, and performs machine learning to estimate and output at least visual field change information based on the input information. It acts as a storage means to hold the machine learning models located in the .
  • the control unit 11 acquires at least one of image information or three-dimensional structure information of the retina of the eye to be examined of the optometry candidate, and uses the acquired image information or three-dimensional structure information of the retina of the eye to be examined of the optometry candidate.
  • the information is input to a machine learning model held in the storage unit 12, and its output is obtained as an estimated value of change in visual field related information.
  • the control unit 11 outputs an estimated value related to the change in this visual field related information. The details of the operation of this control section 11 will be described later.
  • the storage unit 12 includes a memory device and a disk device, and in addition to functioning as the storage unit described above, it also holds programs executed by the control unit 11.
  • This program may be provided stored in a computer-readable, non-transitory recording medium, and may be copied to the storage unit 12. Furthermore, this storage section 12 also operates as a work memory for the control section 11.
  • the input/output unit 13 includes an input device such as a keyboard.
  • the input/output unit 13 outputs information input according to a user's instruction to the control unit 11.
  • this input/output unit 13 is equipped with an interface such as a USB (Universal Serial Bus), and can receive three-dimensional structure information of the retina output from an OCT (Optical Coherence Tomography) inspection device, or an image of the retina obtained from a fundus camera, etc. Input such as information is accepted via this interface and output to the control unit 11.
  • USB Universal Serial Bus
  • the input/output unit 13 may include an interface (such as a network interface) that outputs information to an external device or the like according to instructions input from the control unit 11.
  • an interface such as a network interface
  • the display unit 14 is a display device or the like, and displays and outputs information according to instructions input from the control unit 11.
  • Measurement point refers to a point corresponding to each position of a plurality of visual targets arranged within a predetermined angular range from the fixation point in HFA (Humphrey perimeter) or the like.
  • the measured value at the measurement point becomes a threshold value (for example, the lowest visible brightness value) of the optotype that can be visually recognized by the optometrist (patient).
  • threshold simply refers to this threshold.
  • MD value is the average deviation, and indicates the weighted average deviation between the measured value of a person of the same age as the patient in normal visual field and the measured value of the patient at all measurement points. It is something.
  • MD Slope MD slope refers to the slope (temporal change) of MD values in time series, such as the slope of a regression line of MD values at multiple points in time.
  • VFI value (Visual Field Index) is an index of the visual field within a 24-degree range centered on the fixation point, and is an index that expresses the degree of abnormality of the visual field, with greater weight placed on the four central points. be.
  • TD value (Hemi Field MD) is the total deviation between the upper half visual field and the lower half visual field.
  • the visual field index is the central 24 degree MD value around the fixation point, Center 10 degree MD value, Center 24 degree VFI value, Center 24 degree TD value, Center 10 degree TD value, etc. (other ranges are also possible)
  • Total deviation Total deviation represents the difference between the average threshold value at each measurement point of people of the same age as the patient and the threshold value at the corresponding measurement point of the patient.
  • Pattern deviation Based on the total deviation, the threshold at each measurement point of the patient is corrected towards the normal threshold, reducing height differences across the visual field and emphasizing local subsidence areas. This is the deviation. Even if the visual field threshold decreases overall due to cataract or the like, this pattern deviation makes it easy to see localized visual field loss due to glaucoma.
  • the visual field estimation device 1 executes machine learning processing of a machine learning model.
  • the machine learning process of the machine learning model may be executed in another computer other than the visual field estimating device 1.
  • the machine learning model 20 used by the visual field estimation device 1 of this embodiment includes an input section 21, at least one neural network (NN) 22a, b..., and an output section 23, as illustrated in FIG. .
  • NN neural network
  • the neural networks 22a, b... each hold information on neural network parameters (weights, biases, etc.), and output data based on the input data input from the input unit 21 and the parameters, which will be described later. generate.
  • the output data of the neural networks 22a, b, . . . includes visual field change information in a predetermined visual field range (for example, 24 degrees at the center, 10 degrees at the center, or a mixture thereof).
  • the visual field change information may be the value itself of the slope of the regression line of the time-series change of the visual field index such as the MD slope (threshold value in clinical terms), the total deviation, or the pattern deviation. It may be. Since the calculations in the neural network 22 (hereinafter referred to as the neural network 22 when there is no need to distinguish between the neural networks 22a, b, etc.) are widely known, a detailed explanation thereof will be omitted here.
  • the neural networks 22a, b, . . . may include one corresponding to the left eye to be examined and one corresponding to the right eye to be examined. That is, in an example of this embodiment, the machine learning model 20 is - A neural network that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the center 24 degrees for the left eye, - A neural network that uses image information of the retina of the left eye as input data and outputs visual field change information of the center 24 degrees for the left eye, - A neural network that uses three-dimensional structure information of the retina of the right eye as input data and outputs visual field change information of the center 24 degrees for the left eye, - A neural network that uses image information of the retina of the right eye as input data and outputs visual field change information of the center 24 degrees for the left eye, - A neural network that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the central 10 degrees for the left eye,
  • the input unit 21 receives input data to be input to any one of the neural networks 22a, b, . It outputs to networks 22a, b, .
  • this input unit 21 inputs, along with the input data, information indicating the type of the input data (retina three-dimensional structure information, retina image information, etc.), and whether the input data relates to the left eye or the right eye. It also accepts input along with information indicating whether the item is a specific item or not.
  • the input unit 21 then outputs the input data based on the type of the input data and information indicating whether the data relates to the left eye or the right eye (hereinafter referred to as output destination selection information). Select the neural network 22 to be used. This selection can be made by setting an information table in which information indicating which neural network 22 is to be output to in association with the output destination selection information is set and stored in the storage unit 12 in advance, and by referring to the setting. do it.
  • the input unit 21 outputs input data to the selected neural network 22.
  • the number of neural networks 22 selected here is not necessarily one.
  • the input unit 21 selects the following among the neural network 22: - A neural network that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the center 24 degrees for the left eye; - A neural network that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the central 10 degrees for the left eye, (In other words, all or at least some of the neural networks that share the same type of input data) are selected, and input data is output to these neural networks.
  • the output unit 23 generates output data based on data output by the neural network 22 that receives the input data, out of at least one neural network 22 included in the machine learning model 20.
  • this output unit 23 When input data is input to only one neural network 22, this output unit 23 outputs the output data output from that one neural network 22 as is. Further, when input data is input to a plurality of neural networks 22, this output unit 23 synthesizes the output data of the plurality of neural networks 22, and outputs the combined output data.
  • a weighted average of the output data of the plurality of neural networks 22 may be calculated, and various methods widely known as methods for synthesizing the output data of the plurality of neural networks can be appropriately selected and employed.
  • control unit 11 of the visual field estimation device 1 that executes machine learning processing functionally includes an acquisition unit 31, a preprocessing unit 32, and a machine learning processing unit 33, as illustrated in FIG. 3(a). It consists of:
  • the neural network 22 included in the machine learning model 20 is: - A neural network 22a that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the center 24 degrees for the left eye; - A neural network 22b that uses image information of the retina of the left eye as input data and outputs visual field change information at the center of 24 degrees for the left eye; - A neural network 22c that uses three-dimensional structure information of the retina of the right eye as input data and outputs visual field change information of the center 24 degrees for the left eye; - A neural network 22d that uses image information of the retina of the right eye as input data and outputs visual field change information of the center 24 degrees for the left eye; - A neural network 22e that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the central 10 degrees for the left eye; - A neural network 22f that uses image information of the retina of the left eye as input data and outputs visual field change information of the center 10 degrees for the left eye
  • the neural network 22 that uses three-dimensional structural information as input data, one suitable for machine learning of three-dimensional information is selected.
  • An example of such a neural network 22 is a three-dimensional CNN (Convolutional Neural Network).
  • this is just an example, and it does not necessarily have to be CNN.
  • the neural network 22 that uses retinal image information as input data, one suitable for machine learning of two-dimensional information is selected.
  • An example of such a neural network 22 is a two-dimensional CNN. Note that this is also an example, and does not necessarily have to be CNN.
  • a widely known two-dimensional CNN such as EfficientNet or ResNet, for example, can be used, so a detailed description thereof will be omitted here.
  • the user of the visual field estimating device 1 prepares in advance the three-dimensional structure information of the retina at a plurality of examination points and the retinal structure information for a plurality of eyes of a plurality of patients.
  • information related to the visual field at each time point of the examination is also acquired.
  • This visual field related information may be the time change of the threshold itself for each measurement point included in the visual field, or the time change of the visual field index such as MD value, or the visual field index or threshold such as total deviation or pattern deviation.
  • the term "field of view index” may include information obtained based on visual field indexes and thresholds such as total deviation and pattern deviation).
  • This visual field related information at the time of the corresponding examination is the visual field range of the subject's eye obtained at multiple past points (the visual field range corresponding to the output data of the neural network 22, such as center 24 degrees, center 10 degrees, etc.). It is determined based on the actual measured value of the threshold value. This preparation by the user can be similar to conventional testing procedures.
  • the user of the visual field estimating device 1 then provides, for each eye to be examined and each time of the examination, eye identification information unique to each eye to be examined, information specifying whether the eye to be examined is the left or right eye, and information at the time of the examination.
  • the storage unit 12 stores an entry in which examination point information representing the examination point, retinal image information and three-dimensional structure information at the examination point (hereinafter referred to as input target information), and corresponding visual field change information of the subject's eye are associated with each other. Store it in .
  • the user of the visual field estimation device 1 operates the visual field estimation device 1 and instructs it to start machine learning processing.
  • the acquisition unit 31 of the visual field estimation device 1 reads and acquires the entries stored in the storage unit 12 one by one in a predetermined order (in the order of acquisition or randomly), for example. Note that general machine learning processing techniques such as batch processing may be used here. In this case, the acquisition unit 31 reads out a number of entries corresponding to the size of the mini-batch and provides them for machine learning processing.
  • the preprocessing unit 32 performs predetermined preprocessing on the input target information included in the entry acquired by the acquisition unit 31 and outputs it to the machine learning processing unit 33.
  • the preprocessing unit 32 performs widely known processing, such as predetermined noise reduction processing, noise addition processing, and histogram flattening processing, on the three-dimensional structure information that is input target information, for example.
  • the preprocessing unit 32 also reduces/enlarges the input target information in a three-dimensional space (or two-dimensional space if the input target information is two-dimensional information), and inverts/rotates the input target information with respect to a predetermined plane or axis. Processing such as the following may also be performed.
  • the machine learning processing unit 33 outputs the preprocessed image information of the retina of the subject's eye output by the preprocessing unit 32 as input data to the input unit 21 of the machine learning model 20, and also It further outputs to the input unit 21 information specifying which eye it belongs to and information indicating that the input data is retinal image information.
  • the machine learning processing unit 33 outputs the preprocessed three-dimensional structure information of the retina of the subject's eye output by the preprocessing unit 32 as input data to the input unit 21 of the machine learning model 20, and Information specifying whether the eye to be examined is the left or right eye and information indicating that the input data is three-dimensional structure information of the retina are further output to the input unit 21.
  • the input unit 21 of the machine learning model 20 outputs retinal image information as input data to the neural networks 22b and 22f corresponding to the left eye.
  • the input unit 21 also outputs the three-dimensional structure information of the retina as input data to the neural networks 22a and 22e corresponding to the left eye.
  • the output unit 23 of the machine learning model 20 synthesizes the output data of the neural network 22a and the output data of the neural network 22b, which have the same type of output data (corresponding visual field range), and calculates a visual field change of 24 degrees at the center. Output as information estimation results.
  • the output unit 23 combines the output data of the neural network 22e and the output data of the neural network 22f, and outputs the result as an estimation result of visual field change information at the center 10 degrees.
  • the synthesis by the output unit 23 here is, for example, a weighted average of the estimated value of the visual field change information included in the output data of the neural network 22a and the estimated value of the visual field change information included in the output data of the neural network 22b. This is done by generating and outputting visual field change information.
  • the weight for each output data may be updated recursively by the machine learning processing unit 33 together with the parameters of the neural networks 22a and 22b.
  • the machine learning processing unit 33 receives the output data output by the output unit 23 of the machine learning model 20, and compares the output data with the visual field change information of the corresponding visual field range included in the entry acquired by the acquisition unit 21. Then, based on the difference, the weights used by the output unit 23 included in the machine learning model 20 and the parameters of the neural network 22 that outputs the output data of the corresponding visual field range are updated. As a method for updating these parameters, etc., widely known methods such as back propagation can be employed.
  • the machine learning processing unit 33 combines the output data of the neural network 22a and the output data of the neural network 22b at a center of 24 degrees and the output data included in the entry acquired by the acquisition unit 21. , is the corresponding viewing range.
  • the weight used when the output unit 23 included in the machine learning model 20 combines the output data of the neural network 22a and the output data of the neural network 22b based on the difference from the visual field change information of the center 24 degrees;
  • the parameters of the neural network 22a and the neural network 22b are updated.
  • the machine learning processing unit 33 also uses the output data of the center 10 degrees obtained by combining the output data of the neural network 22e and the output data of the neural network 22f, and the correspondence included in the entry acquired by the acquisition unit 21. This is the field of view.
  • the weight used when the output unit 23 included in the machine learning model 20 synthesizes the output data of the neural network 22e and the output data of the neural network 22f based on the difference from the visual field change information of the center 10 degrees;
  • the parameters of the neural network 22e and the neural network 22f are updated.
  • the visual field estimating device 1 similarly performs machine learning on the neural networks 22c and 22d, the neural networks 22g and h, and the weights used to synthesize their output data for the right eye.
  • control unit 11 of the visual field estimation device 1 simply sets the machine learning model 20 to a state in which the three-dimensional structure information or image information of the retina is input, and the relationship between these is machine learned so as to output visual field change information.
  • the machine learning model 20 inputs three-dimensional structure information or image information of the retina, and in addition to visual field change information, visual field related information such as the threshold value itself for each measurement point and visual field indicators such as MD value (total deviation , pattern deviation, and other information obtained based on visual field indicators and thresholds), etc.), the relationship between the three-dimensional structure information and image information of the retina and the information to be outputted is machine learning. It is also possible to set the state as follows.
  • the visual field-related information used for machine learning (such as thresholds for each measurement point and visual field indicators (including information obtained based on visual field indicators and thresholds, such as total deviation and pattern deviation))
  • the visual field estimating device 1 estimates the visual field change information of the patient (optometry candidate) as follows using the machine learning model 20 that has been machine learned as described above.
  • the control unit 11 of the visual field estimation device 1 that performs the estimation process of visual field change information functionally includes an acquisition unit 41, a preprocessing unit 42, and an estimation processing unit 43, as illustrated in FIG. 3(b). It consists of:
  • the user of the visual field estimating device 1 sequentially selects the left and right eyes of the patient as the subject of estimation, and uses the machine learning model 20 to obtain (two-dimensional) image information of the retina of the subject eye, three-dimensional structure information of the retina, etc.
  • the same type of information used in the machine learning process is obtained using OCT, etc.
  • the control unit 11 of the visual field estimating device 1 accepts input of (two-dimensional) image information of the retina of the eye to be examined or three-dimensional structure information of the retina acquired by the user, and also inputs information on the three-dimensional structure of the retina. Accepts information indicating whether the optometry is for the left or right eye.
  • the preprocessing unit 42 performs predetermined preprocessing on at least one of the image information and three-dimensional structure information acquired by the acquisition unit 41.
  • the processing of this preprocessing unit 42 is the same as the operation of the preprocessing unit 32 for input target information in machine learning processing, so repeated explanation here will be omitted.
  • the estimation processing unit 43 outputs the result of preprocessing the image information of the retina of the eye to be examined among the outputs of the preprocessing unit 42 to the input unit 21 of the machine learning model 20 as input data, and also outputs the result of preprocessing the image information of the retina of the eye to be examined as input data. It further outputs to the input unit 21 information specifying whether it is the left or right eye and information indicating that the input data is image information of the retina.
  • the estimation processing unit 43 outputs the result of preprocessing the three-dimensional structure information of the retina of the subject's eye among the outputs of the preprocessing unit 42 to the input unit 21 of the machine learning model 20 as input data, and Information specifying whether the subject's eye is the left or right eye and information indicating that the input data is three-dimensional structure information of the retina are further output to the input unit 21 .
  • each input data is output to the corresponding neural network 22, and the output data output by the neural network 22 (if the output data of multiple neural networks 22 is used, the data obtained by combining the output data) ) is output.
  • the output data is two pieces: visual field change information related to the visual field at the center of 24 degrees of the eye to be examined, and visual field change information related to the visual field at the center of 10 degrees of the eye to be examined.
  • the estimation processing unit 43 uses the output data of the machine learning model 20 and outputs the output data itself or information obtained by subjecting the output data to a predetermined process to the display unit 14.
  • the predetermined process here is, for example, a process of estimating the patient's future visual field. An example of this processing will be explained next along with an example of the output mode.
  • the visual field estimating device 1 does not use the visual field change information itself estimated by the machine learning model 20, but the information on the patient's current visual field (even if it is actually measured, the relationship between the output of OCT, etc. and the visual field).
  • the estimation result of the current or future visual field is calculated using the visual field change information estimated by the machine learning model 20 (which may be estimated using a learned neural network, etc.) and the visual field change information estimated by the machine learning model 20. It may also be generated and output.
  • the visual field estimating device 1 uses information about the current visual field of the patient's eye for which visual field change information has been obtained, and calculates the visual field of the patient's eye at a predetermined point in the future after the change represented by the visual field change information. generate information. For example, if the visual field change information is the MD slope and the visual field change information is estimated to be -1.1 dB/year for the central 24-degree visual field, then the average MD value at each measurement point within the central 24-degree visual field is In other words, this means that each year it will increase by about 0.88 times the current level.
  • the visual field estimating device 1 estimates the MD value at a predetermined point in the future as in this example.
  • the visual field estimation device 1 uses a machine learning model 20 that performs machine learning on OCT output, etc. and visual field change information related to the threshold value for each measurement point, and calculates the threshold value at each measurement point in the same manner as in the above example.
  • An estimated value may also be obtained.
  • the visual field estimating device 1 displays a screen as illustrated in FIG. 4 to estimate the visual field at each predetermined period of time or at least one point in the future determined based on the patient's life expectancy. Output the estimation results.
  • the example screen in FIG. 4 shows an example in which the estimation result of the patient's visual field is displayed as a grayscale visual field image, which is widely known as the output of a Humphrey perimeter.
  • a grayscale visual field image which is widely known as the output of a Humphrey perimeter.
  • the image of the grayscale visual field is shown as a simple circle in Figure 4, but in reality, an image in which the portion corresponding to the measurement point of the visual field is colored gray is displayed within this circle.
  • the Rukoto is used to illustrate a simple circle in Figure 4, but in reality, an image in which the portion corresponding to the measurement point of the visual field is colored gray is displayed within this circle.
  • the visual field estimating device 1 calculates the estimated visual fields for both the visual field at the center of 24 degrees and the visual field at the center of 10 degrees for each of the left and right eyes of the patient at a certain point in time from the left in the column direction. , right eye center 24 degrees (A), right eye center 10 degrees (B), left eye center 10 degrees (C), and left eye center 24 degrees (D). Further, information on each visual field may be displayed in association with visual field change information. For example, if the estimated visual field change information about 24 degrees at the center of the right eye is -0.82 dB, the visual field estimating device 1 displays the characters "-0.82 dB" in association with column A (X). This value may indicate not only the MD slope value (visual field change information regarding the MD value) but also the VFI or TD value. Furthermore, it may be possible to switch which value is displayed.
  • the visual field estimating device 1 displays images of the four types of grayscale visual fields at each of the present (P), 5 years later (Q), and 10 years later (R) as multiple time points in the row direction. Arrange and display in chronological order.
  • the visual field estimating device 1 may display in another manner different from this.
  • the plurality of time points may be set in consideration of average life expectancy instead of in 5-year increments, or a switching button (Y) may be provided so that the user can switch between them as appropriate.
  • the visual field estimating device 1 may display the estimation results of the visual field at more points in time. In this case, if the number of rows (or columns) that can be displayed is limited due to the size of the screen, the display may be made possible by scrolling using a known interface such as a scroll bar. Alternatively, it may be possible to switch the display.
  • the switch button (Y) It may be fixed at a timing other than the timing that takes life expectancy into account, such as every five years, and displayed in a manner (for example, grayed out) to indicate that the switching operation is not possible.
  • the visual field estimating device 1 obtains the average life expectancy of the patient using a life expectancy table (such as the one published by the Ministry of Health, Labor and Welfare). do. Specifically, if the patient is a 65-year-old man (currently), the average life expectancy is determined to be 20 years.
  • a life expectancy table such as the one published by the Ministry of Health, Labor and Welfare.
  • the visual field estimating device 1 sets the time point corresponding to the first row as the present time, the time point corresponding to the last row as 20 years later, and divides the time point evenly according to the number of rows to be displayed to determine the time point corresponding to the middle row. do. For example, when displaying in three lines, the visual field estimating device 1 estimates information about the visual field now, 10 years from now, and 20 years from now.
  • the example is taken in 5-year increments when life expectancy is not taken into consideration, but this is also an example, and the visual field estimation device 1 can be used at arbitrary timings such as 3-year increments, 2-year increments, etc. according to user instructions and operations.
  • the display may be updated by changing the estimation result of the visual field at each timing.
  • the visual field estimating device 1 may also display how many years in the future the visual field is estimated for each row of images (P, Q, R).
  • the visual field estimating device 1 may also display the result of adding the estimated date and time up to a future point in time, such as "5 years later, March 2027," corresponding to an image in a certain row.
  • the patient's age and date of birth are known, the patient's age at that time may also be indicated, such as "5 years later, 70 years old, March 2027.”
  • the visual field information to be displayed is an image of a grayscale visual field, but this is also an example, and the visual field estimating device 1 numerically indicates the estimated value of the threshold at each measurement point, similar to the Humphrey perimeter. It may be shown using a total deviation, or it may be shown using a pattern deviation. Furthermore, when using the total deviation or pattern deviation, the value at each measurement point may be expressed numerically, or may be expressed as a gray scale pattern.
  • the visual field estimation device 1 separately prepares a machine learning model 20 that has directly learned the relationship between the output of OCT and values obtained based on visual field indicators and thresholds, such as total deviation and pattern deviation. Then, the prepared machine learning model 20 may be used to directly obtain the total deviation, pattern deviation, etc. related to the visual field index and threshold value, and may be displayed as numerical values or a grayscale visual field image.
  • the display mode is ⁇ Three types of grayscale visual field: threshold, total deviation, and pattern deviation. as well as, ⁇ There are three types of numerical values: threshold, total deviation, and pattern deviation. Therefore, there are at least six possible ways in total.
  • the visual field estimating device 1 may display information in the selected mode by displaying a selection button or the like to indicate which mode should be displayed, allowing the user to select the mode. At this time, the visual field estimating device 1 may display information indicating in which mode the image is being displayed on the screen.
  • the visual field estimating device 1 may output the screen displayed here to a printer to print it, in response to a user's instruction.
  • the relationship between the three-dimensional structure information of the retina and the visual field change information for each of the left eye and right eye was machine learned using separate neural networks 22, respectively.
  • the visual field estimating device 1 of this embodiment may perform machine learning processing and estimation processing as follows.
  • the neural network 22 included in the machine learning model 20 is, for example, - A neural network 22a that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the center 24 degrees for the left eye; - A neural network 22b that uses image information of the retina of the left eye as input data and outputs visual field change information at the center of 24 degrees for the left eye; - A neural network 22e that uses three-dimensional structure information of the retina of the left eye as input data and outputs visual field change information of the central 10 degrees for the left eye; - A neural network 22f that uses image information of the retina of the left eye as input data and outputs visual field change information of the center 10 degrees for the left eye; (Due to differences in input data and types of output data, there are four neural networks in total).
  • the user who performs the machine learning process of this machine learning model 20 performs the eye identification unique to each eye to be examined and at each examination time point, similar to the case where machine learning is performed by distinguishing the left and right eyes as described above.
  • information information specifying whether the eye to be examined is the left or right eye, examination time information indicating the examination time, image information and three-dimensional structure information of the retina at the examination time (input target information), An entry in which visual field change information of the corresponding eye to be examined is associated with each other is stored in the storage unit 12.
  • the acquisition unit 31 sequentially reads and acquires the entries stored in the storage unit 12 one by one, and the preprocessing unit 32 acquires the entries acquired by the acquisition unit 31.
  • a predetermined preprocessing is performed on the input target information included in the input target information, and the resultant information is output to the machine learning processing unit 33.
  • the machine learning processing unit 33 checks whether the eye to be examined related to the acquired entry is the left eye or the right eye, and if it is the right eye rather than the left eye as the reference, the preprocessed input target information
  • the machine learning processing unit 33 also reverses the left and right thresholds of the field of view, if the acquired entry includes information on the threshold of the field of view.
  • horizontal reversal means that the image information and three-dimensional structure information of the retina of the eye to be examined, as well as the measurement points (and the corresponding thresholds) within the visual field of the eye to be examined, are mirror-imaged with respect to a plane parallel to the sagittal plane of the human body. It means turning around.
  • the machine learning processing unit 33 does not perform left-right reversal.
  • the machine learning processing unit 33 sends the retinal image information preprocessed by the preprocessing unit 32 to the input unit 21 of the machine learning model 20 (if it is related to the right eye, the retinal image information is horizontally inverted), In addition to outputting the input data, information indicating that the input data is retinal image information is also output to the input unit 21 .
  • the machine learning processing unit 33 uses the preprocessed three-dimensional structure information of the retina of the subject's eye (the three-dimensional structure information of the retina that is horizontally inverted if it is related to the right eye) output by the preprocessing unit 32. It outputs it as input data to the input unit 21 of the machine learning model 20, and also outputs to the input unit 21 information indicating that the input data is three-dimensional structure information of the retina.
  • the input unit 21 of the machine learning model 20 outputs input data that is image information of the retina (regardless of whether the eye to be examined is the left eye or the right eye) to the neural networks 22b and 22f.
  • the input unit 21 also outputs input data that is three-dimensional structure information of the retina (regardless of whether the eye to be examined is the left eye or the right eye) to the neural networks 22a and 22e.
  • the output unit 23 of the machine learning model 20 synthesizes the output data of the neural network 22a and the output data of the neural network 22b, which have the same type of output data (corresponding visual field range), and calculates a visual field change of 24 degrees at the center. Output as information estimation results.
  • the output unit 23 combines the output data of the neural network 22e and the output data of the neural network 22f, and outputs the result as an estimation result of visual field change information at the center 10 degrees.
  • the synthesis by the output unit 23 here is similar to the example already described, so repeated explanation will be omitted.
  • the machine learning processing unit 33 receives the output data output by the output unit 23 of the machine learning model 20, and compares the output data with the visual field change information of the corresponding visual field range included in the entry acquired by the acquisition unit 21. Then, based on the difference, the weights used by the output unit 23 included in the machine learning model 20 and the parameters of the neural network 22 that outputs the output data of the corresponding visual field range are updated. As a method for updating these parameters, etc., widely known methods such as back propagation can be used.
  • the process of estimating the patient's visual field change information using the machine learning model machine learned according to this example is performed as follows.
  • the user of the visual field estimating device 1 sequentially selects the left and right eyes of the patient as the subject of estimation, and obtains (two-dimensional) image information of the retina of the subject eye, three-dimensional structure information of the retina, etc.
  • the same type of information as that used in the machine learning process of the machine learning model 20 is acquired using OCT or the like.
  • the control unit 11 of the visual field estimation device 1 receives input of (two-dimensional) image information of the retina of the eye to be examined and three-dimensional structure information of the retina acquired by the user through the operation of the acquisition unit 41. Information indicating whether the eye to be examined is the left or right eye is accepted.
  • the preprocessing unit 42 performs predetermined preprocessing on the input image information and three-dimensional structure information.
  • the processing of this preprocessing unit 42 is also the same as the operation of the preprocessing unit 32 for input target information in machine learning processing, so repeated explanation here will be omitted.
  • the estimation processing unit 43 determines whether the eye to be examined corresponds to the input image information or the like, the left or right eye. This determination may be made by inputting information from the user.
  • estimation processing unit 43 determines that the eye to be examined corresponding to the image information etc. input here is the right eye rather than the left eye as the reference, the estimation processing unit 43 uses the image information and three-dimensional image information of the retina of the eye to be examined output by the preprocessing unit 42. Each piece of structural information is flipped left and right.
  • the preprocessing unit 42 the result of preprocessing the image information of the retina of the eye to be examined (if the eye to be examined is the right eye, the image information after horizontal inversion) is sent to the input unit 21 of the machine learning model 20. At the same time, information indicating that the input data is retinal image information is further output to the input unit 21.
  • the estimation processing unit 43 uses the result of preprocessing the three-dimensional structure information of the retina of the eye to be examined (if the eye to be examined is the right eye, the three-dimensional structure information after left-right inversion) out of the output of the pre-processing unit 42.
  • the input data is output to the input unit 21 of the machine learning model 20 as input data, and information indicating that the input data is three-dimensional structure information of the retina is further output to the input unit 21.
  • the machine learning model 20 outputs each input data to the corresponding neural network 22.
  • input data that is image information of the retina (regardless of whether the eye to be examined is the left eye or the right eye) is output to the neural networks 22b and 22f.
  • the machine learning model 20 also outputs input data of three-dimensional structure information of the retina (regardless of whether the eye to be examined is the left eye or the right eye) to the neural networks 22a and 22e.
  • the output unit 23 of the machine learning model 20 combines the output data of the neural network 22a and the output data of the neural network 22b, which have a common output data type (corresponding visual field range), and outputs the result as an estimation result of visual field change information for the central 24 degrees.
  • the output unit 23 combines the output data of the neural network 22e and the output data of the neural network 22f, and outputs the result as an estimation result of visual field change information at the center 10 degrees.
  • the estimation processing unit 43 uses the output data of the machine learning model 20 and outputs the output data itself or information obtained using this output data to the display unit 14.
  • the visual field estimating device 1 of this embodiment basically has the above configuration, and operates as shown in the following example.
  • the visual field estimating device 1 of this embodiment performs an operation of performing machine learning processing and an operation of performing inference using the machine learning model 20 in a machine learned state.
  • a user who executes the machine learning process of the machine learning model 20 obtains at least one of three-dimensional structure information of the retina and image information of the retina obtained by OCT (input target information) from past test results for a plurality of patients;
  • the field of view in a predetermined range any range such as 30 degrees at the center, 24 degrees at the center, 10 degrees at the center, a mixture of 24 degrees at the center, and 10 degrees at the center
  • Visual field related information which is a temporal change in a visual field index or threshold based on the information, is collected.
  • the visual field information may be information actually measured using a Humphrey perimeter, etc.
  • the visual field related information that changes over time may be visual field indicators such as MD values on multiple days, or a regression line for the threshold value. It can be obtained as a parameter.
  • the user who executes the machine learning process determines whether, for the same subject's eye of the same patient, if there is a set of input target information and visual field information acquired on substantially different acquisition dates, the information is included in each set. From the visual field information, visual field change information such as MD slope is obtained, for example, by a conventional method.
  • the user who executes the machine learning process issues unique eye identification information for each eye to be examined, and records the eye identification information unique to the eye to be examined and the date and time of the examination (the date and time when the three-dimensional structure information, etc., was acquired).
  • a plurality of entries are stored in which the examination date and time information representing the examination date and time, information on the three-dimensional structure of the retina at the time of the examination (input target information), information on the visual field at the time, and the obtained visual field change information are associated. It is stored in section 12.
  • the visual field estimation device 1 sequentially reads out entries stored in the storage unit 12 one by one. Then, preprocessing such as predetermined noise reduction processing is performed on the input target information (for example, three-dimensional structure information) included in the entry.
  • the visual field estimation device 1 outputs the preprocessed input target information to the machine learning model 20 as input data.
  • the machine learning model 20 outputs output data corresponding to the input data.
  • the machine learning model 20 calculates visual field change information (MD slope, etc.) in a predetermined visual field range (for example, 24 degrees at the center, 10 degrees at the center, or a mixture thereof). It will be machine learned to output. Therefore, the visual field estimating device 1 receives the output data of the machine learning model 20, compares the output data with the visual field change information included in the read entry, and uses back propagation to calculate the machine learning model based on the difference. The parameters of the neural network included in the learning model 20 are updated, and the machine learning model 20 is subjected to machine learning.
  • a predetermined visual field range for example, 24 degrees at the center, 10 degrees at the center, or a mixture thereof.
  • the subject's eye is divided into the left eye and the right eye, and the visual field change information to be output corresponds to the range of change in the visual field.
  • ⁇ Machine learning model that takes input data such as 3D structure information of the retina of the left eye and outputs visual field change information at the center of 24 degrees for the left eye ⁇ Takes 3D structure information of the retina of the right eye as input data, and outputs visual field change information of the center of the left eye.
  • Machine learning model that outputs visual field change information of 24 degrees ⁇ Machine learning model that uses 3D structure information of the retina of the left eye as input data and outputs visual field change information of the central 10 degrees for the left eye ⁇ 3D retina of the right eye
  • a machine learning model that takes structural information etc. as input data and outputs visual field change information of the center 10 degrees for the left eye...
  • a plurality of machine learning models 20 may be obtained in advance.
  • the user who estimates the patient's visual field change information must obtain the information used in the machine learning process of the machine learning model 20, such as the three-dimensional structure information of the retina, for the patient's eyes (left eye and right eye, respectively) that are the subject of estimation.
  • the same type of information is obtained using OCT or the like.
  • the user inputs the obtained three-dimensional structure information of the retina into the visual field estimation device 1.
  • the user when inputting the three-dimensional structural information of the retina of the patient's left eye, etc., the user inputs the three-dimensional structural information of the retina of the left eye to the visual field estimation device 1, and inputs the three-dimensional structural information of the retina of the left eye as input data
  • the machine learning model 20 outputs change information, and the machine learning model 20 takes input data such as three-dimensional structure information of the retina of the left eye and outputs visual field change information at the center of 10 degrees for the left eye.
  • the visual field estimating device 1 receives this information, performs preprocessing such as predetermined noise reduction processing, inputs the preprocessed information to each instructed machine learning model 20, and performs each machine learning model. Estimated values are obtained for visual field change information (MD slope, etc.) for the visual field at the center of 24 degrees, output by the model 20, visual field change information for the visual field at the center of 10 degrees, and so on.
  • preprocessing such as predetermined noise reduction processing
  • the user when inputting information such as three-dimensional structure information of the retina of the patient's right eye, the user inputs the three-dimensional structure information of the retina of the right eye as input data to the visual field estimation device 1, and inputs information on visual field change at the center 24 degrees of the right eye. and a machine learning model 20 that takes input data such as three-dimensional structure information of the retina of the right eye and outputs visual field change information of the center 10 degrees for the right eye.
  • the visual field estimating device 1 receives this information, performs preprocessing such as predetermined noise reduction processing, inputs the preprocessed information to each instructed machine learning model 20, and performs each machine learning model. Estimated values are obtained for visual field change information (MD slope, etc.) for the visual field at the center of 24 degrees, output by the model 20, visual field change information for the visual field at the center of 10 degrees, and so on.
  • preprocessing such as predetermined noise reduction processing
  • the visual field estimating device 1 acquires visual field change information for the right eye center at 24 degrees, the right eye center at 10 degrees, the left eye center at 10 degrees, and the left eye center at 24 degrees.
  • the visual field estimating device 1 also accepts information about the patient's visual field at the current time.
  • This visual field information is estimated using a neural network that has machine learned the relationship between the output of OCT and the visual field (center 10 degrees and center 24 degrees of the left and right eyes, respectively), for example. Anything is fine.
  • the visual field estimating device 1 uses information on the estimated visual field at 24 degrees at the center of the patient's right eye and visual field change information estimated at the 24 degrees at the center of the patient's right eye, and estimates the patient's vision after 5 years and 10 years. Find the estimated threshold of the field of view. Similarly, the visual field estimating device 1 calculates estimated thresholds of the patient's visual field 5 years later and 10 years later for each of 10 degrees at the center of the right eye, 10 degrees at the center of the left eye, and 24 degrees at the center of the left eye.
  • a change in the visual field is estimated based on the current three-dimensional structure information of the retina obtained by OCT and image information of the retina, and the future visual field is estimated and displayed based on the estimated change. .
  • the machine learning model 20 uses retinal three-dimensional structure information and retinal image information as input data, and outputs information on changes in visual field (visual field related information) such as the MD value of the corresponding eye to be examined. This machine learning model 20 was used to obtain visual field change information.
  • the machine learning model of this embodiment is not limited to this example.
  • the machine learning model used by the visual field estimating device 1 of this embodiment also includes information related to other glaucoma risk factors such as intraocular pressure and corneal plasticity, age, and gender.
  • the machine learning model 20' may further accept other information as input data and generate output data including visual field change information.
  • Such a machine learning model 20' is, for example, a multilayer perceptron with one hidden layer, a Lasso regression model, a Ridge regression model, a random forest regression model, a support vector regression model (the kernel function used is a linear, (Polynomial, Gaussian), etc., or any combination of these may be used instead of singly.
  • This machine learning model 20' is subjected to machine learning processing in the following manner using the machine learning model 20 described in the examples up to this point.
  • the machine learning model 20 is first subjected to machine learning by the machine learning process described above.
  • the three-dimensional structure information of the retina and image information of the retina are used as input data, and information on the visual field of the subject's eye at the time when the three-dimensional structure information and image information are obtained (each measurement point A machine learning model (hereinafter referred to as a visual field estimation model) that has been machine learned to output a threshold value, etc.) is prepared.
  • the user who performs machine learning on the machine learning model 20' acquires three-dimensional structure information and image information at least once using OCT, etc., and separately measures the visual field at multiple points in time, and calculates the MD slope of the machine learning model 20'.
  • a plurality of eyes to be examined for which actual measured values of visual field change information such as (referred to as actual visual field change information) have been obtained preferably, the eyes to be examined are different from those used for machine learning of the machine learning model 20). Collect information about.
  • the temporal change in the visual field index is determined as follows. Note that here, it is assumed that information specifying the time point at which each image information etc. was acquired (time point of examination) is recorded together with the image information etc.
  • the user uses the visual field estimating device 1 to sequentially input image information and the like obtained at the plurality of time points for one eye to be examined as input data to the machine learning model 20. Then, the visual field estimating device 1 obtains a plurality of visual field change information corresponding to each input data for each of the center 24 degrees and the center 10 degrees obtained as output data of the machine learning model 20. The visual field estimating device 1 further outputs the average value (an arithmetic mean may be used), standard deviation, number of samples, etc. of the visual field change information obtained here.
  • the average value of the visual field change information output here is hereinafter referred to as a "direct estimated value" for distinction.
  • the visual field estimating device 1 uses the visual field estimation model to sequentially input image information about the eye to be examined, etc. input to the machine learning model 20, respectively, as input data. Then, the visual field estimation device 1 arranges the visual field estimation results obtained by the visual field estimation model according to the input image information, etc. on a time axis according to the information at the time of acquisition of the corresponding image information, etc. A regression line or the like is obtained for the value of the visual field index such as the MD value, and visual field change information (referred to as "indirect estimated value" for distinction), which is the slope of the regression line, is obtained.
  • the visual field estimation device 1 divides the visual field into multiple periods and collects visual field change information for each sample obtained within each period.
  • the visual field change information corresponding to each period may be averaged (an arithmetic average may be used) and output.
  • the visual field estimating device 1 also calculates correlation coefficients such as Pearson's product-moment correlation coefficient and Spearman's rank correlation coefficient, and values related to the p-values for the values of the visual field change information corresponding to each period. seek.
  • the value related to the p value may be a value obtained by calculating the logarithm of the general p value for the correlation coefficient and changing its sign (making it negative).
  • a small value such as 0.001, which is determined not to affect the results, is added to the p value. In this way, the larger the value regarding the p value, the higher the reliability.
  • the value related to this p value exceeds a predetermined threshold, it can be determined that the reliability of the indirect estimated value is relatively high compared to the direct estimated value, and the value related to this p value exceeds the above threshold. If there is no such value (for example, if the value related to the p value is "0"), it can be determined that the reliability of the direct estimated value is higher than that of the indirect estimated value, so one can be selectively selected based on this determination. In addition, it may be used as visual field change information of the eye to be examined.
  • the direct estimated value and the indirect estimated value may be weighted and averaged to be used as the visual field change information of the subject's eye.
  • the weight values in this case are assumed to be machine learned together with the machine learning model 20'.
  • the visual field estimation device 1 converts the indirect estimated value into the indirect estimated value of the learning target population.
  • the average value or "0" and the value related to the p value is also "0".
  • the visual field estimating device 1 also obtains the average value, standard deviation, slope of the regression line, p value, etc. obtained in the past regarding information related to risk factors of glaucoma such as intraocular pressure for the eye to be examined. . If information on intraocular pressure is not obtained, set the value to the average value of normal people (15 mmHg) or the average value of the learning target population (mainly including people with glaucoma). . In this case, the slope of the regression line is "0". However, this is just an example, and if the intraocular pressure information is not obtained, there are known methods for estimating the missing data (for example, MCflow: https:/ /arxiv.org/pdf/2003.12628.pdf) may be used to set the estimated value.
  • MCflow https:/ /arxiv.org/pdf/2003.12628.pdf
  • the visual field estimating device 1 may also obtain the average value or the like.
  • this CH value if there is no CH value obtained by the test, the average value of a normal person may be set as the average value of the CH values for the eye to be examined. However, if there is no value obtained from the test for this CH value, it will be treated as missing data, similar to the intraocular pressure information above, and a value estimated using a known method for estimating missing data will be set. You may.
  • the visual field estimating device 1 further acquires the age of the person having the eye to be examined (age at the time of final examination) when image information etc. were finally obtained by OCT or the like. At this time, information on the degree of myopia, gender, corneal thickness, etc. is also acquired.
  • the visual field estimating device 1 calculates, based on the information of the plurality of eyes to be examined collected by the user, about each eye to be examined related to the collected information.
  • ⁇ Direct estimated value, ⁇ Indirect estimated value, ⁇ Correlation coefficients and p-values related to indirect estimates, etc. as input data, and preferably also, ⁇ Mean value of intraocular pressure, standard deviation, etc. ⁇ Average value of CH value, etc. ⁇ Age at the time of final examination; ⁇ Degree of myopia, ⁇ sex, - Input data including at least one such as corneal thickness to the machine learning model 20' as input data.
  • the visual field estimating device 1 compares the measured visual field change information based on the actual visual field test results for the eye to be examined with the output data of the machine learning model 20', and performs backpropagation (in the case of a multilayer model) and Each parameter of the machine learning model 20' is adjusted by processing such as fitting (in the case of a regression model), and the relationship between each of the input data and the visual field change information is machine learned.
  • a user who wishes to estimate a patient's visual field change information using this machine learning model 20' acquires three-dimensional structure information and image information of the retina of the patient's eye using OCT, etc., and estimates the visual field. Input to device 1.
  • the user can check the past intraocular pressure test results and CH value test results of this patient's eye, the age when image information was acquired last (age at the time of final test), degree of myopia, gender, corneal thickness, etc. etc. is input into the visual field estimating device 1 to instruct it to estimate visual field change information.
  • the visual field estimating device 1 directly obtains an estimated value using the machine learning model 20 that has been machine learned. Further, the visual field estimating device 1 is a machine that takes three-dimensional structure information of the retina and image information of the retina as input data and outputs information about the visual field of the subject's eye at the time when the three-dimensional structure information and image information are obtained. The visual field is estimated using the learned visual field estimation model, and an indirect estimated value obtained based on the information of the estimated visual field, and its correlation coefficient, p value, etc. are obtained.
  • the visual field estimating device 1 calculates the average value, standard deviation, slope of the regression line, p value, etc. of the inputted past intraocular pressure test results. Note that if there is no test result for intraocular pressure, the visual field estimation device 1 calculates the average value of intraocular pressure using the average value of normal people (15 mmHg), the average value of the population to be learned, or a predetermined missing data estimation method. Set it to the value determined using . In this case, the slope of the regression line is set to "0".
  • the visual field estimating device 1 also calculates the average value thereof. Regarding this CH value, if there is no CH value obtained by the test, the visual field estimation device 1 calculates the average value of a normal person using the average value of the CH value for the eye to be examined or a predetermined missing data estimation method. Set it to the value determined by
  • the visual field estimating device 1 then performs the following on the patient's eye to be examined: ⁇ Direct estimated value, ⁇ Indirect estimated value, ⁇ Correlation coefficients and p-values related to indirect estimates, etc. ⁇ Mean value of intraocular pressure, standard deviation, etc. as input data, and more preferably, ⁇ Average value of CH value, etc. ⁇ Age at the time of final examination; ⁇ Degree of myopia, ⁇ sex, - At least one of the corneal thicknesses is further included in the input data and input to the machine learning model 20'.
  • the visual field estimating device 1 obtains the estimation result of the visual field change information regarding the subject's eye, which is output by the machine learning model 20'. Based on this estimation result, the visual field estimating device 1 may output an estimation result of the visual field at a predetermined point in the future.
  • This display example has already been explained, so repeated explanation here will be omitted.
  • the estimation result of the visual field is output based on which eye, left or right, the estimation result of the visual field change information is about.
  • indirect estimated values may be obtained based on actual test results.
  • the correlation coefficient, p value, etc. related to the indirect estimated value will also be determined based on the actual test results.
  • indirect estimated values it is determined based on the patient's (or person's eye) situation, etc. whether to use the estimated value obtained by the visual field estimation model or the estimated value obtained from the actual test results. You can. For example, in the case of the eye of a person who is not good at fixation in a visual field test, indirect estimated values obtained by a visual field estimation model may be selectively used. In this way, the visual field estimation device 1 compares the respective p values to determine whether to adopt the estimated value obtained by the visual field estimation model or the estimated value obtained from the actual test results. The one with the larger value may be adopted. Alternatively, the estimated value obtained by the visual field estimation model and the estimated value obtained from the actual test results may be weighted averaged using weights according to the respective p values, and an indirect estimated value may be obtained.
  • the visual field estimation device 1 uses estimated values of changes in visual field thresholds. may be used to obtain direct or indirect estimates.
  • the visual field estimating device 1 uses the machine learning model 20 to input three-dimensional structure information and image information of the retina obtained by OCT, etc., and performs machine learning processing to output the value of change in the visual field threshold for each measurement point. This may be done in advance, thereby obtaining a value (direct estimated value) of the change in the threshold value for each measurement point.
  • the threshold value for each measurement point is estimated and determined at multiple time points, and the regression line for each measurement point is determined using the threshold value for each measurement point at the multiple time points. It is also possible to calculate the value of the change in the threshold value and use it as an indirect estimated value.
  • the above-mentioned problem may be solved by simultaneously using visual field estimation using a variational autoencoder ( ⁇ -VAE) 50.
  • ⁇ -VAE variational autoencoder
  • the ⁇ -VAE 50 is configured to include an input layer 51, a VAE encoder 52, an intermediate layer 53, a VAE decoder 54, and an output layer 55.
  • the intermediate layer 53 outputs the values of the latent variables output by the VAE encoder 52, and the number of latent variables included in this intermediate layer 53 (that is, nodes on the output side of the VAE encoder 52) is selected experimentally. . This number can be, for example, 4, 8, 16, 32, 64, etc.
  • the user obtains three-dimensional structural information and image information of the retina using OCT, etc., and obtains actual information at multiple points in time.
  • Machine learning of ⁇ -VAE50 is performed as follows using a plurality of eyes to be examined for which visual field test results have been obtained and regression lines of changes in visual field have been obtained as learning targets.
  • a visual field estimation device 1 or the like (the machine learning device itself may be used)
  • three-dimensional structure information and image information of the retina obtained by OCT, etc. are input into a visual field estimation model that has been machine learned in advance, and the OCT, etc.
  • the threshold values for the measurement points total of 120 points
  • the central 24-degree visual field includes 52 measurement points
  • the central 10-degree visual field includes 68 measurement points. That's it.
  • this is also an example, and it is also possible to use only the measurement points within the field of view at 24 degrees at the center or only the measurement points within the field of view at 10 degrees at the center.
  • the machine learning processing device inputs the estimation results obtained here (threshold estimated values at each measurement point of a total of 120 points) as input data to the input layer 51 of the ⁇ -VAE 50, and outputs the output layer 55. obtain.
  • the machine learning processing device then calculates the value of each measurement point (the value of the measurement point at a predetermined point in the future calculated by the regression line) obtained based on the test results of the actual visual field of the subject's eye corresponding to the input data. , the values of the corresponding measurement points output to the output layer 55 are compared, and the weights and biases between each layer included in the VAE encoder 52 of the ⁇ -VAE 50, the VAE decoder 54, etc. are calculated using a backpropagation method or the like.
  • Machine learning of ⁇ -VAE50 is performed by updating parameters such as.
  • the visual field estimating device 1 performs machine learning processing in this way when estimating visual field information of a certain subject's eye at multiple points in the future using the visual field change information estimated by the machine learning models 20 and 20'. Processing is performed as follows using the ⁇ -VAE50 obtained.
  • the visual field estimating device 1 sequentially inputs the information on the visual field estimated for the subject's eye at the plurality of points in time (hereinafter referred to as the first estimated future visual field) to the ⁇ -VAE 50 which is in a machine learning state. do. Then, the visual field estimating device 1 obtains the values of the latent variables output by the intermediate layer 53 of the ⁇ -VAE 50 corresponding to the visual field information at each time point.
  • the visual field estimating device 1 calculates a regression line for a plurality of points with the value of each latent variable as the Y-axis and the estimated future time point as the X-axis, and also In addition to calculating correlation coefficients such as Pearson's product-moment correlation coefficient and Spearman's rank correlation coefficient, the p-value (as explained above, the logarithm of the general p-value is calculated and its sign is set as negative). to ask for something. At this time, the visual field estimating device 1 may also obtain and record the number of samples (the estimated number of future time points).
  • the visual field estimating device 1 uses the regression line determined for each latent variable to estimate the value of each corresponding latent variable at a point in time after a long period of time (for example, a point in time after the average life expectancy).
  • the visual field estimation device 1 inputs the values of the latent variables estimated here to the respective corresponding input nodes of the VAE decoder 54 of the ⁇ -VAE 50. Then, the visual field estimating device 1 converts the estimated value of the threshold value at each measurement point within the visual field outputted by the output layer 55 at this time into the estimated value of the threshold value at each measurement point at the point in time after the aforementioned long period. Suppose there is.
  • prediction is performed for each measurement point within the field of view, taking into account information on measurement points surrounding the measurement point.
  • the visual field estimating device 1 may further output the average value of the p-values of the latent variables as an index of reliability.
  • the visual field estimating device 1 calculates, for each measurement point (for each of the 120 measurement points in the above example), a future (after a long-term period) threshold value obtained using ⁇ -VAE50, and a threshold value for the relevant measurement.
  • a future (after a long-term period) threshold value obtained using ⁇ -VAE50
  • a threshold value for the relevant measurement For each point, the value of the first estimated future visual field estimated at multiple points in time is plotted with the corresponding point in time on the X-axis and the above value on the Y-axis, and the future (long-term period A regression line relating to the value of the first estimated future visual field estimated at a plurality of points in time, passing through the point on the XY plane corresponding to the threshold value in (after ), is determined by the method of least squares or the like.
  • the visual field estimating device 1 includes, for each measurement point, corresponding visual field change information and past or current visual field (a threshold value for each visual field measurement point actually obtained in an examination, a threshold value estimated by a visual field estimation model, etc.) etc.) can be used to determine a future estimated threshold value (referred to as a second estimated future visual field).
  • a second estimated future visual field instead of using the second estimated future visual field as it is, the visual field estimating device 1 may estimate the visual field after a long period of time as follows.
  • the visual field estimating device 1 calculates the p-value P1 of the slope (visual field change information without using ⁇ -VAE50) of the regression line (referred to as the first regression line) used to obtain the first estimated future visual field, and ⁇ -VAE50. Based on the average value P2av of the p value P2 regarding the slope of the regression line (for convenience, referred to as the second regression line) for each measurement point obtained using , the weight for the second regression line is determined as P2av/(P1+P2av).
  • the visual field estimating device 1 uses, for each measurement point, the slope value ⁇ 1 of the first regression line and the slope value ⁇ 2 of the second regression line regarding the measurement point, to estimate the visual field change regarding the measurement point.
  • the slope ⁇ of ⁇ ⁇ 1 ⁇ P1/(P1+P2av)+ ⁇ 2 ⁇ P2av/(P1+P2av) Find it as.
  • the visual field estimating device 1 uses information on the patient's visual field at multiple points in time (even if it is actually measured, a neural network etc. that has machine learned the relationship between the output of OCT, etc. and the visual field). ) and the slope ⁇ of the visual field change for each measurement point found here, have a corresponding slope ⁇ for each measurement point, and A regression line for the measurement point is obtained by the least squares method on the threshold information in the visual field.
  • the visual field estimating device 1 outputs the threshold value at each measurement point obtained here by displaying it in the manner illustrated in FIG. 4 or the like.
  • the mixing ratio of the slope value ⁇ 1 of the first regression line and the slope value ⁇ 2 of the second regression line regarding the measurement point is determined directly based on the respective p values, but in this embodiment, the optimum value of the mixture ratio may be experimentally obtained by machine learning using a mixture ratio machine learning model (a general multilayer perceptron, a Lasso regression model, a Ridge regression model, etc.).
  • the machine learning of this mixed ratio machine learning model is based on the regression line (first regression It can be used as an estimated future field of view obtained using a straight line).
  • the object to be mixed is not limited to the slope value ⁇ 1 of the first regression line and the slope value ⁇ 2 of the second regression line regarding the measurement point, and the visual field estimation device 1 can mix the regression line obtained from the actually measured visual field.
  • the mixture ratio such as the slope value is determined using the above-mentioned mixture ratio machine learning model, and the slope value ⁇ 1 of the first regression line, the slope value ⁇ 2 of the second regression line regarding the measurement point, and the actually measured visual field are determined.
  • a value obtained by weighting and averaging the slope of the regression line obtained from the above using the mixing ratio may be used as the slope of the visual field change of the subject's eye for estimating the visual field.
  • the visual field estimating device 1 estimates the visual field (threshold value at each measurement point within) sufficiently in the future (for example, after the average life expectancy) by using the slope of the visual field change of the subject's eye obtained here. Then, the estimated threshold of the visual field and the threshold at each measurement point of the actually measured visual field are arranged on the XY plane, with the Y-axis value as the threshold and the measured time point as the X-axis value. Using the point, minimize the sum of the distances from the other points corresponding to the actually measured threshold by passing through the point on the XY plane corresponding to the threshold in the field of view sufficiently far in the future estimated above using the least squares method. Obtain a straight line. The slope of this straight line becomes the progress speed of the visual field (visual field change information) at the measurement point.
  • the visual field estimating device 1 may thereby obtain visual field change information for each measurement point, and estimate the visual field status of the subject's eye at a future point in time based on the obtained visual field change information.
  • the following data augmentation processing may be performed on the input data.
  • Machine learning processing may be performed using the average value of the estimated value obtained by inputting the data into the machine learning model 20 and the estimated value obtained by inputting the (original) data before inversion into the machine learning model 20. .

Landscapes

  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Biophysics (AREA)
  • Ophthalmology & Optometry (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Eye Examination Apparatus (AREA)
PCT/JP2022/034831 2022-09-16 2022-09-16 視野推定装置、ニューラルネットワークの製造方法、及びプログラム Ceased WO2024057548A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024546678A JPWO2024057548A1 (https=) 2022-09-16 2022-09-16
PCT/JP2022/034831 WO2024057548A1 (ja) 2022-09-16 2022-09-16 視野推定装置、ニューラルネットワークの製造方法、及びプログラム

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/034831 WO2024057548A1 (ja) 2022-09-16 2022-09-16 視野推定装置、ニューラルネットワークの製造方法、及びプログラム

Publications (1)

Publication Number Publication Date
WO2024057548A1 true WO2024057548A1 (ja) 2024-03-21

Family

ID=90274777

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/034831 Ceased WO2024057548A1 (ja) 2022-09-16 2022-09-16 視野推定装置、ニューラルネットワークの製造方法、及びプログラム

Country Status (2)

Country Link
JP (1) JPWO2024057548A1 (https=)
WO (1) WO2024057548A1 (https=)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021043980A1 (en) * 2019-09-06 2021-03-11 Carl Zeiss Meditec, Inc. Machine learning methods for creating structure-derived visual field priors

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021043980A1 (en) * 2019-09-06 2021-03-11 Carl Zeiss Meditec, Inc. Machine learning methods for creating structure-derived visual field priors

Also Published As

Publication number Publication date
JPWO2024057548A1 (https=) 2024-03-21

Similar Documents

Publication Publication Date Title
JP7750827B2 (ja) 視力を予測するための眼の画像を処理するための深層学習の使用
Sigal et al. Modeling individual-specific human optic nerve head biomechanics. Part I: IOP-induced deformations and influence of geometry
CN114390907B (zh) 用于创建结构导出的视场先验的机器学习方法
Liu et al. Diagnosing persistent hypertransmission defects on en face OCT imaging of age-related macular degeneration
Sekhar et al. Sensitivity of Swedish interactive threshold algorithm compared with standard full threshold algorithm in Humphrey visual field testing
Medeiros et al. Combining structural and functional measurements to improve estimates of rates of glaucomatous progression
Lahav et al. Reduced mesopic and photopic foveal contrast sensitivity in glaucoma
Wall et al. Long-and short-term variability of automated perimetry results in patients with optic neuritis and healthy subjects
Barot et al. Reading performance in infantile nystagmus
CN115553707A (zh) 一种基于眼动追踪的对比敏感度测量方法及设备
Menda et al. Association of postfungal keratitis corneal scar features with visual acuity
Lempert The axial length/disc area ratio in anisometropic hyperopic amblyopia: a hypothesis for decreased unilateral vision associated with hyperopic anisometropia
Martínez-Albert et al. Repeatability assessment of biometric measurements with different refractive states and age using a swept-source biometer
Cole et al. Variability in plus disease diagnosis using single and serial images
Nagarsheth et al. Relationship between Fuchs endothelial corneal dystrophy severity and glaucoma and/or ocular hypertension
Song et al. Clinical prediction performance of glaucoma progression using a 2-dimensional continuous-time hidden Markov model with structural and functional measurements
Rowatt et al. Field evaluation of the Welch Allyn SureSight™ Vision Screener: Incorporating the Vision in Preschoolers study recommendations
Kling et al. Optical coherence tomography quantifies gradient refractive index and mechanical stiffness gradient across the human lens
JP7657782B2 (ja) ハンチントン病（ｈｄ）を評価する手段および方法
Charng et al. Nonlinear reduction in hyperautofluorescent ring area in retinitis pigmentosa
Patel et al. Comparison of quality and output of different optimal perimetric testing approaches in children with glaucoma
Zhu et al. Quantifying discordance between structure and function measurements in the clinical assessment of glaucoma
Gomes et al. Wavefront sensing: A breakthrough for objective evaluation of dynamic accommodation in accommodative dysfunctions
WO2024057548A1 (ja) 視野推定装置、ニューラルネットワークの製造方法、及びプログラム
Myung et al. Evaluation of vascular disease progression in retinopathy of prematurity using static and dynamic retinal images

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22958867

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024546678

Country of ref document: JP

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22958867

Country of ref document: EP

Kind code of ref document: A1