WO2020117021A1

WO2020117021A1 - Self fundus image capture device, self fundus image capture and interpretation device, and self fundus image capture and interpretation system

Info

Publication number: WO2020117021A1
Application number: PCT/KR2019/017258
Authority: WO
Inventors: 강욱; 신일형; 박상민; 박기호; 오백록; 장주영
Original assignee: 인더스마트 주식회사; 서울대학교병원
Priority date: 2018-12-07
Filing date: 2019-12-09
Publication date: 2020-06-11
Also published as: KR20200069547A; KR102184761B1

Abstract

Disclosed is a self fundus image capturing device comprising: a display unit on which an image for a focal point is displayed; a first sensor unit which specifies the state of the pupils of an examinee; a second sensor unit which measures the blinking of the eyes the examinee; a camera which captures at least one among an image of the pupils of the examinee and an image of the fundi of the examinee; and a control unit which, if, on the basis of the measurement value of the first sensor unit, the measurement value of the second sensor unit, and the pupil image of the examinee, the fundus image of the examinee is within a preset fundus image information range, determines the captured fundus image of the examinee to be a fundus image for interpretation of a lesion.

Description

Self fundus imaging device, self fundus imaging and reading device and self fundus imaging and reading system

The present invention is a self fundus image taking device, a self funding image taking and reading device, and a self funding image taking and reading system, more specifically, the subject can take a fundus image by himself or herself without the help of a medical staff, as well as a captured fundus The present invention relates to a self fundus imaging device, a self fundus imaging and reading device, and a self fundus imaging and reading system capable of reading a lesion from an image and supporting medical diagnosis.

Fundus imaging is one of the most used ophthalmic imaging for the purpose of diagnosis or recording in ophthalmology, and is mainly used for ophthalmic disease examination.

However, such a fundus image has a large difference in sharpness depending on the proficiency of the medical staff who shoots the fundus image. There is a problem of increasing probability.

Meanwhile, with the recent development of AI learning models, various AI learning models have been used in reading medical images.

For example, deep learning models, including machine learning models, and more specifically, Convolutional Neural Networks, Recurrent Neural Networks (RNNs), detect, classify, and characterize medical imaging images. It is being used to learn.

In particular, the fundus field in which image reading is important also supports reading/diagnosis of the fundus image of an examinee using the learning model.

However, in order to improve the accuracy of the fundus image reading in the case of fundus image reading using this learning model as well as the fundus image reading by a medical staff, it is important to first obtain a high quality clear fundus image without noise in the fundus image itself. .

To this end, medical staff is taking a lot of time and effort to take fundus images.

However, as described above, differences in clarity, etc. of the fundus image occur according to the proficiency of the medical staff who shoots the fundus image, so there is still a problem of misdiagnosis, whether by a medical staff or by a learning model.

The present invention has been devised to solve the above problems, and the object of the present invention is that the subject's pupil is enlarged so that the fundus image can be taken when the pupil is enlarged. It is to provide a fundus image taking device that can take a fundus image.

Another object of the present invention is a self fundus image capable of obtaining a clear fundus image suitable as a fundus image to be read as the fundus image to be reconstituted so that when the photographed fundus image is unsuitable as a fundus image to be read as a lesion, It is to provide a photographing device.

Another object of the present invention is to provide a self fundus image photographing apparatus that is configured to simultaneously photograph the left eye and the right eye when taking a fundus image, thereby obtaining equal clarity of both fundus images.

Another object of the present invention is to read the lesion by applying an artificial intelligence algorithm to the fundus image to be read by the lesion, a self fundus imaging and reading device supporting a diagnosis of a lesion of a medical staff, and a self fundus imaging and reading system Is to provide.

In order to achieve the above object, the apparatus for photographing a fundus image according to the present invention includes: a display unit displaying a focus image, a first sensor unit measuring a pupil state of a test subject, and a second sensor measuring a blink of the eyelid of the test subject. The camera unit photographs at least one of the pupil image of the examinee and the fundus image of the examinee, and the measured value of the first sensor portion, the measured value of the second sensor portion, and the pupil image of the examinee based on the pupil image of the examinee It may include a control unit for controlling the camera unit to shoot the fundus image, and if the photographed fundus image of the subject is within a preset range of the fundus image, the determined fundus image of the subject is determined as a fundus image to be read.

The pupil state of the subject may be measured by the pulse of the subject.

Preferably, the first sensor unit includes an ECG (Electro-CardioGram) sensor or a PPG (Pulse-PlethysmoGram) sensor.

It is preferable that the first sensor unit is.

The second sensor unit may include at least one left eye sensor and a right eye sensor.

The second sensor unit may include an Electro OculoGram (EOG) sensor.

The focusing image may include a focusing image for the left eye and a focusing image for the right eye.

The controller may control the camera unit to first capture one fundus image among the fundus image of the left eye and the fundus image of the right eye, and then photograph the remaining fundus image.

The control unit may control the focus image for the left eye and the focus image for the right eye so that the focus image for the left eye and the focus image for the right eye displayed on the display unit are separated from each other.

The controller may control the camera unit to simultaneously photograph the fundus image of the left eye and the fundus image of the right eye of the examinee.

When the pupil size in the pupil image of the subject photographed by the camera unit is greater than or equal to a pupil size of a preset pupil image, and there is no blink of the subject by measurement of the second sensor unit, the subject The camera unit may be controlled to photograph the fundus image.

The controller may determine whether the pupil size in the pupil image of the subject photographed by the camera unit is equal to or greater than a pupil size of a preset pupil image based on a machine learning algorithm or a DSP algorithm.

The controller may determine whether the photographed fundus image of the examinee is within a preset information range of the fundus image based on the machine learning algorithm.

The controller may control the camera unit to re-photograph the fundus image of the examinee when the photographed fundus image of the examinee is outside the preset range of the fundus image.

When the photographed fundus image of the examinee is within a preset range of the fundus image, the controller may determine the photographed fundus image of the examinee as a lesion reading target fundus image.

The self fundus image photographing and reading device according to the present invention for achieving the other object includes a first display unit displaying a focus image, a first sensor unit measuring a pupil's pupil state, and blinking of the subject's eyelid A second sensor unit to measure, a camera unit that captures at least one of the pupil image of the subject and the fundus image of the subject, and the measured value of the first sensor unit, the measured value of the second sensor unit, and the pupil image of the subject On the basis of this, the camera unit is controlled to photograph the fundus image of the testee, and when the fundus image of the testee is within a preset range of the fundus image, the fundus image of the testee is determined as a fundus image to be read by the lesion, and the It may include a control unit for reading the lesion from the fundus image to be read.

The control unit may read the lesion from the fundus image to be read, based on the machine learning algorithm.

The test subject may further include a second display unit displaying the result of reading the lesion from the fundus image to be read.

An auto fundus image photographing and reading system according to the present invention for achieving the above another object, an auto fundus image photographing apparatus, and a lesion reading apparatus for reading a lesion from a fundus image to be read from a lesion received from the auto fundus image photographing apparatus Including, the self fundus image taking device, the display unit for displaying the focus image, the first sensor unit for measuring the pupil state of the subject, the second sensor unit for measuring the blink of the eyelid of the subject, the subject The camera unit photographing at least one of the pupil image and the fundus image of the examinee, and the measured value of the first sensor portion, the measured value of the second sensor portion, and the pupil image of the examinee based on the pupil image of the examinee It may include a control unit for controlling the camera unit, and if the photographed fundus image of the subject is within a preset range of the fundus image, the determined fundus image of the subject is determined as a fundus image to be read.

The lesion reading apparatus may read a lesion from the fundus image to be read based on the machine learning algorithm.

According to the present invention, since the size of the pupil of the examinee is enlarged so that the fundus image can be taken, the examinee has an advantage of taking a clear fundus image by himself without the help of a medical staff.

According to the present invention, when the photographed fundus image is unsuitable as a fundus image to be read as a lesion, it is configured to re-photograph the fundus image, which is advantageous in obtaining a suitable fundus image that is suitable as the fundus image to be read.

According to the present invention, it is configured to simultaneously photograph the left and right eyes when photographing the fundus image, and thus has the advantage of obtaining uniform clarity and the like for both fundus images.

According to the present invention, by applying a machine learning algorithm or a deep learning algorithm to the fundus image to be read, the lesion is read, and thus there is an advantage of supporting the diagnosis of the lesion of the medical staff.

1 schematically shows a configuration diagram of an auto fundus imaging apparatus according to an embodiment of the present invention.

FIG. 2 schematically shows an image for focus displayed on a display unit of an autonomous fundus imaging apparatus according to an embodiment of the present invention.

3 schematically shows the ECG signal measured by the ECG sensor.

Figure 4 schematically shows the PPG signal measured by the PPG sensor.

5 is a view schematically illustrating a process of horizontally focusing an examinee according to an embodiment of the present invention.

6 illustrates a fundus image of the left eye and a fundus image of the right eye photographed by a camera according to an embodiment of the present invention.

7 schematically shows a configuration diagram of an auto fundus image photographing and reading device according to an embodiment of the present invention.

8 schematically illustrates a configuration diagram of an autonomous fundus image photographing and reading device according to another embodiment of the present invention.

9 schematically shows a configuration diagram of an autonomous fundus imaging and reading system according to an embodiment of the present invention.

10 schematically illustrates the appearance of an auto fundus imaging apparatus according to an embodiment of the present invention.

11 schematically illustrates a form in which an auto fundus imaging device according to an embodiment of the present invention is connected to another support device.

It should be noted that the technical terms used herein are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted as meanings generally understood by those having ordinary knowledge in the technical field to which the present invention belongs, unless defined otherwise. It should not be interpreted as a meaning or an excessively reduced meaning. In addition, when the technical terms used in this specification are incorrect technical terms that do not accurately represent the spirit of the present invention, it should be understood as being replaced by technical terms that can be correctly understood by those skilled in the art.

In addition, the general terms used in the present invention should be interpreted as defined in the dictionary or in context before and after, and should not be interpreted as an excessively reduced meaning.

In addition, the singular expression used in this specification includes the plural expression unless the context clearly indicates otherwise. In the present application, the terms "consisting of" or "comprising" should not be construed as including the various components, or various steps described in the specification, among which some components or some steps are It may not be included, or it should be construed to further include additional components or steps.

Hereinafter, the present invention will be described in more detail through examples, but the scope of the present invention is not limited to the following examples.

1 schematically shows a configuration diagram of an auto fundus imaging apparatus according to an embodiment of the present invention, and FIG. 2 shows a focus displayed on a display unit of the auto fundus imaging apparatus according to an embodiment of the present invention It is a schematic illustration of a dragon image.

Referring to FIG. 1, an auto fundus imaging apparatus 100 according to an embodiment of the present invention includes a display unit 110, a first sensor unit 120, a second sensor unit 130, and a camera unit 140 ), and the computing unit (C).

The computing unit C may include a control unit 150, a memory 160, a storage 170, an input/output interface 184, and the like. The controller 150 may include one or more processors.

Referring to FIG. 2, an image for focus is displayed on the display unit 110 to focus an examinee before taking a fundus image.

The display unit 110 may include a first region 112 in which the left focus image L is displayed and a second region 114 in which the right focus image R is displayed.

A wall W is provided between the first region 112 and the second region 114 so that both regions are physically separated.

Referring back to FIG. 1, the first sensor unit 120 may measure the pupil state of the examinee. The pupil state of the subject may be measured by the pulse of the subject.

The first sensor unit 120 may be any suitable sensor technology including a pressure sensor, a temperature sensor, a mechanical sensor, a motion sensor, an optical sensor, and an electronic sensor, but is not limited thereto. According to various embodiments, for example, at least one sensor among an Electro MyoGraphy (EMG) sensor, an Electro CardioGram (ECG) sensor, and a Pulse PlethysmoGram (PPG) sensor may be used as the first predecessor 120.

The ECG sensor is also referred to as an EKG sensor, and is a sensor that measures an electrocardiogram, a weak electrical signal that reflects the electrical activity stage of the heart that can be measured on the body surface.

3 schematically shows the ECG signal measured by the ECG sensor. As illustrated in FIG. 3, it can be divided into a constant low-frequency region and an irregular high-frequency region according to how regular and irregular the R-R interval is.

Visually, even if the interval between the R-wave and the R-wave looks uniform, the R-R interval is gradually changing when the value is actually quantitatively extracted. In a normal person, the change in the R-R interval occurs because the autonomic nervous system controls antagonism in the heart rhythm generator. Therefore, by analyzing changes in the R-R interval, it is possible to grasp the active aspects of the sympathetic and parasympathetic nervous systems that make up the autonomic nervous system.

When the subject is in the pupil state, the R-R interval appears irregular. Therefore, it can be confirmed that the subject is in the pupil state through the ECG sensor. The PPG sensor is a sensor that measures pulse waves, and measures the pulse by sensing the flow of a radial artery on the wrist.

Figure 4 schematically shows the PPG signal measured by the PPG sensor. The IBI shown in FIG. 4, like the R-R interval shown in FIG. 3, may be judged to be in a tension state when the interval is irregular.

On the other hand, the first analog front end (Analog Front-end, 122) includes a gain stage (Gain, 122a) and an analog-to-digital converter (Analog-Digital Convertor, 122b).

The first analog front end 122 is electrically connected to the first sensor unit 120 and converts a signal detected by the first sensor unit 120 into a digital signal and provides it to a processor (control unit, 150). do.

The gain stage 122a is composed of a single-ended differential amplifier or a measurement amplifier to adjust the output signal width of the first sensor unit 120 having a narrow output signal range.

The analog-to-digital converter 122b converts the output signal of the adjusted first sensor unit 120 to a digital signal.

The processor 150 may be a CPU, an application processor (AP), a micro controller, a digital signal processor (DSP), or the like, but may also be a processing module that automatically processes through a machine learning model, but is not limited thereto.

The processor 150 may communicate with the display adapter 180 to display the operation and user interface of the self fundus imaging apparatus on a display device (not shown).

The auto fundus imaging apparatus according to an embodiment of the present invention may transmit and receive commands including messages, data, information, and one or more programs (ie, application codes) through the network adapter 182.

The network adapter 182 may include separate or integrated antennas to enable transmission and reception over a network link. The network adapter 182 may connect to a network and communicate with a remote computing device (not shown), such as a remote fundus image reading device. The network may include, but is not limited to, at least one of a LAN, WLAN, PSTN, and cellular phone network.

The network adapter 182 may include at least one of a network interface and a mobile communication module for accessing the network. The mobile communication module is connectable to a mobile communication network for each generation (for example, 2G to 5G mobile communication networks).

The memory 160 may store an operating system, a driver, an application program, data, and a database required for the operation of the self fundus imaging device according to an embodiment of the present invention, but is not limited thereto.

In addition, the memory 160 may include a computer-readable medium in the form of a volatile memory such as Random Acces Memory (RAM), and a non-volatile memory such as ROM (Read Only Memory) and flash memory, and further, a disk drive example For example, a hard disk drive (Hard Disk Drive), a solid state drive (Solid State Drive), may include, but are not limited to.

Further, the memory 160 may typically include data such as imaging data, program modules such as an operating system and imaging software that can be immediately connected to be operated by a processor.

The second sensor unit 130 may include at least one sensor capable of tracking pupil movement, eye blinking, eye vision, and the like, of an examinee who is using a self fundus imaging device.

The pupil image described herein refers to an image of a pupil state that can be obtained through the first sensor unit 120 or the second sensor unit 130, or a pupil image for overall eye tracking. it means.

The second sensor unit 130 may be any sensor technology including a motion sensor, an optical sensor, and an electronic sensor, but is not limited thereto. As one of various embodiments, an EOG (Electro OculoGram) sensor may be used as the second sensor unit 130, and the EOG sensor measures the safety latitude of an examinee.

The second sensor unit 130 may include a left eye sensor and a right eye sensor. At this time, the sensor for the left eye may be preferably at least one, more preferably at least two, and the sensor for the right eye may also be preferably at least one, more preferably at least two.

When the size of the pupil of the test subject is turned off, the fundus image must be photographed to have appropriate sharpness and the like as the fundus image to be read.

The fundus image described in this specification means an image photographed to search for lesions of the eye, and may be a still image or a motion image.

In the present invention, in order to check when the pupil size of the test subject is increased, it is confirmed whether the pulse of the test subject is irregularly running through the ECK or PPG sensor as described above. In addition, it is configured to photograph the fundus image of the examinee after confirming that the eye of the examinee blinks through the EOG sensor. That is, it is configured to shoot the fundus image at a time when there is no blinking of the eye.

The camera unit 140 includes an image sensor (not shown) that captures an image of an object and converts the image into an image signal photoelectrically, and photographs a pupil image and a fundus image of the examinee. The photographed pupil image is provided to a control unit (processor, 150) and processed based on an image processing or machine learning model.

In addition, the camera unit 140 photographs the pupil image of the examinee for the purpose of determining whether the pupil size of the examinee is suitable for capturing the fundus image before photographing the fundus image of the examinee.

As one of various embodiments, a Fundus Camera may be used as the camera unit 140.

The controller 150 controls the camera unit 140 to photograph the fundus image of the examinee based on the measured value of the first sensor portion, the measured value of the second sensor portion, and the pupil image of the examinee.

That is, the control unit 150 has an irregular pulse of the subject measured by the first sensor unit 120, and the pupil size in the pupil image of the subject photographed by the camera unit 140 is preset. When the pupil size is greater than or equal to the pupil size in the pupil image, and there is no blinking of the subject's eye by measurement of the second sensor unit 130, the camera unit 140 may be controlled to photograph the fundus image of the subject.

At this time, the control unit 150 is based on a machine learning model or a digital signal processor (DSP) algorithm, the pupil size of the pupil image photographed by the camera unit 140 is set in the pupil image preset. It is possible to determine whether the pupil is larger than the size.

In order to grasp the most appropriate time for taking a fundus image, various pupil state information may be used in addition to the pupil size described above.

Deep learning algorithms, which are one of the machine learning algorithms, have various models such as deep neural networks (DNNs) and convolutional neural networks (CNNs).

Deep neural network (DNN) is an artificial neural network (ANN) composed of several hidden layers between an input layer and an output layer. Deep neural networks (DNNs) can model complex non-linear relationships, just like a normal artificial neural network.

For example, in a deep neural network structure for an object identification model, each object may be represented by a hierarchical configuration of basic elements of an image. At this time, the additional layers can aggregate features of the gradually collected lower layers. This feature of the deep neural network allows modeling of complex data with fewer units (nodes) than a similarly performed artificial neural network.

Convolutional neural networks (CNNs) are a type of multilayer perceptrons designed to use minimal preprocessing. The convolutional neural network (CNN) consists of one or several convolutional layers and common artificial neural network layers on top of it, and additionally uses weights and pooling layers. Thanks to this structure, a convolutional neural network (CNN) can fully utilize input data of a two-dimensional structure. Compared with other deep learning structures, the convolutional neural network (CNN) shows good performance in both video and audio fields. Convolutional neural networks (CNNs) can also be trained through standard reverse propagation. Convolutional neural networks (CNNs) are easier to train than other feedforward artificial neural network techniques and have the advantage of using fewer parameters.

In deep learning, a Convolutional Deep Belief Network (CDBN) was developed.It is structurally very similar to the existing Convolutional Deep Belief Network (CBN), so it can be used well in a two-dimensional structure, and at the same time, deep trust neural networks (Deep) You can also take advantage of pre-training on the Belief Network (DBN).

The convolutional deep trust neural network (CDBN) provides a general structure that can be used for various image and signal processing techniques, and is used in various benchmark results for standard image data such as CIFAR.

Recurrent neural network (Recurrent Neural Network, RNN) refers to a neural network in which a connection between units constituting an artificial neural network constitutes a directed cycle. Cyclic neural networks can utilize memory inside the neural network to process arbitrary input. Due to these characteristics, the circulatory neural network is used in fields such as handwriting recognition and exhibits a high recognition rate.

Various types of image processing such as Image Synthesis and White Balancing are performed on a digital image signal through a DSP algorithm.

The control unit 150 may control the camera unit 140 to first capture one fundus image among the fundus image of the left eye and the fundus image of the right eye, and then photograph the remaining fundus image.

The control unit 150 may focus the left eye focus image L and the right eye focus image L and the right eye so that the left eye focus image L and the right eye focus image R displayed on the display 110 are separated from each other. You can control the image R for focus. This will be described in detail with reference to FIG. 5.

Referring to FIG. 5, on the display unit 110, a left focus image L and a right focus image R are displayed on the first area 112 and the second area 112, respectively. In this case, the focus of the examinee is formed in the region S.

However, when the left-eye focusing image L and the right-eye focusing image R are moved away from each other, the subject's focus is formed in the region S'. Therefore, compared to the case where the focus is formed in the S region, the focus of the examinee becomes close to the horizontal state.

When the subject's focus is close to the horizontal state, the fundus image of the left eye and the fundus image of the right eye can be simultaneously photographed.

The control unit 150 may control the camera unit 140 to simultaneously shoot the left and right eye images when the left eye and the right eye images are simultaneously photographed. Through this, both the left eye and the right eye can obtain clear fundus images, and the fundus images of both eyes are illustrated in FIG. 6.

Meanwhile, when the fundus image of the subject to be photographed is within a preset range of the fundus image, the control unit 150 determines the fundus image of the subject to be read as a lesion fundus image. For example, the information of the preset fundus image may include tissue (vessel, fovea, optic disc, macula) basically seen in the fundus image, contrast or brightness of the fundus image. The information listed above is only an example and is not necessarily limited thereto.

In more detail, the controller 150 may apply a machine learning model to determine whether the fundus image of the examinee is within a predetermined range of fundus images.

The machine learning algorithm described above, with sufficient iteration, such as detecting an image in the fundus image, classifying it, and learning a feature, determines whether the subject's fundus image is within the information range of a preset fundus image with high accuracy. You can.

As a result of the determination, when the photographed fundus image of the examinee is outside the preset range of the fundus image, the controller 150 may control the camera unit 140 to re-photograph the fundus image of the examinee.

As a result of the determination, when the photographed fundus image of the examinee is within a preset range of the fundus image, the controller 150 may determine the photographed fundus image of the examinee as a target image of the lesion to be read.

7 schematically illustrates a configuration of an autonomous fundus image photographing and reading device according to an embodiment of the present invention.

Referring to FIG. 7, the self fundus image photographing and reading device 200 includes a display unit 210, a first sensor unit 220, a second sensor unit 230, a camera unit 240, and a computing unit C ). The computing unit C may include a control unit 250, a memory 260, a storage 270, an input/output interface 284, and the like. The controller 150 may include one or more processors.

Since the display unit 210, the first sensor unit 220, the second sensor unit 230, and the camera unit 240 are the same as those described with reference to FIGS. 1 to 6, description thereof will be omitted. I will do it.

The control unit 250 performs the same function as the control unit 150 described above with reference to FIGS. 1 to 6 and additionally, reads the lesion from the fundus image determined as the fundus image to be read.

At this time, the control unit 250 may apply a machine learning algorithm to read the lesion from the fundus image to support the diagnosis of the medical staff. Since the content of the machine learning algorithm is the same as described above, the description will be omitted.

8 schematically illustrates a configuration of an auto fundus image photographing and reading device according to another embodiment of the present invention.

Referring to FIG. 8, the self fundus image capturing and reading device 300 includes a first display unit 310, a first sensor unit 320, a second sensor unit 330, a camera unit 340, and a computing unit ( C) and a second display unit 390 may be included.

The computing unit C may include a control unit 350, a memory 360, a storage 370, an input/output interface 384, and the like. The control unit 350 may include one or more processors.

The first display unit 310, the first sensor unit 320, the second sensor unit 330, the camera unit 340, and the control unit 350 are the same as those described with reference to FIG. 7. Therefore, the description thereof will be omitted.

The second display unit 390 may display the read result when the control unit 350 reads the lesion from the fundus image to be read by the subject by applying a machine learning algorithm.

The second display unit 390 may be provided outside the self fundus image photographing and reading device 300 unlike the first display unit 310 provided inside the auto fundus image taking and reading device 300. In this way, an examinee who has finished taking a self fundus image can easily check the reading result of his fundus image.

Referring to FIG. 9, an auto fundus image capturing and reading system 1000 according to an embodiment of the present invention may include an auto fundus image capturing apparatus 1100 and a lesion reading apparatus 1200.

The self fundus image photographing apparatus 1100 includes a display unit 1100 displaying a focus image, a first sensor unit 1120 measuring a pupil's pupil state, and a second sensor unit measuring a blink of the eyelid of the examinee 1130, a camera unit 1140 photographing at least one of a pupil image of the examinee and a fundus image of the examinee, and the controller 1150.

The control unit 1150 is based on the measured value of the first sensor unit 1110, the measured value of the second sensor unit 1120, and the camera unit 1140 to photograph the fundus image of the examinee based on the pupil image of the examinee ), and if the photographed fundus image of the examinee is within a preset range of the fundus image, the photographed fundus image of the examinee may be determined as a fundus image to be read.

Descriptions of the display unit 1100, the first sensor unit 1120, the second sensor unit 1130, the camera unit 1140, and the control unit 1150 are the same as those described with reference to FIGS. 1 to 6 above. Therefore, the description will be omitted.

The lesion reading apparatus 1200 is connected to the self fundus imaging apparatus 1110 through a wired or wireless communication network. The wired/wireless communication network includes, for example, Wireless LAN (WLAN), Wireless Fidelity (WiFi) Direct, Digital Living Network Alliance (DLNA), Wireless broadband (Wibro), World Interoperability for Microwave Access (Wimax), High Speed Downlink Packet (HSDPA) Access), Long Term Evolution (LTE), Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wi- Fi (Wireless-Fidelity) and Wi-Fi Direct.

The lesion reading apparatus 1200 may receive a lesion reading target fundus image transmitted from the self fundus imaging apparatus 1110 and read a lesion from the lesion reading target fundus image.

The read result may be displayed on a display unit (not shown) provided in the lesion reading apparatus 1200. Alternatively, the read result may be transmitted to another remote device (not shown) or the self fundus imaging apparatus 1110.

Referring to FIG. 10, the self- fundus imaging apparatus 400 according to an embodiment of the present invention includes a main body 410 and a handle 420 rotatably attached to the main body 400. You can.

The handle part 420 may be provided with the first sensor part described above, but is not limited thereto. When the first sensor part is provided on the handle part, while the testee is holding the handle part 420, the tension state of the testee, that is, irregularity of the pulse, may be measured by the first sensor part.

Referring to FIG. 11, the self fundus imaging apparatus 600 according to the present invention is attached to a support device 700 having a folding arm 710.

After the test subject is positioned on the support device 700, the self fundus imaging device 600 attached to the support device 700 may be fixed to the position of the subject's eye, and then fundus imaging may be performed. At this time, the support device 700 is provided with a folding arm 710, the testee can place the self fundus imaging device 600 in the proper position of the eye.

As long as the examinee simply positions the self fundus imaging apparatus 600 at the proper position of his or her eye, a fundus image with high clarity may be captured when the pupil's pupil size is enlarged without the assistance of medical staff.

In addition, whether the photographed fundus image can be used as a fundus image to be diagnosed with a lesion is determined based on a machine learning algorithm, and reading of an ophthalmic disease with respect to a lesion diagnosis image can be performed based on a machine learning algorithm. There is an advantage.

This will be described in more detail as follows.

When the examinee places the self fundus image taking device 600 on his face, the EOG sensor of the self funding image taking device 600 tracks the entire pupil. Thereafter, the ECG sensor or the PPG sensor tracks the pupil image to check whether the pupil is expanded, and when the pupil of the subject gazes at the focus image displayed on the display unit, whether to shoot in a single mode (Dual Mode) Whether to shoot with the controller (DSP) controls the focus image, and in this state, the EOG sensor detects blinking of the eye.

When the optimal conditions for fundus image shooting in such a tracking or detection state are satisfied, the fundus image of the subject is photographed, and the captured fundus image is judged to be compared with a preset fundus image using a DSP algorithm, or a fundus image through a machine learning model. The goodness of fit (Good or Poor as a fundus image to be diagnosed with a lesion) is determined.

If it is not suitable as a fundus image to be diagnosed with a lesion, the fundus image is re-photographed.

The method for photographing the fundus image using the self fundus image capturing apparatus is only an example, and the present invention is not limited thereto. As described above, satisfaction of the optimal conditions for fundus imaging may be performed simultaneously or sequentially.

In the above, specific preferred embodiments of the present invention have been described, but the present invention is not limited to the specific embodiments described above, and the technical field to which the present invention pertains without departing from the gist of the present invention claimed in the claims. Various modifications can be carried out by anyone who has ordinary knowledge in the course, and such modifications are within the scope of the claims.

Claims

A display unit displaying an image for focus;

A first sensor unit for measuring the pupil state of the subject;

A second sensor unit measuring the eye blink of the test subject;

A camera unit that photographs at least one of a pupil image of the examinee and a fundus image of the examinee; And

Based on the measured value of the first sensor unit, the measured value of the second sensor unit, and the pupil portion of the test subject to control the camera unit to shoot the fundus image of the test subject, the captured fundus image of the test subject is preset And a controller configured to determine the fundus image of the examinee as the target image of the lesion to be read when within the information range.
According to claim 1,

The pupil fundus imaging apparatus according to claim 1, wherein the pupil state of the examinee is measured by the pulse of the examinee.
According to claim 1,

The first sensor unit is an ECG (Electro-CardioGram) sensor or PPG (Pulse-PlethysmoGram) sensor characterized in that it comprises a self fundus image taking device.
According to claim 1,

The second sensor unit,

At least one sensor for the left eye and the sensor for the right eye; Self fundus image capturing apparatus comprising a.
According to claim 1,

The second sensor unit self- fundus imaging device, characterized in that it comprises an EOG (Electro OculoGram) sensor.
According to claim 1,

The focus image is a focusing image for the left eye and a focusing image for the right eye.
According to claim 1,

The control unit photographs a first fundus image of a fundus image of the left eye and a fundus image of the right eye of the examinee first, and then controls the camera unit to capture the remaining fundus image.
The method of claim 6,

The control unit controls the image for focus on the left eye and the image for focus on the right eye so that the image for focus on the left eye and the image for focus on the right eye displayed on the display unit are separated from each other. Device.
The method of claim 8,

The control unit controls the camera unit to simultaneously photograph the fundus image of the left eye and the fundus image of the right eye of the examinee.
According to claim 1,

When the pupil size of the pupil image of the pupil image photographed by the camera unit is greater than or equal to a pupil size of a preset pupil image, and there is no blink of the subject by measurement of the second sensor unit, the subject Self fundus image taking device, characterized in that for controlling the camera unit to shoot the fundus image.
The method of claim 10,

The control unit based on the machine learning algorithm or DSP algorithm, self fundus image characterized in that it determines whether the pupil size in the pupil image of the subject photographed by the camera unit is greater than or equal to the pupil size of a preset pupil image Shooting device.
According to claim 1,

The control unit is based on the machine learning algorithm, self- fundus image taking device, characterized in that for determining whether the fundus image of the subject to be photographed is within a preset range of the fundus image.
The method of claim 12,

The control unit controls the camera unit to re-photograph the fundus image of the subject when the photographed fundus image of the subject is outside a preset range of the fundus image.
The method of claim 12,

The control unit determines if the photographed fundus image of the examinee is within a preset range of the fundus image, and determines the photographed fundus image of the examinee as a fundus image to be read as a lesion.
A first display unit displaying an image for focus;

A first sensor unit for measuring the pupil state of the subject;

A second sensor unit measuring the eye blink of the test subject;

A camera unit that photographs at least one of a pupil image of the examinee and a fundus image of the examinee; And

Based on the measured value of the first sensor unit, the measured value of the second sensor unit, and the pupil portion of the subject to control the camera unit to shoot the fundus image of the test subject, the captured fundus image of the test subject is preset And a controller configured to determine a fundus image of the examinee, which is photographed when the information is within the information range, as a fundus image to be read by a lesion, and to read a lesion from the fundus image to be read.
The method of claim 15,

The control unit based on the machine learning algorithm, characterized in that to determine whether the photographed fundus image of the subject is within a preset range of the fundus image self-fundus image taking and reading device.
The method of claim 15,

The control unit based on the machine learning algorithm, self- fundus image taking and reading device, characterized in that for reading the lesion from the fundus image to be read the lesion.
The method of claim 15,

And a second display unit displaying a result of reading the lesion from the fundus image to be read by the subject to be examined.
Self fundus imaging device; And

It includes; a lesion reading device for reading a lesion from the fundus image to be read from the lesion fundus image received from the self fundus imaging device;

The self fundus imaging device,

A display unit displaying an image for focus;

A first sensor unit for measuring the pupil state of the subject;

A second sensor unit measuring the eye blink of the test subject;

A camera unit that photographs at least one of a pupil image of the examinee and a fundus image of the examinee; And

Based on the measured value of the first sensor unit, the measured value of the second sensor unit, and the pupil portion of the subject to control the camera unit to shoot the fundus image of the test subject, the captured fundus image of the test subject is preset And a controller configured to determine the fundus image of the examinee as the target image of the lesion to be read when within the information range.
The method of claim 19,

The control unit is based on the machine learning algorithm, self-fundus image capture and reading system characterized in that it determines whether or not the photographed fundus image of the subject is within a preset range of the fundus image.
The method of claim 19,

The lesion reading apparatus is based on a machine learning algorithm, characterized in that for reading the lesion from the fundus image to be read, the fundus image taking and reading system.