WO2022062436A1 - Amblyopia training method, apparatus and device, storage medium, and program product - Google Patents

Amblyopia training method, apparatus and device, storage medium, and program product Download PDF

Info

Publication number
WO2022062436A1
WO2022062436A1 PCT/CN2021/095234 CN2021095234W WO2022062436A1 WO 2022062436 A1 WO2022062436 A1 WO 2022062436A1 CN 2021095234 W CN2021095234 W CN 2021095234W WO 2022062436 A1 WO2022062436 A1 WO 2022062436A1
Authority
WO
WIPO (PCT)
Prior art keywords
amblyopia
image
training
eye
amblyopic
Prior art date
Application number
PCT/CN2021/095234
Other languages
French (fr)
Chinese (zh)
Inventor
何庭波
李江
张朋
李瑞华
刘迎春
臧磊
郭帮辉
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority to CN202011019488.4A priority Critical patent/CN114255204A/en
Priority to CN202011019488.4 priority
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022062436A1 publication Critical patent/WO2022062436A1/en

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0012Biomedical image inspection
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61FFILTERS IMPLANTABLE INTO BLOOD VESSELS; PROSTHESES; DEVICES PROVIDING PATENCY TO, OR PREVENTING COLLAPSING OF, TUBULAR STRUCTURES OF THE BODY, e.g. STENTS; ORTHOPAEDIC, NURSING OR CONTRACEPTIVE DEVICES; FOMENTATION; TREATMENT OR PROTECTION OF EYES OR EARS; BANDAGES, DRESSINGS OR ABSORBENT PADS; FIRST-AID KITS
    • A61F9/00Methods or devices for treatment of the eyes; Devices for putting-in contact lenses; Devices to correct squinting; Apparatus to guide the blind; Protective devices for the eyes, carried on the body or in the hand
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H5/00Exercisers for the eyes
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • G06T3/40Scaling the whole image or part thereof
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61HPHYSICAL THERAPY APPARATUS, e.g. DEVICES FOR LOCATING OR STIMULATING REFLEX POINTS IN THE BODY; ARTIFICIAL RESPIRATION; MASSAGE; BATHING DEVICES FOR SPECIAL THERAPEUTIC OR HYGIENIC PURPOSES OR SPECIFIC PARTS OF THE BODY
    • A61H2205/00Devices for specific parts of the body
    • A61H2205/02Head
    • A61H2205/022Face
    • A61H2205/024Eyes
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30041Eye; Retina; Ophthalmic

Abstract

An amblyopia training method, apparatus and device, a storage medium, and a program product, relating to the technical field of image processing. The method comprises: determining the dominant eye and the amblyopic eye of the eyes of a user; determining an amblyopia type of the amblyopic eye, the amblyopia type comprising strabismic amblyopia or anisometropic amblyopia; and when the amblyopia type of the amblyopic eye is determined as the strabismic amblyopia, performing homography transformation processing on an amblyopia training image to perform an amblyopia training, and when the amblyopia type of the amblyopic eye is determined as the anisometropic amblyopia, performing image size adjustment processing on the amblyopia training image to perform the amblyopia training. In the present method, the amblyopia type of the amblyopic eye is determined, and according to the amblyopia type of the amblyopic eye, an amblyopia training is performed after the amblyopia training image is processed. That is to say, by distinguishing between the amblyopia types, a targeted training can be performed according to different symptoms, thus improving the effect of the amblyopia training.

Description

弱视训练方法、装置、设备、存储介质及程序产品Amblyopia training method, device, equipment, storage medium and program product
本申请要求于2020年09月24日提交的申请号为202011019488.4、发明名称为“弱视训练方法、装置、设备及存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese patent application with the application number 202011019488.4 and the invention titled "Amblyopia training method, device, equipment and storage medium" filed on September 24, 2020, the entire contents of which are incorporated into this application by reference .
技术领域technical field
本申请实施例涉及图像处理技术领域,特别涉及一种弱视训练方法、装置、设备、存储介质及程序产品。The embodiments of the present application relate to the technical field of image processing, and in particular, to an amblyopia training method, apparatus, device, storage medium, and program product.
背景技术Background technique
弱视是指眼球无器质性病变但最佳矫正视力低于正常视力的一种眼病。通常由单眼斜视、屈光参差、高度屈光不正、视觉剥夺等原因引起,如不及时治疗可引导弱视加重,甚至失明。所以,亟需一种弱视训练方法来矫正弱视眼的视力。Amblyopia is an eye disease in which there is no organic disease in the eyeball but the best corrected visual acuity is lower than normal vision. It is usually caused by monocular strabismus, anisometropia, high refractive error, visual deprivation, etc. If not treated in time, it can lead to aggravation of amblyopia and even blindness. Therefore, an amblyopia training method is urgently needed to correct the vision of amblyopic eyes.
相关技术提出了一种弱视训练仪,该弱视训练仪能够显示双眼视场内容,双眼视场内容中包含运动目标。该弱视训练仪能够跟踪用户左眼和右眼分别对运动目标的注视轨迹,并根据左眼注视轨迹和右眼注视轨迹分别与运动目标的实际运动轨迹之间的接近程度,确定主视眼和弱视眼。然后,对于弱视眼,通过多次重复显示包含运动目标的视场内容,并记录弱视眼注视运动目标的注视轨迹与运动目标的实际运动轨迹的接近程度。The related art proposes an amblyopia training device, which can display binocular visual field content, and the binocular visual field content includes a moving target. The amblyopia training instrument can track the gaze trajectories of the user's left eye and right eye to the moving target respectively, and determine the dominant eye and the Amblyopic eye. Then, for the amblyopic eye, the content of the field of view containing the moving object is displayed repeatedly, and the closeness of the gaze track of the amblyopic eye to the moving object's actual movement track is recorded.
然而,对于不同成因或者不同程度的弱视患者,按照上述的弱视训练仪进行训练之后,训练效果有的不是很理想。However, for patients with different causes or degrees of amblyopia, after training according to the above-mentioned amblyopia training device, the training effect is not very satisfactory.
发明内容SUMMARY OF THE INVENTION
本申请实施例提供了一种弱视训练方法、装置、设备、存储介质及程序产品,可以解决相关技术中训练效果不理想的问题。技术方案如下:The embodiments of the present application provide an amblyopia training method, device, equipment, storage medium and program product, which can solve the problem of unsatisfactory training effect in the related art. The technical solution is as follows:
第一方面,提供了一种弱视训练方法,应用于弱视训练设备,该方法包括:确定用户的主视眼和弱视眼,确定弱视眼的弱视类型,弱视类型包括斜视性弱视和屈光参差性弱视,当弱视眼的弱视类型被确定为斜视性弱视时,对弱视训练图像进行单应性变换处理来进行弱视训练,当弱视眼的弱视类型被确定为屈光参差性弱视时,对弱视训练图像进行图像尺寸调整处理来进行弱视训练。In a first aspect, an amblyopia training method is provided, which is applied to an amblyopia training device, the method comprising: determining a user's dominant eye and amblyopic eye, and determining amblyopia type of the amblyopic eye, the amblyopia type includes strabismus amblyopia and anisometropia Amblyopia, when the type of amblyopia of the amblyopic eye is determined to be strabismus amblyopia, the amblyopia training image is subjected to homography transformation processing for amblyopia training, when the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia, the amblyopia training The images are subjected to image resizing processing for amblyopia training.
在本申请实施例中,通过确定弱视眼的弱视类型,按照弱视眼的弱视类型,对弱视训练图像处理后来进行弱视训练。也就是说,通过区分弱视类型,从而按照不同的症状进行针对性训练,能够提高弱视训练效果。In the embodiment of the present application, by determining the amblyopia type of the amblyopic eye, the amblyopia training image is processed and then amblyopia training is performed according to the amblyopia type of the amblyopic eye. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved.
其中,确定弱视眼的弱视类型的实现过程包括:在主视眼对应的显示区域中显示第一测试图像,在弱视眼对应的显示区域中显示第二测试图像,通过眼动跟踪的方式,确定主视眼在显示的第一测试图像中的凝视位置的坐标,以及弱视眼在显示的第二测试图像中的凝视位 置的坐标,得到第一凝视位置坐标和第二凝视位置坐标。根据第一凝视位置坐标和第二凝视位置坐标,确定弱视眼的弱视类型。Wherein, the realization process of determining the type of amblyopia of the amblyopic eye includes: displaying the first test image in the display area corresponding to the dominant eye, displaying the second test image in the display area corresponding to the amblyopic eye, and determining by eye tracking The coordinates of the gaze position of the dominant eye in the displayed first test image and the coordinates of the gaze position of the amblyopic eye in the displayed second test image are used to obtain the first gaze position coordinates and the second gaze position coordinates. According to the coordinates of the first gaze position and the coordinates of the second gaze position, the type of amblyopia of the amblyopic eye is determined.
作为一种示例,通过眼动跟踪的方式,确定主视眼在显示的第一测试图像中的凝视位置的坐标的实现过程为:确定位置变换矩阵,通过眼动跟踪的方式,确定主视眼的眼球位置坐标,然后将主视眼的眼球位置坐标与位置变换矩阵相乘,得到主视眼在第一测试图像中的凝视位置的坐标,也即是,第一凝视位置坐标。同理,通过眼动跟踪的方式,确定弱视眼在显示的第二测试图像中的凝视位置的坐标的实现过程为:确定位置变换矩阵,通过眼动跟踪的方式,确定弱视眼的眼球位置坐标,然后将弱视眼的眼球位置坐标与位置变换矩阵相乘,得到弱视眼在第二测试图像中的凝视位置的坐标,也即是,第二凝视位置坐标。As an example, the realization process of determining the coordinates of the gaze position of the dominant eye in the displayed first test image by means of eye tracking is as follows: determining a position transformation matrix, and determining the dominant eye by means of eye tracking Then multiply the eyeball position coordinates of the dominant eye with the position transformation matrix to obtain the coordinates of the gaze position of the dominant eye in the first test image, that is, the first gaze position coordinates. Similarly, the realization process of determining the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking is as follows: determining the position transformation matrix, and determining the eyeball position coordinates of the amblyopic eye by means of eye tracking. , and then multiply the eyeball position coordinates of the amblyopic eye by the position transformation matrix to obtain the coordinates of the gaze position of the amblyopic eye in the second test image, that is, the second gaze position coordinates.
其中,位置变换矩阵是指眼球位置与图像中的目标点的位置之间的变换矩阵。The position transformation matrix refers to the transformation matrix between the position of the eyeball and the position of the target point in the image.
作为一种示例,根据第一凝视位置坐标和第二凝视位置坐标,确定弱视眼的弱视类型的实现过程为:根据第一凝视位置坐标和第二凝视位置坐标,确定双眼偏差信息,双眼偏差信息是指弱视眼的视线方向与主视眼的视线方向之间的偏差信息。如果双眼偏差信息大于或等于第一阈值,则确定弱视眼的弱视类型为斜视性弱视。如果双眼偏差信息小于第一阈值,则确定弱视眼的弱视类型为屈光参差性弱视。As an example, the realization process of determining the amblyopia type of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position is as follows: according to the coordinates of the first gaze position and the coordinates of the second gaze position, determining the binocular deviation information, the binocular deviation information It refers to the deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye. If the binocular deviation information is greater than or equal to the first threshold, it is determined that the amblyopia type of the amblyopic eye is strabismus amblyopia. If the binocular deviation information is less than the first threshold, it is determined that the type of amblyopia of the amblyopic eye is anisometropic amblyopia.
需要说明的是,正常情况下,双眼偏差信息应该最小接近于0。也就是说,双眼偏差信息越大,说明斜视的程度越严重,双眼偏差信息越小,说明斜视的程度越轻微。所以,在确定用户的双眼中存在弱视眼的情况下,可以确定双眼偏差信息是否大于或等于第一阈值,如果双眼偏差信息大于或等于第一阈值,表征用户的弱视眼的弱视类型为斜视性弱视。如果双眼偏差信息小于第一阈值,表征用户的弱视眼的弱视类型为屈光参差性弱视。It should be noted that under normal circumstances, the binocular deviation information should be at least close to 0. That is to say, the larger the binocular deviation information, the more serious the degree of strabismus, and the smaller the binocular deviation information, the lesser the degree of strabismus. Therefore, when it is determined that there is an amblyopic eye in the user's eyes, it can be determined whether the binocular deviation information is greater than or equal to the first threshold, and if the binocular deviation information is greater than or equal to the first threshold, it indicates that the amblyopia type of the user's amblyopic eye is strabismus amblyopia. If the binocular deviation information is less than the first threshold, the amblyopia type representing the amblyopic eye of the user is anisometropic amblyopia.
也就是说,在确定用户的双眼中存在弱视眼的情况下,通过第一阈值即可确定弱视眼的弱视类型为斜视性弱视还是屈光参差性弱视。That is, when it is determined that there is an amblyopic eye in both eyes of the user, the amblyopia type of the amblyopic eye can be determined as strabismus amblyopia or anisometropic amblyopia through the first threshold.
其中,第一阈值是指参考距离范围内的任一数值,该参考距离范围是指用来区分斜视性弱视和屈光参差性弱视的一个距离范围。而且,第一阈值与弱视训练设备的角分辨率相关。比如,在弱视训练设备的显示屏的平均角分辨率为20的情况下,第一阈值为100pixel(像素)。在弱视训练设备的显示屏的平均角分辨率为30的情况下,第一阈值为150pixel。The first threshold refers to any value within a reference distance range, and the reference distance range refers to a distance range used to distinguish strabismus amblyopia and anisometropic amblyopia. Furthermore, the first threshold is related to the angular resolution of the amblyopia training device. For example, in the case where the average angular resolution of the display screen of the amblyopia training device is 20, the first threshold is 100 pixels (pixels). When the average angular resolution of the display screen of the amblyopia training device is 30, the first threshold is 150 pixels.
值得注意的是,基于上述描述,弱视通常由单眼斜视、屈光参差、高度屈光不正、视觉剥夺等原因引起,但是,对于高度屈光不正和视觉剥夺这两种类型来说,由于在生理上已经对眼睛造成了损伤,通过弱视训练已经无法矫正弱视眼的视力。所以,本申请实施例通过区分斜视性弱视和屈光参差性弱视这两种类型,来进行弱视训练,并不涉及高度屈光不正和视觉剥夺这两种弱视类型。It is worth noting that, based on the above description, amblyopia is usually caused by monocular strabismus, anisometropia, high refractive error, visual deprivation, etc. However, for the two types of high refractive error and visual deprivation, due to the physiological It has already caused damage to the eyes, and the vision of amblyopic eyes cannot be corrected through amblyopia training. Therefore, in the embodiments of the present application, amblyopia training is performed by distinguishing two types of strabismus amblyopia and anisometropic amblyopia, and does not involve the two types of amblyopia, high refractive error and visual deprivation.
其中,确定用户的双眼中的主视眼和弱视眼的实现过程为:检测用户的双眼的屈光度,将用户的双眼中屈光度低的眼睛确定为主视眼,将用户的双眼中屈光度高的眼睛确定为弱视眼。Wherein, the realization process of determining the dominant eye and the amblyopic eye in the user's two eyes is: detecting the diopter of the user's two eyes, determining the eye with a low refractive index in the user's two eyes as the dominant eye, and determining the user's eye with a high refractive power in both eyes. Determined to be amblyopic.
屈光度一般是指近视度数、远视度数或者散光度,通俗地讲,屈光度也称为视力。视力主要是指眼底视网膜成像的能力。作为一种示例,弱视训练设备的两个显示区域中分别显示视力表。在用户的双眼分别注视视力表的过程中,通过调整虚像距,来检测用户的双眼的屈光度。其中,视力表可以为终端设备投屏到弱视训练设备上的,当然,视力表也可以是弱视训练设备自身存储的,本申请实施例对此不做限定。Diopter generally refers to the degree of myopia, farsightedness or astigmatism. Visual acuity mainly refers to the ability to image the retina of the fundus. As an example, an eye chart is displayed in the two display areas of the amblyopia training device respectively. When the user's eyes are looking at the eye chart respectively, the diopter of the user's eyes is detected by adjusting the virtual image distance. The eye chart may be projected by the terminal device to the amblyopia training device. Of course, the eye chart may also be stored by the amblyopia training device itself, which is not limited in this embodiment of the present application.
在一些实施例中,对弱视训练图像进行单应性变换处理的实现过程为:在主视眼对应的显示区域中显示弱视训练图像,对弱视训练图像进行单应性变换处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。同理,对弱视训练图像进行图像尺寸调整来进行弱视训练的实现过程为:在主视眼对应的显示区域中显示弱视训练图像,对弱视训练图像进行图像尺寸调整后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。In some embodiments, the implementation process of performing homography transformation on the amblyopia training image is as follows: displaying the amblyopia training image in the display area corresponding to the dominant eye, performing homography transformation on the amblyopia training image and displaying it on the amblyopic eye In the corresponding display area, amblyopia training is performed so that the images seen by the user's eyes can be combined. Similarly, the realization process of performing amblyopia training by adjusting the image size of the amblyopia training image is as follows: displaying the amblyopia training image in the display area corresponding to the dominant eye, and adjusting the image size of the amblyopia training image and displaying it on the display corresponding to the amblyopia eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.
在另一些实施例中,为了改善弱视眼的成像能力,还可以确定主视眼对应的图像对比度和弱视眼对应的图像对比度,以使用户的双眼感知能力相同。这样,能够按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示弱视训练图像,按照弱视眼对应的图像对比度,根据弱视眼的弱视类型对弱视训练图像进行单应性变换处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。或者,按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示弱视训练图像,按照弱视眼对应的图像对比度,对弱视训练图像进行图像尺寸调整处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。In other embodiments, in order to improve the imaging ability of the amblyopic eye, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye may also be determined, so that the user's binocular perception ability is the same. In this way, the amblyopic training image can be displayed in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and the amblyopia training image can be subjected to homography transformation processing according to the image contrast corresponding to the amblyopic eye and the amblyopia type of the amblyopic eye Then, it is displayed in the display area corresponding to the amblyopic eye, so that the images seen by the user's eyes can be combined to perform amblyopia training. Or, display the amblyopic training image in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and perform image size adjustment processing on the amblyopic training image according to the image contrast corresponding to the amblyopic eye and display it on the display corresponding to the amblyopic eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.
对于上述两种实施例来说,根据弱视眼的弱视类型对弱视训练图像进行处理的操作,以及进行弱视训练的操作均相同,区别仅仅在于是否需要确定双眼各自对应的图像对比度,进而在弱视训练的过程按照双眼各自对应的图像对比度来显示弱视训练图像。接下来,以第二种实施例为例,对本申请实施例提供的弱视训练过程进行介绍。For the above two embodiments, the operation of processing the amblyopia training image according to the type of amblyopia of the amblyopic eye and the operation of performing amblyopia training are the same, the difference is only whether it is necessary to determine the image contrasts corresponding to the eyes, and then in the amblyopia training operation. The process of displaying amblyopia training images according to the respective image contrasts of the eyes. Next, the second embodiment is taken as an example to introduce the amblyopia training process provided by the embodiment of the present application.
在一些实施例中,确定主视眼对应的图像对比度和弱视眼对应的图像对比度的实现过程包括:在主视眼对应的显示区域中显示第三测试图像,在弱视眼对应的显示区域中显示第四测试图像。之后,降低显示的第三测试图像的对比度,增加显示的第四测试图像的对比度。当检测到对比度确定指令时,将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度,该对比度确定指令是用户根据显示的第三测试图像和第四测试图像反馈自身的双眼能够感知的对比度相同时触发的。In some embodiments, the implementation process of determining the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye includes: displaying a third test image in a display area corresponding to the dominant eye, displaying a third test image in a display area corresponding to the amblyopic eye Fourth test image. After that, the contrast of the displayed third test image is decreased, and the contrast of the displayed fourth test image is increased. When the contrast determination instruction is detected, the contrast of the reduced third test image is determined as the image contrast corresponding to the dominant eye, and the contrast of the increased fourth test image is determined as the image contrast corresponding to the amblyopic eye, and the contrast determination instruction is It is triggered when the user feeds back that the contrasts that his eyes can perceive are the same according to the displayed third test image and the fourth test image.
也即是,通过在弱视训练设备的两个显示区域上分别显示一个测试图像,通过用户的主观反馈来调节两个测试图像的对比度,从而使用户的双眼能够感知的对比度相同。而且,在本申请实施例中,通过降低第三测试图像的对比度,以及增加第四测试图像的对比度,来保证用户的双眼能够感知的对比度相同。也就是说,通过降低主视眼的对比度,增加弱视眼的对比度,从而在后续进行弱视训练过程中,对主视眼和弱视眼分别显示不同对比度的画面,以强化弱视眼以及抑制主视眼的成像能力,达到弱视训练的目的。That is, by displaying a test image on the two display areas of the amblyopia training device respectively, the contrast of the two test images is adjusted through the user's subjective feedback, so that the user's eyes can perceive the same contrast. Furthermore, in this embodiment of the present application, the contrast ratio of the third test image is reduced and the contrast ratio of the fourth test image is increased to ensure that the user's eyes can perceive the same contrast ratio. That is to say, by reducing the contrast of the dominant eye and increasing the contrast of the amblyopic eye, in the subsequent amblyopic training process, the dominant eye and the amblyopic eye are displayed with different contrast images respectively to strengthen the amblyopic eye and suppress the dominant eye. Imaging ability to achieve the purpose of amblyopia training.
上述实现过程是根据用户的主观反馈来确定主视眼对应的图像对比度和弱视眼对应的图像对比度,但是,用户主观反馈的对比度往往可能存在偏差。所以,在另一些实施例中,在主视眼对应的显示区域中显示多个第三测试图像,在弱视眼对应的显示区域中显示多个第四测试图像,该多个第三测试图像中包括第一运动目标,该多个第四测试图像中包括第二运动目标。这样,在将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度之前,还包括:通过眼动跟踪的方式,确定主视眼在多个第三测试图像中对第一运动目标的注视轨迹,以及弱视眼在多个第四测试图像中对第二运动目标的注视轨迹,得到第一注视轨迹和第二注视轨迹。获取第一运动目标 的实际运动轨迹和第二运动目标的实际运动轨迹,得到第一实际运动轨迹和第二实际运动轨迹。如果第一注视轨迹与第一实际运动轨迹匹配,且第二注视轨迹与第二实际运动轨迹匹配,则将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度。In the above implementation process, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye are determined according to the user's subjective feedback, but the contrast of the user's subjective feedback may often be biased. Therefore, in some other embodiments, multiple third test images are displayed in the display area corresponding to the dominant eye, and multiple fourth test images are displayed in the display area corresponding to the amblyopic eye. A first moving object is included, and a second moving object is included in the plurality of fourth test images. In this way, before determining the contrast of the reduced third test image as the image contrast corresponding to the dominant eye, and determining the increased contrast of the fourth test image as the image contrast corresponding to the amblyopic eye, the method further includes: by means of eye tracking. , determine the gaze trajectory of the dominant eye to the first moving target in a plurality of third test images, and the gaze trajectory of the amblyopic eye to the second moving target in a plurality of fourth test images, and obtain the first gaze trajectory and the second Gaze track. The actual motion trajectory of the first moving target and the actual motion trajectory of the second moving target are obtained, and the first actual motion trajectory and the second actual motion trajectory are obtained. If the first gaze trajectory matches the first actual motion trajectory, and the second gaze trajectory matches the second actual motion trajectory, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the main eye, and the contrast of the third test image after the decrease The contrast of the four test images was determined as the image contrast corresponding to the amblyopic eye.
也即是,通过眼动跟踪的方式,能够进一步准确地确定通过用户的主观反馈确定的双眼对应的图像对比度是否准确,从而避免主观反馈带来的误差。That is, by means of eye tracking, it can be further accurately determined whether the image contrast corresponding to the eyes determined by the user's subjective feedback is accurate, so as to avoid errors caused by the subjective feedback.
基于上述描述,弱视类型包括斜视性弱视和屈光参差性弱视,对于不同的弱视类型,对弱视训练图像进行处理的操作不同,接下来将分别进行介绍。Based on the above description, the types of amblyopia include strabismus amblyopia and anisometropic amblyopia. For different types of amblyopia, the operations for processing amblyopia training images are different, which will be introduced separately next.
弱视眼的弱视类型为斜视性弱视The type of amblyopia in the amblyopic eye is strabismus amblyopia
在这种情况下,弱视训练图像中包括目标对象,目标对象用于进行弱视训练。此时,按照双眼偏差信息对弱视训练图像进行单应性变换。将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像,按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示第一训练图像,按照弱视眼对应的图像对比度,在弱视眼对应的显示区域中显示第二训练图像。通过眼动跟踪的方式,确定主视眼在第一训练图像中的凝视位置,以及弱视眼在第二训练图像中的凝视位置,得到第一凝视位置和第二凝视位置。如果第一凝视位置与第一训练图像中目标对象的实际位置重合,且第二凝视位置与第二训练图像中目标对象的实际位置重合,则减小弱视训练图像的单应性变换量,返回将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像的步骤,直至弱视训练结束,或者弱视训练图像不进行单应性变换的情况下,第二凝视位置与第二训练图像中目标对象的实际位置重合为止。In this case, the target object is included in the amblyopia training image, and the target object is used for amblyopia training. At this time, homography transformation is performed on the amblyopia training image according to the binocular deviation information. The amblyopia training image before homography transformation is used as the first training image, and the amblyopia training image after homography transformation is used as the second training image. According to the image contrast corresponding to the dominant eye, in the display area corresponding to the dominant eye The first training image is displayed, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye. By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined to obtain the first gaze position and the second gaze position. If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the amount of homography transformation of the amblyopia training image, and return The step of taking the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia training ends, or the amblyopia training image is not subjected to homography transformation. In this case, the second gaze position coincides with the actual position of the target object in the second training image.
单应性变换是指一张图像到另一张图像的映射关系,通常包括平移和/或旋转。由于眼睛通常对旋转不够敏感,所以,本申请实施例中,单应性变换主要是指平移。另外,基于上述双眼偏差的运算过程,双眼偏差信息可以分为水平方向上的偏差信息和垂直方向上的偏差信息,这样,按照双眼偏差信息对弱视训练图像进行单应性变换的实现过程为:将双眼偏差信息中的水平方向上的偏差信息确定为水平方向上的平移量,将双眼偏差信息中垂直方向上的偏差信息确定为垂直方向上的平移量。之后,按照水平方向的平移量和垂直方向上的平移量,对弱视训练图像进行平移,从而实现弱视训练图像的单应性变换。A homography transformation refers to the mapping relationship of one image to another, usually including translation and/or rotation. Since eyes are generally not sensitive enough to rotation, in this embodiment of the present application, the homography transformation mainly refers to translation. In addition, based on the operation process of the above-mentioned binocular deviation, the binocular deviation information can be divided into deviation information in the horizontal direction and deviation information in the vertical direction. In this way, the realization process of performing homography transformation on the amblyopia training image according to the binocular deviation information is: The deviation information in the horizontal direction in the binocular deviation information is determined as the translation amount in the horizontal direction, and the deviation information in the vertical direction in the binocular deviation information is determined as the translation amount in the vertical direction. Then, the amblyopia training image is translated according to the translation amount in the horizontal direction and the translation amount in the vertical direction, so as to realize the homography transformation of the amblyopia training image.
对于斜视性弱视来说,由于弱视眼的视线方向与主视眼的视线方向之间存在偏差,所以需要对弱视训练图像进行单应性变换,以调整目标对象在弱视训练图像中的位置。然后在主视眼对应的显示区域中显示未调整的弱视训练图像,在弱视眼对应的显示区域中显示调整后的弱视训练图像,从而对弱视眼进行训练的过程中,用户的双眼看到的图像能够合像,也即是,双眼能够看到相同的画面。而且,双眼同时观看对应的显示区域中的弱视训练图像,能够保证双眼的立体视觉。For strabismus amblyopia, due to the deviation between the line of sight of the amblyopic eye and the line of sight of the dominant eye, it is necessary to perform homography transformation on the amblyopia training image to adjust the position of the target object in the amblyopia training image. Then, the unadjusted amblyopia training image is displayed in the display area corresponding to the dominant eye, and the adjusted amblyopia training image is displayed in the display area corresponding to the amblyopic eye. The images can be combined, that is, both eyes can see the same picture. Moreover, the two eyes simultaneously watch the amblyopia training image in the corresponding display area, which can ensure the stereoscopic vision of the two eyes.
此外,基于上面的描述,主视眼对应的图像对比度和弱视眼对应的图像对比度可能不同,这样,按照主视眼对应的图像对比度显示第一训练图像,按照弱视眼对应的图像对比度显示第二训练图像,在改善弱视眼的成像能力的同时,还能确保用户的舒适度,使用户更容易接受训练方法及训练过程,改善训练效果,提高用户粘度。In addition, based on the above description, the image contrast corresponding to the dominant eye may be different from the image contrast corresponding to the amblyopic eye. In this way, the first training image is displayed according to the image contrast corresponding to the dominant eye, and the second training image is displayed according to the image contrast corresponding to the amblyopic eye. The training images not only improve the imaging ability of the amblyopic eye, but also ensure the comfort of the user, make it easier for the user to accept the training method and training process, improve the training effect, and increase the user's viscosity.
弱视眼的弱视类型为屈光参差性弱视The type of amblyopia in the amblyopic eye is anisometropic amblyopia
在这种情况下,弱视训练图像中包括目标对象,目标对象用于进行弱视训练。此时,对 弱视训练图像中目标对象的位置和/或尺寸进行多次调整,得到多个训练图像,确定弱视眼相对于主视眼的图像缩放比例,按照该图像缩放比例,对该多个训练图像的尺寸进行缩放。将缩放前的多个训练图像作为多个第三训练图像,将缩放后的多个训练图像作为多个第四训练图像,按照主视眼对应的图像对比度,在主视眼对应的显示区域中依次显示多个第三训练图像,按照弱视眼对应的图像对比度,在弱视眼对应的显示区域中依次显示多个第四训练图像,且多个第三训练图像和多个第四训练图像的显示顺序和切换频率相同。In this case, the target object is included in the amblyopia training image, and the target object is used for amblyopia training. At this time, the position and/or size of the target object in the amblyopic training image is adjusted multiple times to obtain multiple training images, the image scaling ratio of the amblyopic eye relative to the dominant eye is determined, and according to the image scaling ratio, the multiple training images are obtained. The dimensions of the training images are scaled. Taking the multiple training images before scaling as multiple third training images, and using the multiple training images after scaling as multiple fourth training images, in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye Display a plurality of third training images in sequence, and sequentially display a plurality of fourth training images in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, and display a plurality of third training images and a plurality of fourth training images The sequence and switching frequency are the same.
对于屈光参差性弱视来说,由于弱视眼的屈光度高于主视眼的屈光度,所以弱视眼看到的图像的尺寸与主视眼看到的图像的尺寸可能不同,因此,在对弱视训练图像中目标对象的位置和/或尺寸进行多次调整,得到多个训练图像之后,需要确定弱视眼相对于主视眼的图像缩放比例,进而按照该图像缩放比例对该多个训练图像的尺寸进行缩放,以使主视眼和弱视眼看到的图像能够合像,也即是,双眼能够看到相同的画面。For anisometropic amblyopia, since the diopter of the amblyopic eye is higher than that of the dominant eye, the size of the image seen by the amblyopic eye may be different from that seen by the dominant eye. Therefore, in the training images for amblyopia After adjusting the position and/or size of the target object multiple times to obtain multiple training images, it is necessary to determine the image scaling ratio of the amblyopic eye relative to the dominant eye, and then scale the sizes of the multiple training images according to the image scaling ratio. , so that the images seen by the dominant eye and the amblyopic eye can be combined, that is, both eyes can see the same picture.
其中,确定弱视眼相对于主视眼的图像缩放比例的实现过程为:在主视眼对应的显示区域中显示第五测试图像,在弱视眼对应的显示区域中显示第六测试图像,第五测试图像包括第一测试目标,第六测试图像包括第二测试目标,第一测试目标和第二测试目标的比例相同。缩放第六测试图像的比例。当检测到比例确定指令时,确定第一测试目标与缩放后的第二测试目标之间的比例,得到图像缩放比例,该比例确定指令是用户反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时触发的。Wherein, the realization process of determining the image scaling ratio of the amblyopic eye relative to the dominant eye is: displaying the fifth test image in the display area corresponding to the dominant eye, displaying the sixth test image in the display area corresponding to the amblyopic eye, and displaying the fifth test image in the display area corresponding to the dominant eye. The test image includes a first test target, the sixth test image includes a second test target, and the ratio of the first test target and the second test target is the same. Scale the sixth test image. When the scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio. Fired when the target and the second test target.
也即是,对于同一个眼睛来说,第五测试图像中的第一测试目标和第六测试图像中的第二测试目标最初的比例相同。然后,在主视眼注视第五测试图像,弱视眼注视第六测试图像的过程中,通过缩放第六测试图像的比例,由用户反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时,通过第一测试目标的尺寸和缩放后的第二测试目标的尺寸,确定图像缩放比例。That is, for the same eye, the first test object in the fifth test image and the second test object in the sixth test image are initially in the same proportion. Then, when the dominant eye is looking at the fifth test image and the amblyopic eye is looking at the sixth test image, by scaling the ratio of the sixth test image, the user feedbacks that both eyes of the user can see the first test target and the sixth test image with the same proportions. During the second test target, the image scaling ratio is determined by the size of the first test target and the scaled size of the second test target.
本申请实施例按照不同的弱视类型进行弱视训练之后,还可以反馈弱视训练效果,从而便于用户针对性调整训练计划。接下来对其中的两种反馈弱视训练效果的方式进行介绍。After performing amblyopia training according to different types of amblyopia in the embodiment of the present application, the effect of amblyopia training can also be fed back, so as to facilitate the user to adjust the training plan in a targeted manner. Next, two of the ways to feedback the effect of amblyopia training will be introduced.
第一种方式,通过眼动跟踪的方式,确定弱视眼在弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置。如果弱视眼凝视位置与弱视眼对应的显示区域显示的图像中目标对象的实际位置重合,则将弱视眼对应的显示区域显示的图像中目标对象的显示方式修改为参考显示方式,以指示弱视训练效果。In the first manner, the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye is determined by means of eye tracking, and the gaze position of the amblyopic eye is obtained. If the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, modify the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye to the reference display mode to indicate amblyopia training Effect.
其中,参考显示方式可以是指高亮、变色等方式,本申请实施例对此不做限定。The reference display mode may refer to modes such as highlighting, discoloration, and the like, which are not limited in this embodiment of the present application.
第二种方式,通过眼动跟踪的方式,确定弱视眼在弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置。根据弱视眼凝视位置和目标对象的实际位置,绘制弱视训练曲线,以指示弱视训练效果。In the second method, the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye is determined by means of eye tracking, and the gaze position of the amblyopic eye is obtained. According to the gaze position of the amblyopic eye and the actual position of the target object, the amblyopia training curve is drawn to indicate the amblyopia training effect.
作为一种示例,统计本次弱视训练周期内的训练总次数,以及本次弱视训练周期内弱视眼凝视位置与目标对象的实际位置重合的次数与训练总次数的百分比。然后,结合历史的多次弱视训练周期的数据来绘制弱视训练曲线。也即是,将多个弱视训练周期作为横轴,将每个弱视训练周期内确定的百分比作为纵轴,绘制弱视训练曲线。As an example, count the total training times in this amblyopia training cycle, and the percentage of the times that the gaze position of the amblyopic eye coincides with the actual position of the target object in the current amblyopia training cycle and the total training times. Then, the amblyopia training curve is drawn by combining the data of multiple amblyopia training cycles in history. That is, taking multiple amblyopia training cycles as the horizontal axis, and taking the percentage determined in each amblyopia training cycle as the vertical axis, the amblyopia training curve is drawn.
需要说明的是,上述两种方式可以单独使用,也可以结合使用,本申请实施例对此不做限定。当然,实际应用中,还可以通过其他的方式来反馈弱视训练效果。比如,在每个弱视训练周期进行弱视训练之后,可以检测弱视眼的屈光度,然后,以多个弱视训练周期作为横 轴,以弱视眼的屈光度为纵轴,绘制弱视训练曲线。It should be noted that the above two manners may be used alone or in combination, which are not limited in the embodiments of the present application. Of course, in practical applications, the effect of amblyopia training can also be fed back in other ways. For example, after the amblyopia training is performed in each amblyopia training cycle, the diopter of the amblyopic eye can be detected, and then the amblyopia training curve is drawn with multiple amblyopic training cycles as the horizontal axis and the diopter of the amblyopic eye as the vertical axis.
第二方面,提供了一种弱视训练装置,弱视训练装置具有实现上述第一方面中弱视训练方法行为的功能。弱视训练装置包括至少一个模块,该至少一个模块用于实现上述第一方面所提供的弱视训练方法。In a second aspect, an amblyopia training device is provided, and the amblyopia training device has the function of implementing the behavior of the amblyopia training method in the first aspect. The amblyopia training device includes at least one module, and the at least one module is used to implement the amblyopia training method provided in the first aspect.
第三方面,提供了一种弱视训练设备,弱视训练设备包括处理器和存储器,存储器用于存储执行上述第一方面所提供的弱视训练方法的程序,以及存储用于实现上述第一方面所提供的弱视训练方法所涉及的数据。处理器被配置为用于执行存储器中存储的程序。弱视训练设备还可以包括通信总线,该通信总线用于该处理器与存储器之间建立连接。In a third aspect, an amblyopia training device is provided, the amblyopia training device includes a processor and a memory, and the memory is used for storing a program for executing the amblyopia training method provided in the first aspect, and for implementing the amblyopia training method provided in the first aspect. The data involved in the amblyopia training method. The processor is configured to execute programs stored in the memory. The amblyopia training device may also include a communication bus for establishing a connection between the processor and the memory.
第四方面,提供了一种计算机可读存储介质,计算机可读存储介质中存储有指令,当其在计算机上运行时,使得计算机执行上述第一方面所述的弱视训练方法。In a fourth aspect, a computer-readable storage medium is provided, where instructions are stored in the computer-readable storage medium, when the computer-readable storage medium runs on a computer, the computer executes the amblyopia training method described in the first aspect.
第五方面,提供了一种包含指令的计算机程序产品,当其在计算机上运行时,使得计算机执行上述第一方面的弱视训练方法。In a fifth aspect, there is provided a computer program product comprising instructions, which, when run on a computer, cause the computer to execute the amblyopia training method of the above-mentioned first aspect.
上述第二方面、第三方面、第四方面和第五方面所获得的技术效果与第一方面中对应的技术手段获得的技术效果近似,在这里不再赘述。The technical effects obtained by the second aspect, the third aspect, the fourth aspect and the fifth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here.
本申请实施例提供的技术方案至少可以带来以下有益效果:The technical solutions provided in the embodiments of the present application can at least bring the following beneficial effects:
在本申请实施例中,通过确定弱视眼的弱视类型,按照弱视眼的弱视类型,对弱视训练图像进行不同方式的处理后来进行弱视训练。也就是说,通过区分弱视类型,从而按照不同的症状进行针对性训练,能够提高弱视训练效果。In the embodiment of the present application, by determining the type of amblyopia of the amblyopic eye, the amblyopia training image is processed in different ways according to the type of amblyopia of the amblyopic eye, and then amblyopia training is performed. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved.
附图说明Description of drawings
图1是本申请实施例提供的一种弱视训练系统的架构图;Fig. 1 is the architecture diagram of a kind of amblyopia training system provided by the embodiment of the present application;
图2是本申请实施例提供的一种弱视训练方法的流程图;2 is a flowchart of a method for amblyopia training provided by an embodiment of the present application;
图3是本申请实施例提供的一种确定出的多个双眼偏差信息的示意图;3 is a schematic diagram of a plurality of binocular deviation information determined according to an embodiment of the present application;
图4是本申请实施例提供的一种对斜视性弱视进行训练的示意图;4 is a schematic diagram of training strabismus amblyopia provided by an embodiment of the present application;
图5是本申请实施例提供的一种斜视性弱视恢复到正视的示意图;Fig. 5 is a kind of schematic diagram of strabismus amblyopia returning to front view provided by an embodiment of the present application;
图6是本申请实施例提供的一种按照第一种方式对弱视训练图像进行调整的示意图;6 is a schematic diagram of adjusting amblyopia training images according to the first method provided by an embodiment of the present application;
图7是本申请实施例提供的一种按照第一种方式显示的弱视训练图像的变化示意图;7 is a schematic diagram of a change in amblyopia training images displayed according to the first method provided by an embodiment of the present application;
图8是本申请实施例提供的一种按照第二种方式对弱视训练图像进行调整的示意图;8 is a schematic diagram of adjusting an amblyopia training image according to a second method provided by an embodiment of the present application;
图9是本申请实施例提供的一种按照第二种方式显示的弱视训练图像的变化示意图;9 is a schematic diagram of a change in amblyopia training images displayed in a second manner provided by an embodiment of the present application;
图10是本申请实施例提供的一种按照第三种方式对弱视训练图像进行调整的示意图;10 is a schematic diagram of adjusting amblyopia training images according to a third method provided by an embodiment of the present application;
图11是本申请实施例提供的一种按照第三种方式显示的弱视训练图像的变化示意图;11 is a schematic diagram of a change in amblyopia training images displayed in a third manner provided by an embodiment of the present application;
图12是本申请实施例提供的一种按照第四种方式对弱视训练图像进行调整的示意图;12 is a schematic diagram of adjusting an amblyopia training image according to a fourth method provided by an embodiment of the present application;
图13是本申请实施例提供的一种按照第四种方式显示的弱视训练图像的变化示意图;13 is a schematic diagram of a change in an amblyopia training image displayed in a fourth manner according to an embodiment of the present application;
图14是本申请实施例提供的一种在双眼显示测试图像确定图像缩放比例的示意图;14 is a schematic diagram of determining an image scaling ratio by displaying a test image on both eyes according to an embodiment of the present application;
图15是本申请实施例提供的一种双眼合像的示意图;15 is a schematic diagram of a binocular combined image provided by an embodiment of the present application;
图16是本申请实施例提供的第一种指示弱视训练效果的示意图;16 is a schematic diagram of the first indication of amblyopia training effect provided by an embodiment of the present application;
图17是本申请实施例提供的第二种指示弱视训练效果的示意图;17 is a schematic diagram of a second indication of amblyopia training effect provided by an embodiment of the present application;
图18是本申请实施例提供的第三种指示弱视训练效果的示意图;18 is a schematic diagram of a third indication amblyopia training effect provided by an embodiment of the present application;
图19是本申请实施例提供的一种弱视训练装置的结构示意图;19 is a schematic structural diagram of an amblyopia training device provided by an embodiment of the present application;
图20是本申请实施例提供的一种弱视训练设备的结构示意图;20 is a schematic structural diagram of an amblyopia training device provided by an embodiment of the present application;
图21是本申请实施例提供的一种终端的结构示意图;FIG. 21 is a schematic structural diagram of a terminal provided by an embodiment of the present application;
图22是本申请实施例提供的另一种终端的结构示意图。FIG. 22 is a schematic structural diagram of another terminal provided by an embodiment of the present application.
具体实施方式detailed description
为使本申请实施例的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.
在对本申请实施例提供的弱视训练方法进行详细地解释说明之前,先对本申请实施例涉及的术语和实施环境进行介绍。Before the amblyopia training method provided by the embodiments of the present application is explained in detail, the terms and implementation environments involved in the embodiments of the present application are introduced.
首先,对本申请实施例涉及的术语进行介绍。First, the terms involved in the embodiments of the present application are introduced.
弱视:是指眼球无器质性病变但最佳矫正视力低于正常视力的一种眼病。通常由单眼斜视、屈光参差、高度屈光不正、视觉剥夺等原因引起。Amblyopia: refers to an eye disease in which the eyeball has no organic lesions but the best corrected visual acuity is lower than normal vision. It is usually caused by monocular strabismus, anisometropia, high refractive error, and visual deprivation.
单眼斜视:是指中枢管制失调,眼外肌力量不平衡,双眼不能同时注视目标,视轴呈分离状态,其中一眼注视目标,另一眼偏离目标的一种现象。也可以理解为,因眼位偏斜后引起异常的双眼相互作用,斜视眼的黄斑中心窝接受的不同物象(混淆视)受到抑制,导致斜视眼的最佳矫正视力低于正常视力的一种现象。Monocular strabismus: refers to the imbalance of central control, the imbalance of extraocular muscle strength, the inability of both eyes to look at the target at the same time, the visual axis is in a state of separation, one of the eyes is looking at the target, and the other eye is deviating from the target. It can also be understood that due to the abnormal binocular interaction caused by the deviation of the eye position, the different objects (confusion) received by the fovea of the strabismus are inhibited, resulting in the best corrected visual acuity of the strabismus being lower than normal vision. Phenomenon.
屈光参差:是指双眼的屈光度存在参差。其中,屈光度较高的一个眼睛会形成弱视。通常情况下,屈光参差性弱视为单眼性弱视。anisometropia: refers to the difference in the refractive power of the eyes. Among them, amblyopia is formed in the eye with the higher diopter. Typically, anisometropia is considered monocular amblyopia.
高度屈光不正:是指眼睛在不使用调节时,平行光线通过眼睛的屈光作用后,不能在视网膜上形成清晰的物像,而在视网膜前或后方成像。主要是指远视性屈光不正或高度散光,会因双眼的物象模糊从而引起形觉剥夺。通常情况下,高度屈光不正会造成双眼性弱视。High refractive error: refers to the fact that when the eye does not use adjustment, after the parallel light passes through the refraction of the eye, it cannot form a clear image on the retina, but images in front or behind the retina. Mainly refers to hyperopic refractive error or high astigmatism, which can cause form deprivation due to blurred objects in both eyes. Often, high refractive errors cause binocular amblyopia.
视觉剥夺:是指在不损害眼睛组织结构的情况下阻止视觉信息传人的一种现象。也可以理解为,在视觉关键期内由于屈光间质混淆(角膜白斑、白内障、玻璃体炎症或积血),完全性上睑下垂,造成眼睛视力下降的一种现象。Visual deprivation: refers to a phenomenon that prevents the transmission of visual information to humans without damaging the tissue structure of the eye. It can also be understood as a phenomenon in which complete ptosis is caused due to the confusion of refractive interstitials (leukoplakia, cataract, vitreous inflammation or blood accumulation) and complete ptosis during the critical period of vision.
主视眼:是指屈光度较低的眼睛,与弱视眼为相对概念。也即是,用户的双眼中,屈光度相对较低的眼睛为主视眼。Dominant eye: refers to the eye with a lower refractive power, which is a relative concept to the amblyopic eye. That is, among the eyes of the user, the eye with a relatively low refractive power is the dominant eye.
弱视眼:是指屈光度较高的眼睛,与主视眼为相对概念。也即是,用户的双眼中,屈光度相对较高的眼睛为弱视眼。Amblyopic eye: refers to the eye with higher refractive power, which is a relative concept to the dominant eye. That is, among the eyes of the user, the eye with relatively high refractive power is the amblyopic eye.
眼动跟踪:是指通过测量眼睛的凝视位置或者眼球相对头部的运动而实现对眼球运动的追踪,最常用的手段是通过视频拍摄设备来获取眼睛的位置。Eye tracking: refers to the tracking of eye movement by measuring the gaze position of the eye or the movement of the eye relative to the head. The most commonly used method is to obtain the position of the eye through a video shooting device.
其次,对本申请实施例涉及的实施环境进行介绍。Next, the implementation environment involved in the embodiments of the present application is introduced.
请参考图1,图1是本申请实施例提供的一种弱视训练系统的架构图。该系统包括终端 设备101和弱视训练设备102,终端设备101与弱视训练设备102之间通过无线或有线的方式进行通信。Please refer to FIG. 1 . FIG. 1 is an architecture diagram of an amblyopia training system provided by an embodiment of the present application. The system includes a terminal device 101 and an amblyopia training device 102, and communication between the terminal device 101 and the amblyopia training device 102 is performed in a wireless or wired manner.
终端设备101用于显示弱视训练图像,并将显示的弱视训练图像投屏到弱视训练设备102上。弱视训练设备102包括两个显示区域,这两个显示区域对应于用户的双眼,分别为主视眼对应的显示区域和弱视眼对应的显示区域。弱视训练设备102的两个显示区域用于显示终端设备101投屏的弱视训练图像,弱视训练设备102还用于确定用户的弱视眼的弱视类型,按照弱视类型,对弱视训练图像进行处理,从而通过这两个显示区域上显示的弱视训练图像进行弱视训练,以矫正弱视眼的视力。The terminal device 101 is configured to display the amblyopia training image, and project the displayed amblyopia training image to the amblyopia training device 102 . The amblyopia training device 102 includes two display areas, the two display areas correspond to the eyes of the user, respectively, a display area corresponding to the main eye and a display area corresponding to the amblyopic eye. The two display areas of the amblyopia training device 102 are used to display the amblyopia training image projected by the terminal device 101, and the amblyopia training device 102 is also used to determine the amblyopia type of the user's amblyopic eye, and process the amblyopia training image according to the amblyopia type, thereby Amblyopia training is performed through the amblyopia training images displayed on the two display areas to correct the vision of the amblyopic eye.
在一些实施例中,弱视训练设备102包括显示模组、眼动跟踪模块和图像处理模块。显示模组用于显示终端设备101投屏的弱视训练图像。眼动跟踪模块用于确定用户的弱视眼的弱视类型。图像处理模块用于按照弱视类型,对弱视训练图像进行处理。In some embodiments, the amblyopia training device 102 includes a display module, an eye tracking module, and an image processing module. The display module is used to display the amblyopia training image projected by the terminal device 101 . The eye tracking module is used to determine the type of amblyopia of the user's amblyopic eye. The image processing module is used to process the amblyopia training images according to the type of amblyopia.
可选地,眼动跟踪模块还用于确定用户的眼睛的凝视位置,从而确定弱视训练效果。Optionally, the eye tracking module is further configured to determine the gaze position of the user's eyes, so as to determine the amblyopia training effect.
需要说明的是,终端设备101为任何一种可与用户通过键盘、触摸板、触摸屏、遥控器、语音交互或手写设备等一种或多种方式进行人机交互的电子产品。例如个人计算机(personal computer,PC)、手机、智能手机、个人数字助手(personal digital assistant,PDA)、可穿戴设备、掌上电脑PPC(Pocket PC)、平板电脑、智能车机、智能电视、智能音箱等。弱视训练设备102为任何一种具有两个显示区域的头戴式设备。例如,虚拟现实(virtual reality,VR)眼镜、VR头盔、观影眼镜、增强现实(augmented reality,AR)眼镜、AR头盔、融合现实(mix reality,MR)眼镜、MR头盔等设备。其中,弱视训练设备102的两个显示区域为弱视训练设备102的两个显示屏幕,或者为弱视训练设备102的同一个显示屏幕中的不同显示区域。It should be noted that the terminal device 101 is any electronic product that can perform human-computer interaction with the user in one or more ways, such as a keyboard, a touch panel, a touch screen, a remote control, a voice interaction or a handwriting device. For example, personal computer (PC), mobile phone, smart phone, personal digital assistant (PDA), wearable device, Pocket PC (Pocket PC), tablet computer, smart car, smart TV, smart speaker Wait. The amblyopia training device 102 is any kind of head-mounted device with two display areas. For example, virtual reality (VR) glasses, VR helmets, movie viewing glasses, augmented reality (AR) glasses, AR helmets, mixed reality (mix reality, MR) glasses, MR helmets and other devices. The two display areas of the amblyopia training device 102 are two display screens of the amblyopia training device 102 , or different display areas in the same display screen of the amblyopia training device 102 .
值得注意的是,在上述系统架构中,终端设备101用于将弱视训练图像投屏到弱视训练设备102中,进而由弱视训练设备102对用户的弱视眼进行弱视训练。在其他一些实施例中,上述系统架构不包括终端设备101,也就是说,弱视训练设备102能够存储弱视训练图像,并在弱视训练过程中,直接显示存储的弱视训练图像。It is worth noting that in the above system architecture, the terminal device 101 is used to project the amblyopia training image to the amblyopia training device 102 , and then the amblyopia training device 102 performs amblyopia training on the user's amblyopic eyes. In some other embodiments, the above system architecture does not include the terminal device 101, that is, the amblyopia training device 102 can store amblyopia training images, and directly display the stored amblyopia training images during the amblyopia training process.
接下来,对本申请实施例提供的弱视训练方法进行详细地解释说明。Next, the amblyopia training method provided by the embodiment of the present application is explained in detail.
请参考图2,图2是本申请实施例提供的一种弱视训练方法的流程图,该方法应用于弱视训练设备中,该弱视训练设备包括两个显示区域,这两个显示区域对应于用户的双眼。该方法包括如下步骤。Please refer to FIG. 2. FIG. 2 is a flowchart of an amblyopia training method provided by an embodiment of the present application. The method is applied to an amblyopia training device. The amblyopia training device includes two display areas, and the two display areas correspond to user eyes. The method includes the following steps.
步骤201:确定用户的主视眼和弱视眼。Step 201: Determine the dominant eye and the amblyopic eye of the user.
在一些实施例中,确定用户的主视眼和弱视眼的实现过程为:分别检测用户的双眼的屈光度,将用户的双眼中屈光度低的眼睛确定为主视眼,将用户的双眼中屈光度高的眼睛确定为弱视眼。In some embodiments, the process of determining the dominant eye and the amblyopic eye of the user is as follows: detecting the diopter of the user's both eyes respectively, determining the eye with a low diopter in the user's two eyes as the dominant eye, and determining the user's binocular with a high diopter. eyes were identified as amblyopic.
屈光度一般是指近视度数、远视度数或者散光度,通俗地讲,屈光度也称为视力。视力主要是指眼底视网膜成像的能力。作为一种示例,弱视训练设备的两个显示区域中分别显示视力表。在用户的双眼分别注视视力表的过程中,通过调整虚像距,来检测用户的双眼的屈光度。其中,视力表可以为终端设备投屏到弱视训练设备上的,当然,视力表也可以是弱视训练设备自身存储的,本申请实施例对此不做限定。Diopter generally refers to the degree of myopia, farsightedness or astigmatism. Visual acuity mainly refers to the ability to image the retina of the fundus. As an example, an eye chart is displayed in the two display areas of the amblyopia training device respectively. When the user's eyes are looking at the eye chart respectively, the diopter of the user's eyes is detected by adjusting the virtual image distance. The eye chart may be projected by the terminal device to the amblyopia training device. Of course, the eye chart may also be stored by the amblyopia training device itself, which is not limited in this embodiment of the present application.
步骤202:确定弱视眼的弱视类型,弱视类型包括斜视性弱视或屈光参差性弱视。Step 202: Determine the type of amblyopia of the amblyopic eye, where the type of amblyopia includes strabismus amblyopia or anisometropic amblyopia.
在一些实施例中,确定弱视眼的弱视类型的实现过程为:在主视眼对应的显示区域中显示第一测试图像,在弱视眼对应的显示区域中显示第二测试图像。通过眼动跟踪的方式,确定主视眼在显示的第一测试图像中的凝视位置的坐标,以及弱视眼在显示的第二测试图像中的凝视位置的坐标,得到第一凝视位置坐标和第二凝视位置坐标。根据第一凝视位置坐标和第二凝视位置坐标,确定弱视眼的弱视类型。In some embodiments, the realization process of determining the amblyopia type of the amblyopic eye is: displaying the first test image in the display area corresponding to the dominant eye, and displaying the second test image in the display area corresponding to the amblyopic eye. By means of eye tracking, determine the coordinates of the gaze position of the dominant eye in the displayed first test image, and the coordinates of the gaze position of the amblyopic eye in the displayed second test image, and obtain the coordinates of the first gaze position and the 2. Gaze position coordinates. According to the coordinates of the first gaze position and the coordinates of the second gaze position, the type of amblyopia of the amblyopic eye is determined.
其中,第一测试图像和第二测试图像可以相同,也可以不同。第一测试图像和第二测试图像可以为终端设备投屏到弱视训练设备上的,当然,第一测试图像和第二测试图像也可以是弱视训练设备自身存储的,本申请实施例对此不做限定。The first test image and the second test image may be the same or different. The first test image and the second test image may be projected by the terminal device to the amblyopia training device. Of course, the first test image and the second test image may also be stored by the amblyopia training device itself, which is not the case in the embodiment of the present application. Do limit.
作为一种示例,通过眼动跟踪的方式,确定主视眼在显示的第一测试图像中的凝视位置的坐标的实现过程为:确定位置变换矩阵,通过眼动跟踪的方式,确定主视眼的眼球位置坐标,然后将主视眼的眼球位置坐标与位置变换矩阵相乘,得到主视眼在第一测试图像中的凝视位置的坐标,也即是,第一凝视位置坐标。同理,通过眼动跟踪的方式,确定弱视眼在显示的第二测试图像中的凝视位置的坐标的实现过程为:确定位置变换矩阵,通过眼动跟踪的方式,确定弱视眼的眼球位置坐标,然后将弱视眼的眼球位置坐标与位置变换矩阵相乘,得到弱视眼在第二测试图像中的凝视位置的坐标,也即是,第二凝视位置坐标。As an example, the realization process of determining the coordinates of the gaze position of the dominant eye in the displayed first test image by means of eye tracking is as follows: determining a position transformation matrix, and determining the dominant eye by means of eye tracking Then multiply the eyeball position coordinates of the dominant eye with the position transformation matrix to obtain the coordinates of the gaze position of the dominant eye in the first test image, that is, the first gaze position coordinates. Similarly, the realization process of determining the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking is as follows: determining the position transformation matrix, and determining the eyeball position coordinates of the amblyopic eye by means of eye tracking. , and then multiply the eyeball position coordinates of the amblyopic eye by the position transformation matrix to obtain the coordinates of the gaze position of the amblyopic eye in the second test image, that is, the second gaze position coordinates.
其中,位置变换矩阵是指眼球位置与图像中的目标点的位置之间的变换矩阵。该位置变换矩阵的确定方式可以为:弱视训练设备依次显示多个目标点,然后在用户的眼睛注视该多个目标点时,通过眼动跟踪的方式,确定用户的眼球位置坐标,得到对应的多个眼球位置坐标。接下来,通过如下公式确定位置变换矩阵。The position transformation matrix refers to the transformation matrix between the position of the eyeball and the position of the target point in the image. The determination method of the position transformation matrix may be as follows: the amblyopia training device sequentially displays multiple target points, and then, when the user's eyes are gazing at the multiple target points, determines the user's eyeball position coordinates by means of eye tracking, and obtains the corresponding Multiple eye position coordinates. Next, the position transformation matrix is determined by the following formula.
P 1=T*P 2 P 1 =T*P 2
其中,在上述公式中,P 1是指多个目标点的坐标构成的矩阵,T是指位置变换矩阵,P 2是指用户的眼睛注视该多个目标点时,通过眼动跟踪的方式确定的多个眼球位置坐标。 Among them, in the above formula, P 1 refers to the matrix formed by the coordinates of multiple target points, T refers to the position transformation matrix, and P 2 refers to the user's eyes looking at the multiple target points, determined by eye tracking. of multiple eye position coordinates.
作为一种示例,根据第一凝视位置坐标和第二凝视位置坐标,确定弱视眼的弱视类型的实现过程为:根据第一凝视位置坐标和第二凝视位置坐标,确定双眼偏差信息,双眼偏差信息是指弱视眼的视线方向与主视眼的视线方向之间的偏差信息。如果双眼偏差信息大于或等于第一阈值,则确定弱视眼的弱视类型为斜视性弱视。如果双眼偏差信息小于第一阈值,则确定弱视眼的弱视类型为屈光参差性弱视。As an example, the realization process of determining the amblyopia type of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position is as follows: according to the coordinates of the first gaze position and the coordinates of the second gaze position, determining the binocular deviation information, the binocular deviation information It refers to the deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye. If the binocular deviation information is greater than or equal to the first threshold, it is determined that the amblyopia type of the amblyopic eye is strabismus amblyopia. If the binocular deviation information is less than the first threshold, it is determined that the type of amblyopia of the amblyopic eye is anisometropic amblyopia.
双眼偏差信息也称为双眼凝视位置之间的距离,所以,在一些实施例中,弱视训练设备根据第一凝视位置坐标和第二凝视位置坐标,通过如下公式确定双眼偏差信息。The binocular deviation information is also called the distance between the binocular gaze positions. Therefore, in some embodiments, the amblyopia training device determines the binocular deviation information by the following formula according to the first gaze position coordinates and the second gaze position coordinates.
Figure PCTCN2021095234-appb-000001
Figure PCTCN2021095234-appb-000001
其中,在上述公式中,ED为双眼偏差信息,(L x,L y)为第一凝视位置坐标,(R x,R y)为第二凝视位置坐标。 Wherein, in the above formula, ED is the binocular deviation information, (L x , Ly ) are the coordinates of the first gaze position, and (R x , R y ) are the coordinates of the second gaze position.
需要说明的是,正常情况下,双眼偏差信息应该最小接近于0。也就是说,双眼偏差信息越大,说明斜视的程度越严重,双眼偏差信息越小,说明斜视的程度越轻微。所以,在确定用户的双眼中存在弱视眼的情况下,可以确定双眼偏差信息是否大于或等于第一阈值,如果双眼偏差信息大于或等于第一阈值,表征用户的弱视眼的弱视类型为斜视性弱视。如果双眼偏差信息小于第一阈值,表征用户的弱视眼的弱视类型为屈光参差性弱视。It should be noted that under normal circumstances, the binocular deviation information should be at least close to 0. That is to say, the larger the binocular deviation information, the more serious the degree of strabismus, and the smaller the binocular deviation information, the lesser the degree of strabismus. Therefore, when it is determined that there is an amblyopic eye in the user's eyes, it can be determined whether the binocular deviation information is greater than or equal to the first threshold, and if the binocular deviation information is greater than or equal to the first threshold, it indicates that the amblyopia type of the user's amblyopic eye is strabismus amblyopia. If the binocular deviation information is less than the first threshold, the amblyopia type representing the amblyopic eye of the user is anisometropic amblyopia.
也就是说,在确定用户的双眼中存在弱视眼的情况下,通过第一阈值即可确定弱视眼的 弱视类型为斜视性弱视还是屈光参差性弱视。That is, when it is determined that there is an amblyopic eye in both eyes of the user, it can be determined whether the amblyopia type of the amblyopic eye is strabismus amblyopia or anisometropic amblyopia through the first threshold.
其中,第一阈值是指参考距离范围内的任一数值,该参考距离范围是指用来区分斜视性弱视和屈光参差性弱视的一个距离范围。而且,第一阈值与弱视训练设备的角分辨率相关。比如,在弱视训练设备的显示屏的平均角分辨率为20的情况下,第一阈值为100pixel(像素)。在弱视训练设备的显示屏的平均角分辨率为30的情况下,第一阈值为150pixel。The first threshold refers to any value within a reference distance range, and the reference distance range refers to a distance range used to distinguish strabismus amblyopia and anisometropic amblyopia. Furthermore, the first threshold is related to the angular resolution of the amblyopia training device. For example, in the case where the average angular resolution of the display screen of the amblyopia training device is 20, the first threshold is 100 pixels (pixels). When the average angular resolution of the display screen of the amblyopia training device is 30, the first threshold is 150 pixels.
值得注意的是,基于上述描述,弱视通常由单眼斜视、屈光参差、高度屈光不正、视觉剥夺等原因引起,但是,对于高度屈光不正和视觉剥夺这两种类型来说,由于在生理上已经对眼睛造成了损伤,通过弱视训练已经无法矫正弱视眼的视力。所以,本申请实施例通过区分斜视性弱视和屈光参差性弱视这两种类型,来进行弱视训练,并不涉及高度屈光不正和视觉剥夺这两种弱视类型。It is worth noting that, based on the above description, amblyopia is usually caused by monocular strabismus, anisometropia, high refractive error, visual deprivation, etc. However, for the two types of high refractive error and visual deprivation, due to the physiological It has already caused damage to the eyes, and the vision of amblyopic eyes cannot be corrected through amblyopia training. Therefore, in the embodiments of the present application, amblyopia training is performed by distinguishing two types of strabismus amblyopia and anisometropic amblyopia, and does not involve the two types of amblyopia, high refractive error and visual deprivation.
上述实现过程是通过眼动跟踪的方式,进行一次测试来确定双眼偏差信息,从而确定弱视眼的弱视类型。在另一些实施例中,能够通过眼动跟踪的方式,进行多次测试来确定双眼偏差信息,从而确定弱视眼的弱视类型。也即是,在另一些实施例中,确定弱视眼的弱视类型的实现过程为:在主视眼对应的显示区域中显示第一测试图像,在弱视眼对应的显示区域中显示第二测试图像。通过眼动跟踪的方式,确定主视眼在显示的第一测试图像中的多个凝视位置的坐标,以及弱视眼在显示的第二测试图像中的多个凝视位置的坐标,得到多个第一凝视位置坐标和多个第二凝视位置坐标。根据多个第一凝视位置坐标和多个第二凝视位置坐标,确定弱视眼的弱视类型。The above implementation process is to perform a test to determine the binocular deviation information by means of eye tracking, so as to determine the type of amblyopia of the amblyopic eye. In other embodiments, the binocular deviation information can be determined by performing multiple tests by means of eye tracking, so as to determine the type of amblyopia of the amblyopic eye. That is, in other embodiments, the realization process of determining the amblyopia type of the amblyopic eye is: displaying the first test image in the display area corresponding to the dominant eye, and displaying the second test image in the display area corresponding to the amblyopic eye. . By means of eye tracking, the coordinates of multiple gaze positions of the dominant eye in the displayed first test image and the coordinates of multiple gaze positions of the amblyopic eye in the displayed second test image are determined to obtain multiple first A gaze position coordinate and a plurality of second gaze position coordinates. The amblyopia type of the amblyopic eye is determined according to the plurality of first gaze position coordinates and the plurality of second gaze position coordinates.
其中,根据多个第一凝视位置坐标和多个第二凝视位置坐标,确定弱视眼的弱视类型的实现过程为:根据同一时间确定的第一凝视位置坐标和第二凝视位置坐标,确定一个双眼偏差信息,从而得到多个双眼偏差信息,双眼偏差信息是指弱视眼的视线方向与主视眼的视线方向之间的偏差信息。确定该多个双眼偏差信息的统计值,如果该多个双眼偏差信息的统计值大于或等于第一阈值,则确定弱视眼的弱视类型为斜视性弱视。如果该多个双眼偏差信息的统计值小于第一阈值,则确定弱视眼的弱视类型为屈光参差性弱视。Wherein, the realization process of determining the amblyopia type of the amblyopic eye according to a plurality of first gaze position coordinates and a plurality of second gaze position coordinates is as follows: according to the first gaze position coordinates and the second gaze position coordinates determined at the same time, determine a pair of eyes The deviation information is obtained, thereby obtaining a plurality of binocular deviation information, and the binocular deviation information refers to the deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye. The statistical values of the plurality of binocular deviation information are determined, and if the statistical values of the plurality of binocular deviation information are greater than or equal to the first threshold, it is determined that the amblyopia type of the amblyopic eye is strabismus amblyopia. If the statistical value of the plurality of binocular deviation information is less than the first threshold, it is determined that the amblyopia type of the amblyopic eye is anisometropic amblyopia.
比如,假设弱视训练设备在主视眼对应的显示区域中显示第一测试图像,在弱视眼对应的显示区域中显示第二测试图像之后,通过眼动跟踪的方式,确定主视眼在第一测试图像中的7个凝视位置的坐标,以及弱视眼在第二测试图像中的7个凝视位置的坐标,得到7个第一凝视位置坐标和7个第二凝视位置坐标。根据7个第一凝视位置坐标和7个第二凝视位置坐标,确定的7个双眼偏差信息如图3所示。通过确定这7个双眼偏差信息的统计值,从而能够确定弱视眼的弱视类型。For example, it is assumed that the amblyopia training device displays the first test image in the display area corresponding to the dominant eye, and after displaying the second test image in the display area corresponding to the amblyopic eye, determines that the dominant eye is in the first test image by means of eye tracking The coordinates of the seven gaze positions in the test image and the coordinates of the seven gaze positions of the amblyopic eye in the second test image are used to obtain seven first gaze position coordinates and seven second gaze position coordinates. According to the 7 first gaze position coordinates and the 7 second gaze position coordinates, the determined 7 binocular deviation information is shown in FIG. 3 . By determining the statistical values of the seven binocular deviation information, the type of amblyopia of the amblyopic eye can be determined.
需要说明的是,该多个双眼偏差信息的统计值是指该多个双眼偏差信息的平均值、中值等等,本申请实施例对此不做限定。It should be noted that the statistical value of the plurality of binocular deviation information refers to an average value, a median value, and the like of the plurality of binocular deviation information, which is not limited in this embodiment of the present application.
步骤203:当弱视眼的弱视类型被确定为斜视性弱视时,对弱视训练图像进行单应性变换处理来进行弱视训练,当弱视眼的弱视类型被确定为屈光参差性弱视时,对弱视训练图像进行图像尺寸调整处理来进行弱视训练。Step 203: when the type of amblyopia of the amblyopic eye is determined to be strabismus amblyopia, perform homography transformation on the amblyopic training image to perform amblyopia training, and when the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia The training images are subjected to image resizing processing for amblyopia training.
在一些实施例中,对弱视训练图像进行单应性变换处理的实现过程为:在主视眼对应的显示区域中显示弱视训练图像,对弱视训练图像进行单应性变换处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。同理,对弱视训练图像进行图像尺寸调整来进行弱视训练的实现过程为:在主视眼对应的显示区域中显示弱视训练 图像,对弱视训练图像进行图像尺寸调整后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。In some embodiments, the implementation process of performing homography transformation on the amblyopia training image is as follows: displaying the amblyopia training image in the display area corresponding to the dominant eye, performing homography transformation on the amblyopia training image and displaying it on the amblyopic eye In the corresponding display area, amblyopia training is performed so that the images seen by the user's eyes can be combined. Similarly, the realization process of performing amblyopia training by adjusting the image size of the amblyopia training image is as follows: displaying the amblyopia training image in the display area corresponding to the dominant eye, and adjusting the image size of the amblyopia training image and displaying it on the display corresponding to the amblyopia eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.
在另一些实施例中,为了改善弱视眼的成像能力,还可以确定主视眼对应的图像对比度和弱视眼对应的图像对比度,以使用户的双眼感知能力相同。这样,能够按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示弱视训练图像,按照弱视眼对应的图像对比度,对弱视训练图像进行单应性变换处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。或者,按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示弱视训练图像,按照弱视眼对应的图像对比度,对弱视训练图像进行图像尺寸调整处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。In other embodiments, in order to improve the imaging ability of the amblyopic eye, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye may also be determined, so that the user's binocular perception ability is the same. In this way, the amblyopic training image can be displayed in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and the amblyopic training image can be displayed in the corresponding amblyopic eye after homography transformation according to the image contrast corresponding to the amblyopic eye. Amblyopia training is performed in the display area of the user so that the images seen by the user's eyes can be combined. Or, display the amblyopic training image in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and perform image size adjustment processing on the amblyopic training image according to the image contrast corresponding to the amblyopic eye and display it on the display corresponding to the amblyopic eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.
对于上述两种实施例来说,根据弱视眼的弱视类型对弱视训练图像进行处理的操作,以及进行弱视训练的操作均相同,区别仅仅在于是否需要确定双眼各自对应的图像对比度,进而在弱视训练的过程按照双眼各自对应的图像对比度来显示弱视训练图像。接下来,以第二种实施例为例,通过如下步骤(1)-(2)对本申请实施例提供的弱视训练过程进行介绍。For the above two embodiments, the operation of processing the amblyopia training image according to the type of amblyopia of the amblyopic eye and the operation of performing amblyopia training are the same, the difference is only whether it is necessary to determine the image contrasts corresponding to the eyes, and then in the amblyopia training operation. The process of displaying amblyopia training images according to the respective image contrasts of the eyes. Next, taking the second embodiment as an example, the following steps (1)-(2) are used to introduce the amblyopia training process provided by the embodiment of the present application.
(1)确定主视眼对应的图像对比度和弱视眼对应的图像对比度,以使用户的双眼感知能力相同。(1) Determine the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye, so that the user's binocular perception ability is the same.
在一些实施例中,弱视训练设备在主视眼对应的显示区域中显示第三测试图像,在弱视眼对应的显示区域中显示第四测试图像。之后,降低显示的第三测试图像的对比度,增加显示的第四测试图像的对比度。当检测到对比度确定指令时,将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度,该对比度确定指令是用户根据显示的第三测试图像和第四测试图像反馈自身的双眼能够感知的对比度相同时触发的。In some embodiments, the amblyopia training device displays the third test image in the display area corresponding to the dominant eye, and displays the fourth test image in the display area corresponding to the amblyopic eye. After that, the contrast of the displayed third test image is decreased, and the contrast of the displayed fourth test image is increased. When the contrast determination instruction is detected, the contrast of the reduced third test image is determined as the image contrast corresponding to the dominant eye, and the contrast of the increased fourth test image is determined as the image contrast corresponding to the amblyopic eye, and the contrast determination instruction is It is triggered when the user feeds back that the contrasts that his eyes can perceive are the same according to the displayed third test image and the fourth test image.
也即是,通过在弱视训练设备的两个显示区域上分别显示一个测试图像,通过用户的主观反馈来调节两个测试图像的对比度,从而使用户的双眼能够感知的对比度相同。而且,在本申请实施例中,通过降低第三测试图像的对比度,以及增加第四测试图像的对比度,来保证用户的双眼能够感知的对比度相同。也就是说,通过降低主视眼的对比度,增加弱视眼的对比度,从而在后续进行弱视训练过程中,对主视眼和弱视眼分别显示不同对比度的画面,以强化弱视眼以及抑制主视眼的成像能力,达到弱视训练的目的。That is, by displaying a test image on the two display areas of the amblyopia training device respectively, the contrast of the two test images is adjusted through the user's subjective feedback, so that the user's eyes can perceive the same contrast. Furthermore, in this embodiment of the present application, the contrast ratio of the third test image is reduced and the contrast ratio of the fourth test image is increased to ensure that the user's eyes can perceive the same contrast ratio. That is to say, by reducing the contrast of the dominant eye and increasing the contrast of the amblyopic eye, in the subsequent amblyopic training process, the dominant eye and the amblyopic eye are displayed with different contrast images respectively to strengthen the amblyopic eye and suppress the dominant eye. Imaging ability to achieve the purpose of amblyopia training.
其中,降低显示的第三测试图像的对比度,增加显示的第四测试图像的对比度的实现过程为:按照调整步长降低第三测试图像的对比度,以及按照调整步长增加第四测试图像的对比度。显示调整后的第三测试图像和第四测试图像。当检测到对比度调整指令时,继续按照调整步长降低第三测试图像的对比度,以及按照调整步长增加第四测试图像的对比度,并显示对比度调整后的第三测试图像和第四测试图像,直至检测到对比度确定指令为止。其中,对比度调整指令是用户根据显示的第三测试图像和第四测试图像反馈双眼的对比度不同时触发的。The process of reducing the contrast of the displayed third test image and increasing the contrast of the displayed fourth test image is as follows: reducing the contrast of the third test image according to the adjustment step size, and increasing the contrast of the fourth test image according to the adjustment step size . The adjusted third and fourth test images are displayed. When the contrast adjustment instruction is detected, continue to reduce the contrast of the third test image according to the adjustment step size, and increase the contrast of the fourth test image according to the adjustment step size, and display the contrast-adjusted third test image and the fourth test image, until a contrast determination command is detected. Wherein, the contrast adjustment instruction is triggered by the user according to the displayed third test image and the fourth test image when the contrast between the two eyes is different.
需要说明的是,在通过调整步长调整第三测试图像的对比度的过程中,可以对主视眼对应的显示区域进行遮挡,在调整之后再进行显示,由用户确定双眼的对比度是否相同。当然,在通过调整步长调整第三测试图像的对比度的过程中,也可以不对主视眼对应的显示区域进行遮挡。同理,在通过调整步长调整第四测试图像的对比度的过程中,可以对弱视眼对应的 显示区域进行遮挡,在调整之后再进行显示,由用户确定双眼的对比度是否相同。当然,在通过调整步长调整第四测试图像的对比度的过程中,也可以不对弱视眼对应的显示区域进行遮挡,本申请实施例对此不做限定。It should be noted that in the process of adjusting the contrast of the third test image by adjusting the step size, the display area corresponding to the dominant eye can be blocked, and then displayed after adjustment, and the user determines whether the contrast of the two eyes is the same. Of course, in the process of adjusting the contrast of the third test image by adjusting the step size, the display area corresponding to the dominant eye may not be blocked. Similarly, in the process of adjusting the contrast of the fourth test image by adjusting the step length, the display area corresponding to the amblyopic eye can be occluded, and then displayed after the adjustment, and the user determines whether the contrast of the two eyes is the same. Of course, in the process of adjusting the contrast of the fourth test image by adjusting the step size, the display area corresponding to the amblyopic eye may not be blocked, which is not limited in this embodiment of the present application.
此外,第三测试图像的对比度和第四测试图像的对比度可以依次进行调整,比如,先按照调整步长降低第三测试图像的对比度,然后显示调整后的第三测试图像和第四测试图像,如果用户根据显示的第三测试图像和第四测试图像反馈双眼的对比度不同,那么接下来按照调整步长增加第四测试图像的对比度,显示调整后的第三测试图像和第四测试图像,如果用户根据显示的第三测试图像和第四测试图像反馈双眼的对比度不同,那么再继续按照调整步长降低第三测试图像的对比度,直至用户根据显示的第三测试图像和第四测试图像反馈双眼的对比度相同为止。In addition, the contrast of the third test image and the contrast of the fourth test image can be adjusted in sequence. For example, the contrast of the third test image is first decreased according to the adjustment step, and then the adjusted third test image and the fourth test image are displayed. If the user feedbacks that the contrast between the eyes is different according to the displayed third test image and the fourth test image, then increase the contrast of the fourth test image according to the adjustment step, and display the adjusted third test image and the fourth test image, if The user feeds back that the contrast of the eyes is different according to the displayed third test image and the fourth test image, then continues to reduce the contrast of the third test image according to the adjustment step, until the user feeds back the eyes according to the displayed third test image and the fourth test image. the contrast is the same.
当然,先降低第三测试图像的对比度,后增加第四测试图像的对比度只是一种示例,当然,也可以先增加第四测试图像的对比度,然后降低第三测试图像的对比度,本申请实施例对第三测试图像和第四测试图像的调整顺序不作限定。Of course, reducing the contrast of the third test image first, and then increasing the contrast of the fourth test image is just an example. Of course, the contrast of the fourth test image can also be increased first, and then the contrast of the third test image can be reduced. This embodiment of the present application The adjustment order of the third test image and the fourth test image is not limited.
其中,第三测试图像和第四测试图像可以相同,也可以不同。第三测试图像和第四测试图像可以为终端设备投屏到弱视训练设备上的,当然,第三测试图像和第四测试图像也可以是弱视训练设备自身存储的,本申请实施例对此不做限定。The third test image and the fourth test image may be the same or different. The third test image and the fourth test image may be projected by the terminal device to the amblyopia training device. Of course, the third test image and the fourth test image may also be stored by the amblyopia training device itself, which is not the case in the embodiment of the present application. Do limit.
在第三测试图像和第四测试图像为弱视训练设备自身存储的情况下,可以按照调整步长手动调整弱视训练设备来调整测试图像的对比度。但是,在第三测试图像和第四测试图像为终端设备投屏到弱视训练设备上的情况下,不仅可以按照调整步长手动调整弱视训练设备来调整测试图像的对比度,而且还可以按照调整步长手动调整终端设备来调整测试图像的对比度。When the third test image and the fourth test image are stored by the amblyopia training device itself, the contrast of the test images can be adjusted by manually adjusting the amblyopia training device according to the adjustment step. However, when the third test image and the fourth test image are projected from the terminal device to the amblyopia training device, not only can the amblyopia training device be manually adjusted according to the adjustment step to adjust the contrast of the test image, but also the contrast of the test image can be adjusted according to the adjustment step. Adjust the contrast of the test image by manually adjusting the terminal equipment for a long time.
其中,每次进行对比度调整的调整步长可以相同,也可以不同。调整步长可以是事先设置的一个步长,也可以根据第三测试图像和第四测试图像的对比度来计算得到。Wherein, the adjustment step size for each contrast adjustment may be the same or different. The adjustment step size may be a preset step size, or may be calculated according to the contrast between the third test image and the fourth test image.
作为一种示例,确定第三测试图像和第四测试图像的对比度差值,将该对比度差值除以预设次数,得到调整步长。也就是说,按照这种方式,通过对第三测试图像和第四测试图像调整预设次数后,用户的双眼看到的对比度就会相同。As an example, the contrast difference between the third test image and the fourth test image is determined, and the contrast difference is divided by a preset number of times to obtain the adjustment step size. That is to say, in this way, after adjusting the third test image and the fourth test image a preset number of times, the contrast seen by the user's eyes will be the same.
作为另一种示例,确定第三测试图像和第四测试图像的对比度差值,根据该对比度差值从存储的对比度差值与调整步长之间的对应关系中,获取对应的调整步长。其中,对比度差值与调整步长之间的对应关系可以事先按照经验确定得到。As another example, the contrast difference between the third test image and the fourth test image is determined, and the corresponding adjustment step is obtained from the stored correspondence between the contrast difference and the adjustment step according to the contrast difference. Wherein, the corresponding relationship between the contrast difference and the adjustment step size may be determined empirically in advance.
上述实现过程是根据用户的主观反馈来确定主视眼对应的图像对比度和弱视眼对应的图像对比度,但是,用户主观反馈的对比度往往可能存在偏差。所以,在另一些实施例中,在主视眼对应的显示区域中显示多个第三测试图像,在弱视眼对应的显示区域中显示多个第四测试图像,该多个第三测试图像中包括第一运动目标,该多个第四测试图像中包括第二运动目标。这样,在将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度之前,还包括:通过眼动跟踪的方式,确定主视眼在多个第三测试图像中对第一运动目标的注视轨迹,以及弱视眼在多个第四测试图像中对第二运动目标的注视轨迹,得到第一注视轨迹和第二注视轨迹。获取第一运动目标的实际运动轨迹和第二运动目标的实际运动轨迹,得到第一实际运动轨迹和第二实际运动轨迹。如果第一注视轨迹与第一实际运动轨迹匹配,且第二注视轨迹与第二实际运动轨迹匹配, 则将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度。In the above implementation process, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye are determined according to the user's subjective feedback, but the contrast of the user's subjective feedback may often be biased. Therefore, in some other embodiments, multiple third test images are displayed in the display area corresponding to the dominant eye, and multiple fourth test images are displayed in the display area corresponding to the amblyopic eye. A first moving object is included, and a second moving object is included in the plurality of fourth test images. In this way, before determining the contrast of the reduced third test image as the image contrast corresponding to the dominant eye, and determining the increased contrast of the fourth test image as the image contrast corresponding to the amblyopic eye, the method further includes: by means of eye tracking. , determine the gaze trajectory of the dominant eye to the first moving target in a plurality of third test images, and the gaze trajectory of the amblyopic eye to the second moving target in a plurality of fourth test images, and obtain the first gaze trajectory and the second Gaze track. The actual motion trajectory of the first moving target and the actual motion trajectory of the second moving target are acquired, and the first actual motion trajectory and the second actual motion trajectory are obtained. If the first gaze trajectory matches the first actual motion trajectory, and the second gaze trajectory matches the second actual motion trajectory, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the main eye, and the contrast of the third test image after the increase The contrast of the four test images was determined as the image contrast corresponding to the amblyopic eye.
也即是,通过眼动跟踪的方式,能够进一步准确地确定通过用户的主观反馈确定的双眼对应的图像对比度是否准确,从而避免主观反馈带来的误差。That is, by means of eye tracking, it can be further accurately determined whether the image contrast corresponding to the eyes determined by the user's subjective feedback is accurate, so as to avoid errors caused by the subjective feedback.
作为一种示例,第一注视轨迹与第一实际运动轨迹匹配是指第一注视轨迹与第一实际运动轨迹的重合度大于一定阈值,同理,第二注视轨迹与第二实际运动轨迹匹配是指第二注视轨迹与第二实际运动轨迹的重合度大于一定阈值。As an example, matching the first gaze trajectory with the first actual motion trajectory means that the degree of coincidence between the first gaze trajectory and the first actual motion trajectory is greater than a certain threshold. Similarly, the matching of the second gaze trajectory with the second actual motion trajectory is It means that the degree of coincidence between the second gaze trajectory and the second actual motion trajectory is greater than a certain threshold.
其中,第一运动目标和第二运动目标可以相同,也可以不同。另外,第一注视轨迹与第二注视轨迹的确定方式与上述确定双眼偏差时确定的两个凝视位置坐标的方式类似,本申请实施例对此不再赘述。The first moving target and the second moving target may be the same or different. In addition, the manner of determining the first gaze trajectory and the second gaze trajectory is similar to the manner in which the coordinates of the two gaze positions are determined when the binocular deviation is determined, which is not repeated in this embodiment of the present application.
进一步地,如果第一注视轨迹与第一实际运动轨迹不匹配,那么需要对第三测试图像的对比度重新进行调整,且在调整的过程中,通过眼动跟踪的方式,重新确定第一注视轨迹,如果重新确定的第一注视轨迹与第一实际运动轨迹不匹配,那么继续调整,直至重新确定的第一注视轨迹与第一实际运动轨迹匹配为止,得到主视眼对应的图像对比度。同理,如果第二注视轨迹与第二实际运动轨迹不匹配,那么需要对第四测试图像的对比度重新进行调整,且在调整的过程中,通过眼动跟踪的方式,重新确定第二注视轨迹,如果重新确定的第二注视轨迹与第二实际运动轨迹不匹配,那么继续调整,直至重新确定的第二注视轨迹与第二实际运动轨迹匹配为止,得到弱视眼对应的图像对比度。Further, if the first gaze trajectory does not match the first actual motion trajectory, the contrast of the third test image needs to be re-adjusted, and during the adjustment process, the first gaze trajectory is re-determined by means of eye tracking. , if the re-determined first gaze trajectory does not match the first actual motion trajectory, continue to adjust until the re-determined first gaze trajectory matches the first actual motion trajectory to obtain the image contrast corresponding to the dominant eye. Similarly, if the second gaze trajectory does not match the second actual motion trajectory, then the contrast of the fourth test image needs to be re-adjusted, and during the adjustment process, the second gaze trajectory is re-determined by eye tracking. , if the re-determined second gaze trajectory does not match the second actual motion trajectory, continue to adjust until the re-determined second gaze trajectory matches the second actual motion trajectory to obtain the image contrast corresponding to the amblyopic eye.
(2)按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示弱视训练图像,按照弱视眼对应的图像对比度,根据弱视眼的弱视类型对弱视训练图像进行单应性变换处理或者图像大小调整处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。(2) According to the image contrast corresponding to the dominant eye, display the amblyopia training image in the display area corresponding to the dominant eye, and perform homography transformation processing on the amblyopia training image according to the image contrast corresponding to the amblyopic eye and the amblyopia type of the amblyopic eye Or the image size is adjusted and displayed in the display area corresponding to the amblyopic eye, so that the images seen by the user's eyes can be combined to perform amblyopia training.
基于上述描述,弱视类型包括斜视性弱视和屈光参差性弱视,对于不同的弱视类型,对弱视训练图像进行处理的操作不同,接下来将分别进行介绍。Based on the above description, the types of amblyopia include strabismus amblyopia and anisometropic amblyopia. For different types of amblyopia, the operations for processing amblyopia training images are different, which will be introduced separately next.
弱视眼的弱视类型为斜视性弱视The type of amblyopia in the amblyopic eye is strabismus amblyopia
在这种情况下,弱视训练图像中包括目标对象,目标对象用于进行弱视训练。此时,按照双眼偏差信息对弱视训练图像进行单应性变换。将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像,按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示第一训练图像,按照弱视眼对应的图像对比度,在弱视眼对应的显示区域中显示第二训练图像。通过眼动跟踪的方式,确定主视眼在第一训练图像中的凝视位置,以及弱视眼在第二训练图像中的凝视位置,得到第一凝视位置和第二凝视位置。如果第一凝视位置与第一训练图像中目标对象的实际位置重合,且第二凝视位置与第二训练图像中目标对象的实际位置重合,则减小弱视训练图像的单应性变换量,返回将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像的步骤,直至弱视训练结束,或者弱视训练图像不进行单应性变换的情况下,第二凝视位置与第二训练图像中目标对象的实际位置重合为止。In this case, the target object is included in the amblyopia training image, and the target object is used for amblyopia training. At this time, homography transformation is performed on the amblyopia training image according to the binocular deviation information. The amblyopia training image before homography transformation is used as the first training image, and the amblyopia training image after homography transformation is used as the second training image. According to the image contrast corresponding to the dominant eye, in the display area corresponding to the dominant eye The first training image is displayed, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye. By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined to obtain the first gaze position and the second gaze position. If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the amount of homography transformation of the amblyopia training image, and return The step of taking the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia training ends, or the amblyopia training image is not subjected to homography transformation. In this case, the second gaze position coincides with the actual position of the target object in the second training image.
单应性变换是指一张图像到另一张图像的映射关系,通常包括平移和/或旋转。由于眼睛通常对旋转不够敏感,所以,本申请实施例中,单应性变换主要是指平移。另外,基于上述双眼偏差的运算过程,双眼偏差信息可以分为水平方向上的偏差信息和垂直方向上的偏差信 息,这样,按照双眼偏差信息对弱视训练图像进行单应性变换的实现过程为:将双眼偏差信息中的水平方向上的偏差信息确定为水平方向上的平移量,将双眼偏差信息中垂直方向上的偏差信息确定为垂直方向上的平移量。之后,按照水平方向的平移量和垂直方向上的平移量,对弱视训练图像进行平移,从而实现弱视训练图像的单应性变换。A homography transformation refers to the mapping relationship of one image to another, usually including translation and/or rotation. Since eyes are generally not sensitive enough to rotation, in this embodiment of the present application, the homography transformation mainly refers to translation. In addition, based on the operation process of the above-mentioned binocular deviation, the binocular deviation information can be divided into deviation information in the horizontal direction and deviation information in the vertical direction. In this way, the realization process of performing homography transformation on the amblyopia training image according to the binocular deviation information is: The deviation information in the horizontal direction in the binocular deviation information is determined as the translation amount in the horizontal direction, and the deviation information in the vertical direction in the binocular deviation information is determined as the translation amount in the vertical direction. Then, the amblyopia training image is translated according to the translation amount in the horizontal direction and the translation amount in the vertical direction, so as to realize the homography transformation of the amblyopia training image.
另外,后续减小弱视训练图像的单应性变换量可以是指减小弱视训练图像的水平方向上的平移量和\或垂直方向上的平移量,且水平方向和垂直方向上的减小量相同或者不同,而且这个减小量可以按照需求设置。In addition, the subsequent reduction of the homography transformation amount of the amblyopia training image may refer to reducing the translation amount in the horizontal direction and/or the translation amount in the vertical direction of the amblyopia training image, and the reduction amount in the horizontal direction and the vertical direction The same or different, and this reduction can be set as desired.
比如,如图4所示,假设用户的左眼为斜视性弱视,也就是说,用户的左眼斜视,右眼正视,此时,左眼的视线方向和右眼的视线方向之间存在偏差。为了对用户的左眼进行弱视训练,需要将弱视训练图像中的目标对象的位置向左平移。之后,将平移前的弱视训练图像作为第一训练图像,将平移后的弱视训练图像作为第二训练图像,按照右眼对应的图像对比度,在右眼对应的显示区域中显示第一训练图像,按照左眼对应的图像对比度,在左眼对应的显示区域中显示第二训练图像。For example, as shown in Figure 4, it is assumed that the user's left eye is strabismus amblyopia, that is to say, the user's left eye is strabismus, and the right eye is emmetropic. At this time, there is a deviation between the line of sight of the left eye and the line of sight of the right eye . In order to perform amblyopia training on the user's left eye, the position of the target object in the amblyopia training image needs to be shifted to the left. Afterwards, the amblyopia training image before translation is used as the first training image, and the amblyopia training image after translation is used as the second training image, and the first training image is displayed in the display area corresponding to the right eye according to the image contrast corresponding to the right eye, According to the image contrast corresponding to the left eye, the second training image is displayed in the display area corresponding to the left eye.
通过眼动跟踪的方式,确定右眼在第一训练图像中的凝视位置,以及左眼在第二训练图像中的凝视位置,得到第一凝视位置和第二凝视位置。如果第一凝视位置与第一训练图像中目标对象的实际位置重合,且第二凝视位置与第二训练图像中目标对象的实际位置重合,则减小弱视训练图像中目标对象向左的平移量,继续在双眼的显示区域中显示对应的弱视训练图像。如果在弱视训练图像中的目标对象不进行平移的情况下,第二凝视位置与第二训练图像中目标对象的实际位置重合,那么可以确定用户的左眼已经恢复到正视,也即是,如图5所示。By means of eye tracking, the gaze position of the right eye in the first training image and the gaze position of the left eye in the second training image are determined to obtain the first gaze position and the second gaze position. If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the leftward translation of the target object in the amblyopia training image , and continue to display the corresponding amblyopia training image in the display area of both eyes. If the second gaze position coincides with the actual position of the target object in the second training image without the target object in the amblyopia training image being translated, then it can be determined that the user's left eye has returned to emmetropia, that is, as shown in Figure 5.
其中,第一凝视位置与第二凝视位置的确定方式与上述确定双眼偏差时确定的两个凝视位置坐标的方式类似,本申请实施例对此不再赘述。The manner of determining the first gaze position and the second gaze position is similar to the manner in which the coordinates of the two gaze positions are determined when the binocular deviation is determined, which is not repeated in this embodiment of the present application.
对于斜视性弱视来说,由于弱视眼的视线方向与主视眼的视线方向之间存在偏差,所以需要对弱视训练图像进行单应性变换,以调整目标对象在弱视训练图像中的位置。然后在主视眼对应的显示区域中显示未调整的弱视训练图像,在弱视眼对应的显示区域中显示调整后的弱视训练图像,从而对弱视眼进行训练的过程中,用户的双眼看到的图像能够合像,也即是,双眼能够看到相同的画面。而且,双眼同时观看对应的显示区域中的弱视训练图像,能够保证双眼的立体视觉。For strabismus amblyopia, due to the deviation between the line of sight of the amblyopic eye and the line of sight of the dominant eye, it is necessary to perform homography transformation on the amblyopia training image to adjust the position of the target object in the amblyopia training image. Then, the unadjusted amblyopia training image is displayed in the display area corresponding to the dominant eye, and the adjusted amblyopia training image is displayed in the display area corresponding to the amblyopic eye. The images can be combined, that is, both eyes can see the same picture. Moreover, the two eyes simultaneously watch the amblyopia training image in the corresponding display area, which can ensure the stereoscopic vision of the two eyes.
此外,基于上面的描述,主视眼对应的图像对比度和弱视眼对应的图像对比度可能不同,这样,按照主视眼对应的图像对比度显示第一训练图像,按照弱视眼对应的图像对比度显示第二训练图像,在改善弱视眼的成像能力的同时,还能确保用户的舒适度,使用户更容易接受训练方法及训练过程,改善训练效果,提高用户粘度。In addition, based on the above description, the image contrast corresponding to the dominant eye may be different from the image contrast corresponding to the amblyopic eye. In this way, the first training image is displayed according to the image contrast corresponding to the dominant eye, and the second training image is displayed according to the image contrast corresponding to the amblyopic eye. The training images not only improve the imaging ability of the amblyopic eye, but also ensure the comfort of the user, make it easier for the user to accept the training method and training process, improve the training effect, and increase the user's viscosity.
需要说明的是,弱视训练的整个过程可以分多个周期进行,但是每个周期进行弱视训练之后,双眼对应的图像对比度可能会发生变化,所以,通常情况下,在每个周期进行弱视训练之前,都需要确定用户的双眼对应的图像对比度,具体实现过程参见前述描述,此处不再赘述。It should be noted that the whole process of amblyopia training can be carried out in multiple cycles, but after each cycle of amblyopia training, the image contrast corresponding to the eyes may change. Therefore, under normal circumstances, before each cycle of amblyopia training , it is necessary to determine the image contrast corresponding to the user's eyes. For the specific implementation process, refer to the foregoing description, which will not be repeated here.
弱视眼的弱视类型为屈光参差性弱视The type of amblyopia in the amblyopic eye is anisometropic amblyopia
在这种情况下,弱视训练图像中包括目标对象,目标对象用于进行弱视训练。此时,对弱视训练图像中目标对象的位置和/或尺寸进行多次调整,得到多个训练图像,确定弱视眼相 对于主视眼的图像缩放比例,按照该图像缩放比例,对该多个训练图像的尺寸进行缩放。将缩放前的多个训练图像作为多个第三训练图像,将缩放后的多个训练图像作为多个第四训练图像,按照主视眼对应的图像对比度,在主视眼对应的显示区域中依次显示多个第三训练图像,按照弱视眼对应的图像对比度,在弱视眼对应的显示区域中依次显示多个第四训练图像,且多个第三训练图像和多个第四训练图像的显示顺序和切换频率相同。In this case, the target object is included in the amblyopia training image, and the target object is used for amblyopia training. At this time, the position and/or size of the target object in the amblyopic training image is adjusted multiple times to obtain multiple training images, the image scaling ratio of the amblyopic eye relative to the dominant eye is determined, and according to the image scaling ratio, the multiple training images are obtained. The dimensions of the training images are scaled. Taking the multiple training images before scaling as multiple third training images, and using the multiple training images after scaling as multiple fourth training images, in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye Display a plurality of third training images in sequence, and sequentially display a plurality of fourth training images in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, and display a plurality of third training images and a plurality of fourth training images The sequence and switching frequency are the same.
对于屈光参差性弱视来说,由于弱视眼的屈光度高于主视眼的屈光度,所以弱视眼看到的图像的尺寸与主视眼看到的图像的尺寸可能不同,因此,在对弱视训练图像中目标对象的位置和/或尺寸进行多次调整,得到多个训练图像之后,需要确定弱视眼相对于主视眼的图像缩放比例,进而按照该图像缩放比例对该多个训练图像的尺寸进行缩放,以使主视眼和弱视眼看到的图像能够合像,也即是,双眼能够看到相同的画面。For anisometropic amblyopia, since the diopter of the amblyopic eye is higher than that of the dominant eye, the size of the image seen by the amblyopic eye may be different from that seen by the dominant eye. Therefore, in the training images for amblyopia After adjusting the position and/or size of the target object multiple times to obtain multiple training images, it is necessary to determine the image scaling ratio of the amblyopic eye relative to the dominant eye, and then scale the sizes of the multiple training images according to the image scaling ratio. , so that the images seen by the dominant eye and the amblyopic eye can be combined, that is, both eyes can see the same picture.
其中,对弱视训练图像中的目标对象的位置和/或尺寸进行多次调整的方式包括多种,接下来对其中的四种方式进行介绍。Among them, there are many ways to adjust the position and/or size of the target object in the amblyopia training image multiple times, and the following four ways will be introduced.
第一种方式,对弱视训练图像中的目标对象的位置,依次按照上、下、左、右的顺序进行调整,得到四个训练图像。也即是,对于四个训练图像来说,目标对象的位置依次位于图像中的上方位置、下方位置、左方方位和右方位置。In the first method, the position of the target object in the amblyopia training image is adjusted in the order of up, down, left, and right, and four training images are obtained. That is, for the four training images, the position of the target object is located in the upper position, the lower position, the left position and the right position in the image in sequence.
比如,如图6所示,对弱视训练图像中的目标对象的位置,依次按照上、下、左、右的顺序进行调整,得到的四个训练图像分别为训练图像1至训练图像4。这样,在显示区域中依次显示这四个训练图像之后,对于用户来说,会看到如图7所示的图像,也即是,目标对象按照图7中1、2、3、4的位置进行变换。For example, as shown in Figure 6, the position of the target object in the amblyopia training image is adjusted in the order of up, down, left and right, and the four training images obtained are training image 1 to training image 4 respectively. In this way, after the four training images are displayed in sequence in the display area, the user will see the images as shown in Figure 7, that is, the target objects are in the positions 1, 2, 3, and 4 in Figure 7. Transform.
第二种方式,对弱视训练图像中的目标对象的位置,按照旋转的方式进行调整,得到多个训练图像。In the second method, the position of the target object in the amblyopia training image is adjusted in a rotating manner to obtain multiple training images.
比如,如图8所示,对弱视训练图像中的目标对象的位置,按照旋转的方式进行调整之后,得到八个训练图像,分别训练图像1至训练图像8。这样,在显示区域中依次显示这八个训练图像之后,对于用户来说,会看到如图9所示的图像,也即是,目标对象按照图9中的1、2、3、4、5、6、7、8的位置进行变换。For example, as shown in FIG. 8 , after adjusting the position of the target object in the amblyopia training image in a rotating manner, eight training images are obtained, namely training image 1 to training image 8 . In this way, after the eight training images are displayed in sequence in the display area, the user will see the images shown in FIG. The positions of 5, 6, 7, and 8 are transformed.
第三种方式,对弱视训练图像中的目标对象的位置随机进行调整,得到多个训练图像。In the third method, the position of the target object in the amblyopia training image is randomly adjusted to obtain multiple training images.
比如,如图10所示,对弱视训练图像中的目标对象的位置随机调整之后,得到九个训练图像,分别为训练图像1至训练图像9。这样,在显示区域中依次显示这九个训练图像之后,对于用户来说,会看到如图11所示的图像,也即是,目标对象按照图11中的1、2、3、4、5、6、7、8、9的位置进行变换。For example, as shown in FIG. 10 , after randomly adjusting the position of the target object in the amblyopia training image, nine training images are obtained, which are respectively training image 1 to training image 9 . In this way, after the nine training images are displayed in sequence in the display area, the user will see the images as shown in Figure 11, that is, the target object will follow steps 1, 2, 3, 4, and 4 in Figure 11. The positions of 5, 6, 7, 8, and 9 are transformed.
第四种方式,对弱视训练图像中的目标对象的尺寸进行多次调整,得到多个训练图像。In the fourth manner, the size of the target object in the amblyopia training image is adjusted multiple times to obtain multiple training images.
比如,如图12所示,对弱视训练图像中的目标对象的尺寸调整四次,得到四个训练图像,分别为训练图像1至训练图像4。这样,在显示区域中依次显示这四个训练图像之后,对于用户来说,会看到如图13所示的图像,也即是,目标对象呈现出从远到近的形态。For example, as shown in FIG. 12 , the size of the target object in the amblyopia training image is adjusted four times to obtain four training images, which are respectively training image 1 to training image 4 . In this way, after the four training images are displayed in sequence in the display area, the user will see the images shown in FIG. 13 , that is, the target object presents a shape from far to near.
需要说明的是,前三种方式是对目标对象的位置进行调整,最后一种方式是对目标对象的尺寸进行调整。实际应用中,前三种方式在位置调整的基础上,也可以对目标对象的尺寸进行调整,也即是,位置和尺寸叠加进行调整。同理,对于最后一种方式来说,对目标对象的尺寸调整的基础上,也可以对目标对象的位置进行调整,本申请实施例对此不再赘述。It should be noted that the first three methods are to adjust the position of the target object, and the last method is to adjust the size of the target object. In practical applications, the first three methods can also adjust the size of the target object on the basis of the position adjustment, that is, the position and the size are superimposed to adjust. Similarly, for the last method, on the basis of adjusting the size of the target object, the position of the target object may also be adjusted, which will not be repeated in this embodiment of the present application.
本申请实施例通过上下、左右、旋转、随机、远近等运动方式,训练眼外肌,改善弱视 眼的成像能力,从而能够达到对弱视眼进行训练的目的。The embodiment of the present application trains the extraocular muscles and improves the imaging ability of the amblyopic eye through the movement modes such as up and down, left and right, rotation, random, far and near, so as to achieve the purpose of training the amblyopic eye.
其中,确定弱视眼相对于主视眼的图像缩放比例的实现过程为:在主视眼对应的显示区域中显示第五测试图像,在弱视眼对应的显示区域中显示第六测试图像,第五测试图像包括第一测试目标,第六测试图像包括第二测试目标,第一测试目标和第二测试目标的比例相同。缩放显示的第六测试图像的比例。当检测到比例确定指令时,确定第一测试目标与缩放后的第二测试目标之间的比例,得到图像缩放比例,该比例确定指令是用户根据显示的第五测试图像和第六测试图像反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时触发的。Wherein, the realization process of determining the image scaling ratio of the amblyopic eye relative to the dominant eye is: displaying the fifth test image in the display area corresponding to the dominant eye, displaying the sixth test image in the display area corresponding to the amblyopic eye, and displaying the fifth test image in the display area corresponding to the dominant eye. The test image includes a first test target, the sixth test image includes a second test target, and the ratio of the first test target and the second test target is the same. Scales the ratio of the sixth test image displayed. When the scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user feedback according to the displayed fifth test image and the sixth test image Triggered when your own eyes can see the first test target and the second test target with the same proportions.
也即是,对于同一个眼睛来说,第五测试图像中的第一测试目标和第六测试图像中的第二测试目标最初的比例相同。然后,在主视眼注视第五测试图像,弱视眼注视第六测试图像的过程中,通过第六测试图像的比例进行调整,由用户根据显示的第五测试图像和第六测试图像反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时,通过第一测试目标的尺寸和缩放后的第二测试目标的尺寸,确定图像缩放比例。That is, for the same eye, the first test object in the fifth test image and the second test object in the sixth test image are initially in the same proportion. Then, in the process that the dominant eye is watching the fifth test image and the amblyopic eye is watching the sixth test image, the ratio of the sixth test image is adjusted, and the user feeds back his own When both eyes can see the first test target and the second test target with the same scale, the image scaling ratio is determined by the size of the first test target and the size of the scaled second test target.
作为一种示例,将缩放后的第二测试目标的高度与未缩放的第一测试目标的高度之间的比值,确定为竖直方向的缩放比例,将缩放后的第二测试目标的宽度与未缩放的第一测试目标的宽度之间的比值,确定为水平方向的缩放比例。然后,将竖直方向的缩放比例和水平方向的缩放比例确定为弱视眼相对于主视眼的图像缩放比例。As an example, the ratio between the height of the scaled second test target and the height of the unscaled first test target is determined as the scaling ratio in the vertical direction, and the width of the scaled second test target is The ratio between the widths of the unscaled first test targets is determined as the scaling ratio in the horizontal direction. Then, the scaling ratio in the vertical direction and the scaling ratio in the horizontal direction are determined as the image scaling ratios of the amblyopic eye relative to the dominant eye.
在这种示例下,按照该图像缩放比例,对该多个训练图像的尺寸进行缩放的实现过程为:将该多个训练图像的高度乘以竖直方向的缩放比例,将该多个训练图像的宽度乘以水平方向的缩放比例,从而得到缩放后的多个训练图像。In this example, according to the image scaling ratio, the realization process of scaling the sizes of the multiple training images is: multiplying the heights of the multiple training images by the scaling ratio in the vertical direction, and then scaling the multiple training images. The width is multiplied by the horizontal scaling ratio to obtain multiple training images after scaling.
作为另一种示例,将未缩放的第一测试目标的高度与缩放后的第二测试目标的高度之间的比值,确定为竖直方向的缩放比例,将未缩放的第一测试目标的宽度与缩放后的第二测试目标的宽度之间的比值,确定为水平方向的缩放比例。然后,将竖直方向的缩放比例和水平方向的缩放比例确定为弱视眼相对于主视眼的图像缩放比例。As another example, the ratio between the height of the unscaled first test target and the height of the scaled second test target is determined as the scaling ratio in the vertical direction, and the width of the unscaled first test target is The ratio between the scaled width of the second test target and the scaled width of the second test target is determined as the scaling ratio in the horizontal direction. Then, the scaling ratio in the vertical direction and the scaling ratio in the horizontal direction are determined as the image scaling ratios of the amblyopic eye relative to the dominant eye.
在这种示例下,按照该图像缩放比例,对该多个训练图像的尺寸进行缩放的实现过程为:将该多个训练图像的高度除以竖直方向的缩放比例,将该多个训练图像的宽度除以水平方向的缩放比例,从而得到缩放后的多个训练图像。In this example, according to the image scaling ratio, the realization process of scaling the sizes of the multiple training images is: dividing the heights of the multiple training images by the scaling ratio in the vertical direction, and then dividing the multiple training images by the scaling ratio in the vertical direction. The width is divided by the horizontal scaling ratio to obtain multiple training images after scaling.
需要说明的是,上述是通过缩放第六测试图像的比例来确定图像缩放比例,当然,也可以通过缩放第五测试图像的比例来确定图像缩放比例。另外,为了便于双眼合像,第五测试图像和第六测试图像的背景可以相同。It should be noted that the above is to determine the image scaling ratio by scaling the ratio of the sixth test image, of course, the image scaling ratio can also be determined by scaling the ratio of the fifth test image. In addition, in order to facilitate binocular image formation, the backgrounds of the fifth test image and the sixth test image may be the same.
比如,如图14所示,假设左眼为主视眼,右眼为弱视眼,左眼的显示区域中显示第五测试图像,第五测试图像包括第一测试目标,且第一测试目标的高度和宽度均为2厘米,右眼的显示区域中显示第六测试图像,第六测试图像包括第二测试目标,且第二测试目标的高度和宽度也均为2厘米。之后,缩放第六测试图像的比例。当用户看到的图像为图15所示的图像时,也即是,第二测试目标的外轮廓内切于第一测试目标的外轮廓时,确定用户的双眼能够看到比例相同的第一测试目标和第二测试目标。此时,确定缩放后的第二测试目标的高度和宽度均为3厘米,那么,确定图像缩放比例为3:2。这样,对于弱视眼来说,后续需要将多个训练图像的尺寸分别乘以图像缩放比例后来进行弱视训练。For example, as shown in FIG. 14 , assuming that the left eye is the dominant eye and the right eye is the amblyopic eye, a fifth test image is displayed in the display area of the left eye, the fifth test image includes the first test object, and the first test object’s The height and width are both 2 cm, and a sixth test image is displayed in the display area of the right eye. The sixth test image includes a second test target, and the height and width of the second test target are also 2 cm. After that, the scale of the sixth test image is scaled. When the image seen by the user is the image shown in FIG. 15 , that is, when the outer contour of the second test target is inscribed with the outer contour of the first test target, it is determined that the user’s eyes can see the first test target with the same proportion. A test target and a second test target. At this time, it is determined that the height and width of the scaled second test target are both 3 cm, then the image scaling ratio is determined to be 3:2. In this way, for amblyopic eyes, the size of multiple training images needs to be multiplied by the image scaling ratio to perform amblyopia training subsequently.
需要说明的是,本申请实施例不仅可以确定弱视眼相对于主视眼的图像缩放比例,还可 以确定弱视眼相对于主视眼的旋转角度。但是,由于眼睛通常对旋转不够敏感,所以本申请实施例主要是确定图像缩放比例,进而对弱视眼进行弱视训练的图像进行缩放处理。It should be noted that the embodiment of the present application can not only determine the image scaling ratio of the amblyopic eye relative to the dominant eye, but also determine the rotation angle of the amblyopic eye relative to the dominant eye. However, since eyes are usually not sensitive enough to rotation, the embodiment of the present application mainly determines the image scaling ratio, and then performs scaling processing on the image for amblyopia training for amblyopic eyes.
另外,按照不同的弱视类型进行弱视训练时,还可以对弱视眼的屈光度进行校正,然后,按照上述方式进行弱视训练。In addition, when performing amblyopia training according to different types of amblyopia, the diopter of the amblyopic eye can also be corrected, and then the amblyopia training is performed in the above manner.
本申请实施例按照不同的弱视类型进行弱视训练之后,还可以反馈弱视训练效果,从而便于用户针对性调整训练计划。接下来对其中的两种反馈弱视训练效果的方式进行介绍。After performing amblyopia training according to different types of amblyopia in the embodiment of the present application, the effect of amblyopia training can also be fed back, so as to facilitate the user to adjust the training plan in a targeted manner. Next, two of the ways to feedback the effect of amblyopia training will be introduced.
第一种方式,通过眼动跟踪的方式,确定弱视眼在弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置。如果弱视眼凝视位置与弱视眼对应的显示区域显示的图像中目标对象的实际位置重合,则将弱视眼对应的显示区域显示的图像中目标对象的显示方式修改为参考显示方式,以指示弱视训练效果。In the first manner, the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye is determined by means of eye tracking, and the gaze position of the amblyopic eye is obtained. If the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, modify the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye to the reference display mode to indicate amblyopia training Effect.
其中,参考显示方式可以是指高亮、变色等方式,本申请实施例对此不做限定。The reference display mode may refer to modes such as highlighting, discoloration, and the like, which are not limited in this embodiment of the present application.
比如,如图16所示,如果弱视眼的凝视位置与目标对象的实际位置重合,那么对目标对象进行变色处理。这样,对于用户来说,在多次训练之后,可以获知凝视位置与实际位置重合的次数,进而确定弱视训练效果,便于随时调整训练计划。For example, as shown in FIG. 16 , if the gaze position of the amblyopic eye coincides with the actual position of the target object, then the target object is subjected to discoloration processing. In this way, for the user, after several times of training, the number of times the gaze position coincides with the actual position can be known, so as to determine the training effect of amblyopia, which is convenient for adjusting the training plan at any time.
第二种方式,通过眼动跟踪的方式,确定弱视眼在弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置。根据弱视眼凝视位置和目标对象的实际位置,绘制弱视训练曲线,以指示弱视训练效果。In the second method, the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye is determined by means of eye tracking, and the gaze position of the amblyopic eye is obtained. According to the gaze position of the amblyopic eye and the actual position of the target object, the amblyopia training curve is drawn to indicate the amblyopia training effect.
作为一种示例,统计本次弱视训练周期内的训练总次数,以及本次弱视训练周期内弱视眼凝视位置与目标对象的实际位置重合的次数与训练总次数的百分比。然后,结合历史的多次弱视训练周期的数据来绘制弱视训练曲线。也即是,将多个弱视训练周期作为横轴,将每个弱视训练周期内确定的百分比作为纵轴,绘制弱视训练曲线。As an example, count the total training times in this amblyopia training cycle, and the percentage of the times that the gaze position of the amblyopic eye coincides with the actual position of the target object in the current amblyopia training cycle and the total training times. Then, the amblyopia training curve is drawn by combining the data of multiple amblyopia training cycles in history. That is, taking multiple amblyopia training cycles as the horizontal axis, and taking the percentage determined in each amblyopia training cycle as the vertical axis, the amblyopia training curve is drawn.
比如,如图17所示,弱视训练设备在7个弱视训练周期进行了弱视训练,这7个弱视训练周期内,弱视眼凝视位置与目标对象的实际位置重合的次数与训练总次数的百分比分别为50%、60%、70%、80%、90%、95%、100%。此时,绘制出的弱视训练曲线如图17所示。For example, as shown in Figure 17, the amblyopia training device has performed amblyopia training in 7 amblyopia training cycles. During these 7 amblyopia training cycles, the number of times that the gaze position of the amblyopic eye coincides with the actual position of the target object and the percentage of the total number of training times are respectively 50%, 60%, 70%, 80%, 90%, 95%, 100%. At this time, the drawn amblyopia training curve is shown in FIG. 17 .
需要说明的是,上述两种方式可以单独使用,也可以结合使用,本申请实施例对此不做限定。当然,实际应用中,还可以通过其他的方式来反馈弱视训练效果。比如,在每个弱视训练周期进行弱视训练之后,可以检测弱视眼的屈光度,然后,以多个弱视训练周期作为横轴,以弱视眼的屈光度为纵轴,绘制弱视训练曲线。以图18所示,弱视训练设备在11个弱视训练周期进行了弱视训练,这11个弱视训练周期内,弱视眼的屈光度分别为700、650、600、500、450、400、350、300、275、275、275。此时,绘制出的弱视训练曲线如图18所示。It should be noted that the above two manners may be used alone or in combination, which are not limited in the embodiments of the present application. Of course, in practical applications, the effect of amblyopia training can also be fed back in other ways. For example, after performing amblyopia training in each amblyopic training cycle, the diopter of the amblyopic eye can be detected, and then, with multiple amblyopic training cycles as the horizontal axis and the diopter of the amblyopic eye as the vertical axis, the amblyopia training curve is drawn. As shown in Figure 18, the amblyopia training equipment performed amblyopia training in 11 amblyopia training cycles. During these 11 amblyopia training cycles, the diopter of amblyopic eyes were 275, 275, 275. At this time, the drawn amblyopia training curve is shown in FIG. 18 .
另外,本申请实施例可以通过弱视训练设备来反馈弱视训练效果,也可以通过终端设备来反馈弱视训练效果。也即是,弱视训练设备确定弱视训练效果之后,可以直接反馈给用户。当然,弱视训练设备也可以将弱视训练效果发送给终端设备,由终端设备直接反馈给用户。In addition, in the embodiment of the present application, the amblyopia training effect can be fed back through the amblyopia training device, and the amblyopia training effect can also be fed back through the terminal device. That is, after the amblyopia training device determines the effect of the amblyopia training, it can directly feed back to the user. Of course, the amblyopia training device can also send the amblyopia training effect to the terminal device, and the terminal device directly feeds it back to the user.
上述实现过程是用来确定弱视眼的凝视位置,从而反馈弱视训练效果。当然,在另一些实施例中,也可以确定主视眼的凝视位置,从而反馈弱视训练效果。其中,确定主视眼的凝视位置以及反馈弱视训练效果的方法与上述弱视眼的方法相同,本申请实施例对此不再赘述。The above implementation process is used to determine the gaze position of the amblyopic eye, thereby feeding back the effect of amblyopia training. Of course, in other embodiments, the gaze position of the dominant eye can also be determined, so as to feed back the training effect of amblyopia. The method of determining the gaze position of the dominant eye and feeding back the training effect of amblyopia is the same as the method for the amblyopic eye above, which will not be repeated in this embodiment of the present application.
在本申请实施例中,通过确定弱视眼的弱视类型,按照弱视眼的弱视类型,对弱视训练图像处理后进行显示,来进行弱视训练。也就是说,通过区分弱视类型,从而按照不同的症 状进行针对性训练,能够提高弱视训练效果。而且,本申请实施例在弱视训练过程中,双眼对应的显示区域中均显示有弱视训练图像,这样,可以使双眼看到的图像能够合像,提高双眼的立体视觉。再者,本申请实施例在进行弱视训练之后,还能够反馈弱视训练效果,这样,便于用户能够针对性的调整弱视训练计划,提高用户粘度。最后,本申请实施例提供的弱势训练设备集弱视检测、训练、效果反馈于一体,用户能够更灵活进行针对性训练,提示训练效率和训练效果。In the embodiment of the present application, amblyopia training is performed by determining the amblyopia type of the amblyopic eye, and processing the amblyopia training image according to the amblyopia type of the amblyopic eye. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved. Moreover, during the amblyopia training process in the embodiment of the present application, the amblyopia training images are displayed in the display areas corresponding to both eyes, so that the images seen by the two eyes can be combined and the stereoscopic vision of the eyes can be improved. Furthermore, after the amblyopia training is performed in the embodiment of the present application, the effect of the amblyopia training can also be fed back, so that the user can adjust the amblyopia training plan in a targeted manner and improve the user's viscosity. Finally, the weak training device provided by the embodiment of the present application integrates amblyopia detection, training, and effect feedback, so that the user can perform targeted training more flexibly, and prompt training efficiency and training effect.
图19是本申请实施例提供的一种弱视训练装置的结构示意图,该弱视训练装置可以由软件、硬件或者两者的结合实现成为弱视训练设备的部分或者全部,该弱视训练设备可以为图1所示的弱视训练设备。参见图19,该装置包括:第一确定模块1901、第二确定模块1902、第一弱视训练模块1903和第二弱视训练模块1904。19 is a schematic structural diagram of an amblyopia training device provided in an embodiment of the present application. The amblyopia training device can be realized by software, hardware or a combination of the two to become part or all of the amblyopia training device, and the amblyopia training device can be shown in FIG. 1 Amblyopia training device shown. Referring to FIG. 19 , the apparatus includes: a first determination module 1901 , a second determination module 1902 , a first amblyopia training module 1903 and a second amblyopia training module 1904 .
第一确定模块1901,用于确定用户的主视眼和弱视眼;The first determination module 1901 is used to determine the dominant eye and the amblyopic eye of the user;
第二确定模块1902,用于确定弱视眼的弱视类型,弱视类型包括斜视性弱视或屈光参差性弱视;The second determining module 1902 is configured to determine the type of amblyopia of the amblyopic eye, where the type of amblyopia includes strabismus amblyopia or anisometropic amblyopia;
第一弱视训练模块1903,用于当弱视眼的弱视类型被确定为斜视性弱视时,对弱视训练图像进行单应性变换处理来进行弱视训练;The first amblyopia training module 1903 is used to perform amblyopia training on the amblyopia training image by performing homography transformation processing on the amblyopia type when the amblyopia type of the amblyopic eye is determined as strabismus amblyopia;
第二弱视训练模块1904,用于当弱视眼的弱视类型被确定为屈光参差性弱视时,对弱视训练图像进行图像尺寸调整处理来进行弱视训练。The second amblyopia training module 1904 is configured to perform amblyopia training by performing image size adjustment processing on the amblyopia training image when the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia.
可选地,第二确定模块1902包括:Optionally, the second determining module 1902 includes:
显示子模块,用于在主视眼对应的显示区域中显示第一测试图像,在弱视眼对应的显示区域中显示第二测试图像;a display sub-module for displaying the first test image in the display area corresponding to the dominant eye, and displaying the second test image in the display area corresponding to the amblyopic eye;
第一确定子模块,用于通过眼动跟踪的方式,确定主视眼在显示的第一测试图像中的凝视位置的坐标,以及弱视眼在显示的第二测试图像中的凝视位置的坐标,得到第一凝视位置坐标和第二凝视位置坐标;The first determination submodule is used to determine the coordinates of the gaze position of the dominant eye in the displayed first test image and the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking, obtain the coordinates of the first gaze position and the coordinates of the second gaze position;
第二确定子模块,用于根据第一凝视位置坐标和第二凝视位置坐标,确定弱视眼的弱视类型。The second determination submodule is configured to determine the type of amblyopia of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position.
可选地,第二确定子模块具体用于:Optionally, the second determination submodule is specifically used for:
根据第一凝视位置坐标和第二凝视位置坐标,确定双眼偏差信息,双眼偏差信息是指弱视眼的视线方向与主视眼的视线方向之间的偏差信息;According to the coordinates of the first gaze position and the coordinates of the second gaze position, the binocular deviation information is determined, and the binocular deviation information refers to the deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye;
如果双眼偏差信息大于或等于第一阈值,则确定弱视眼的弱视类型为斜视性弱视;If the binocular deviation information is greater than or equal to the first threshold, then determine that the amblyopia type of the amblyopic eye is strabismus amblyopia;
如果双眼偏差信息小于第一阈值,则确定弱视眼的弱视类型为屈光参差性弱视。If the binocular deviation information is less than the first threshold, it is determined that the type of amblyopia of the amblyopic eye is anisometropic amblyopia.
可选地,第一确定模块具体用于:Optionally, the first determining module is specifically used for:
检测用户的双眼的屈光度;detecting the diopter of the user's eyes;
将用户的双眼中屈光度低的眼睛确定为主视眼,将用户的双眼中屈光度高的眼睛确定为弱视眼。An eye with a low refractive power in both eyes of the user is determined as a dominant eye, and an eye with a high refractive power in both eyes of the user is determined as an amblyopic eye.
可选地,第一弱视训练模块1903包括:Optionally, the first amblyopia training module 1903 includes:
第一弱视训练子模块,用于在主视眼对应的显示区域中显示弱视训练图像,对弱视训练图像进行单应性变换处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。The first amblyopia training sub-module is used for displaying the amblyopia training image in the display area corresponding to the dominant eye, and performing homography transformation on the amblyopia training image and displaying it in the display area corresponding to the amblyopia eye, so that the user's eyes can see the amblyopia training image. The obtained images can be combined for amblyopia training.
可选地,第二弱视训练模块1904包括:Optionally, the second amblyopia training module 1904 includes:
第二弱视训练子模块,用于在主视眼对应的显示区域中显示弱视训练图像,对弱视训练图像进行图像尺寸调整处理后显示在弱视眼对应的显示区域中,以使用户的双眼看到的图像能够合像来进行弱视训练。The second amblyopia training sub-module is used to display the amblyopia training image in the display area corresponding to the dominant eye, adjust the image size of the amblyopia training image and display it in the display area corresponding to the amblyopia eye, so that the user's eyes can see the amblyopia training image The images can be combined for amblyopia training.
可选地,该装置还包括:Optionally, the device also includes:
第三确定模块,用于确定主视眼对应的图像对比度和弱视眼对应的图像对比度,以使用户的双眼感知能力相同。The third determining module is configured to determine the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye, so that the user's binocular perception ability is the same.
可选地,第三确定模块具体用于:Optionally, the third determining module is specifically used for:
在主视眼对应的显示区域中显示第三测试图像,在弱视眼对应的显示区域中显示第四测试图像;displaying the third test image in the display area corresponding to the dominant eye, and displaying the fourth test image in the display area corresponding to the amblyopic eye;
降低显示的第三测试图像的对比度,增加显示的第四测试图像的对比度;reducing the contrast of the displayed third test image and increasing the contrast of the displayed fourth test image;
第三确定子模块,用于当检测到对比度确定指令时,将降低后第三测试图像的对比度确定为主视眼对应的图像对比度,将增加后第四测试图像的对比度确定为弱视眼对应的图像对比度,对比度确定指令是用户根据显示的第三测试图像和第四测试图像反馈自身的双眼能够感知的对比度相同时触发的。The third determination sub-module is configured to, when the contrast determination instruction is detected, determine the contrast of the reduced third test image as the image contrast corresponding to the dominant eye, and determine the contrast of the increased fourth test image as the image contrast corresponding to the amblyopic eye Image contrast, the contrast determination instruction is triggered when the user feeds back that the contrast that can be perceived by both eyes of the user is the same according to the displayed third test image and the fourth test image.
可选地,弱视训练图像中包括目标对象,目标对象用于进行弱视训练;Optionally, a target object is included in the amblyopia training image, and the target object is used for amblyopia training;
第一弱视训练子模块具体用于:The first amblyopia training sub-module is specifically used for:
按照双眼偏差信息对弱视训练图像进行单应性变换,双眼偏差信息是指弱视眼的视线方向与主视眼的视线方向之间的偏差信息;Perform homography transformation on the amblyopic training image according to the binocular deviation information, where the binocular deviation information refers to the deviation information between the sight direction of the amblyopic eye and the sight direction of the dominant eye;
将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像,按照主视眼对应的图像对比度,在主视眼对应的显示区域中显示第一训练图像,按照弱视眼对应的图像对比度,在弱视眼对应的显示区域中显示第二训练图像;The amblyopia training image before homography transformation is used as the first training image, and the amblyopia training image after homography transformation is used as the second training image. According to the image contrast corresponding to the dominant eye, in the display area corresponding to the dominant eye Display the first training image, and display the second training image in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye;
通过眼动跟踪的方式,确定主视眼在第一训练图像中的凝视位置,以及弱视眼在第二训练图像中的凝视位置,得到第一凝视位置和第二凝视位置;Determine the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image by means of eye tracking, to obtain the first gaze position and the second gaze position;
如果第一凝视位置与第一训练图像中目标对象的实际位置重合,且第二凝视位置与第二训练图像中目标对象的实际位置重合,则减小弱视训练图像的单应性变换量,返回将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像的步骤,直至弱视训练结束,或者弱视训练图像不进行单应性变换的情况下,第二凝视位置与第二训练图像中目标对象的实际位置重合为止。If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the amount of homography transformation of the amblyopia training image, and return The step of taking the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia training ends, or the amblyopia training image is not subjected to homography transformation. In this case, the second gaze position coincides with the actual position of the target object in the second training image.
可选地,弱视训练图像中包括目标对象,目标对象用于进行弱视训练;Optionally, a target object is included in the amblyopia training image, and the target object is used for amblyopia training;
第二弱视训练子模块具体用于:The second amblyopia training sub-module is specifically used for:
如果弱视眼的弱视类型为屈光参差性弱视,则对弱视训练图像中目标对象的位置和/或尺寸进行多次调整,得到多个训练图像;If the type of amblyopia of the amblyopic eye is anisometropic amblyopia, the position and/or size of the target object in the amblyopia training image is adjusted multiple times to obtain multiple training images;
确定弱视眼相对于主视眼的图像缩放比例;Determine the image scaling ratio of the amblyopic eye relative to the dominant eye;
按照图像缩放比例,对多个训练图像的尺寸进行缩放;Scale the size of multiple training images according to the image scaling ratio;
将缩放前的多个训练图像作为多个第三训练图像,将缩放后的多个训练图像作为多个第四训练图像,按照主视眼对应的图像对比度,在主视眼对应的显示区域中依次显示多个第三训练图像,按照弱视眼对应的图像对比度,在弱视眼对应的显示区域中依次显示多个第四训练图像,且多个第三训练图像和多个第四训练图像的显示顺序和切换频率相同。Taking the multiple training images before scaling as multiple third training images, and using the multiple training images after scaling as multiple fourth training images, in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye Display a plurality of third training images in sequence, and sequentially display a plurality of fourth training images in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, and display a plurality of third training images and a plurality of fourth training images The sequence and switching frequency are the same.
可选地,第二弱视训练子模块还用于:Optionally, the second amblyopia training submodule is also used for:
在主视眼对应的显示区域中显示第五测试图像,在弱视眼对应的显示区域中显示第六测试图像,第五测试图像包括第一测试目标,第六测试图像包括第二测试目标,第一测试目标和第二测试目标的比例相同;A fifth test image is displayed in the display area corresponding to the dominant eye, and a sixth test image is displayed in the display area corresponding to the amblyopic eye. The fifth test image includes the first test object, the sixth test image includes the second test object, and the sixth test image includes the second test object. The ratio of the first test target and the second test target is the same;
缩放显示的第六测试图像的比例;scaling the ratio of the sixth test image displayed;
当检测到比例确定指令时,确定第一测试目标与缩放后的第二测试目标之间的比例,得到图像缩放比例,该比例确定指令是用户根据显示的第五测试图像和第六测试图像反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时触发的。When the scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user feedback according to the displayed fifth test image and the sixth test image Triggered when your own eyes can see the first test target and the second test target with the same proportions.
可选地,该装置还包括:Optionally, the device also includes:
第四确定模块,用于通过眼动跟踪的方式,确定弱视眼在弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置;a fourth determining module, configured to determine the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye by means of eye tracking, so as to obtain the gaze position of the amblyopic eye;
训练效果指示模块,用于如果弱视眼凝视位置与弱视眼对应的显示区域显示的图像中目标对象的实际位置重合,则将弱视眼对应的显示区域显示的图像中目标对象的显示方式修改为参考显示方式,以指示弱视训练效果,和/或,根据弱视眼凝视位置和目标对象的实际位置,绘制弱视训练曲线,以指示弱视训练效果。The training effect indication module is used to modify the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye as a reference if the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye The display mode is used to indicate the amblyopia training effect, and/or, according to the gaze position of the amblyopic eye and the actual position of the target object, an amblyopia training curve is drawn to indicate the amblyopia training effect.
在本申请实施例中,通过确定弱视眼的弱视类型,按照弱视眼的弱视类型,对弱视训练图像处理后进行显示,来进行弱视训练。也就是说,通过区分弱视类型,从而按照不同的症状进行针对性训练,能够提高弱视训练效果。而且,本申请实施例在弱视训练过程中,双眼对应的显示区域中均显示有弱视训练图像,这样,可以使双眼看到的图像能够合像,提高双眼的立体视觉。In the embodiment of the present application, amblyopia training is performed by determining the amblyopia type of the amblyopic eye, and processing the amblyopia training image according to the amblyopia type of the amblyopic eye. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved. Moreover, during the amblyopia training process in the embodiment of the present application, the amblyopia training images are displayed in the display areas corresponding to both eyes, so that the images seen by the two eyes can be combined and the stereoscopic vision of the eyes can be improved.
需要说明的是:上述实施例提供的弱视训练装置在进行弱视训练时,仅以上述各功能模块的划分进行举例说明,实际应用中,可以根据需要而将上述功能分配由不同的功能模块完成,即将装置的内部结构划分成不同的功能模块,以完成以上描述的全部或者部分功能。另外,上述实施例提供的弱视训练装置与弱视训练方法实施例属于同一构思,其具体实现过程详见方法实施例,这里不再赘述。It should be noted that: when the amblyopia training device provided in the above-mentioned embodiment performs amblyopia training, only the division of the above-mentioned functional modules is used for illustration. That is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the amblyopia training device and the amblyopia training method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.
请参考图20,图20是根据本申请实施例示出的一种弱视训练设备的结构示意图。该弱视训练设备包括至少一个处理器2001、通信总线2002、存储器2003以及至少一个通信接口2004。Please refer to FIG. 20 , which is a schematic structural diagram of an amblyopia training device according to an embodiment of the present application. The amblyopia training device includes at least one processor 2001 , a communication bus 2002 , a memory 2003 and at least one communication interface 2004 .
处理器2001可以是一个通用中央处理器(central processing unit,CPU)、网络处理器(network processor,NP)、微处理器、或者可以是一个或多个用于实现本申请方案的集成电路,例如,专用集成电路(application-specific integrated circuit,ASIC),可编程逻辑器件(programmable logic device,PLD)或其组合。上述PLD可以是复杂可编程逻辑器件(complex programmable logic device,CPLD)、现场可编程逻辑门阵列(field-programmable gate array,FPGA)、通用阵列逻辑(generic array logic,GAL)或其任意组合。The processor 2001 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or may be one or more integrated circuits for implementing the solution of the present application, such as , an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a general array logic (generic array logic, GAL) or any combination thereof.
通信总线2002用于在上述组件之间传送信息。通信总线2002可以分为地址总线、数据总线、控制总线等。为便于表示,图中仅用一条粗线表示,但并不表示仅有一根总线或一种类型的总线。The communication bus 2002 is used to transfer information between the aforementioned components. The communication bus 2002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
存储器2003可以是只读存储器(read-only memory,ROM),也可以是随机存取存储器 (random access memory,RAM),也可以是电可擦可编程只读存储器(electrically erasable programmable read-only Memory,EEPROM)、光盘(包括只读光盘(compact disc read-only memory,CD-ROM)、压缩光盘、激光盘、数字通用光盘、蓝光光盘等)、磁盘存储介质或者其它磁存储设备,或者是能够用于携带或存储具有指令或数据结构形式的期望的程序代码并能够由计算机存取的任何其它介质,但不限于此。存储器2003可以是独立存在,并通过通信总线2002与处理器2001相连接。存储器2003也可以和处理器2001集成在一起。The memory 2003 may be read-only memory (ROM), random access memory (RAM), or electrically erasable programmable read-only memory (electrically erasable programmable read-only memory). , EEPROM), optical disc (including compact disc read-only memory, CD-ROM), compact disc, laser disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or capable of Any other medium for carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2003 may exist independently and be connected to the processor 2001 through the communication bus 2002 . The memory 2003 may also be integrated with the processor 2001 .
通信接口2004使用任何收发器一类的装置,用于与其它设备或通信网络通信。通信接口2004包括有线通信接口,还可以包括无线通信接口。其中,有线通信接口例如可以为以太网接口。以太网接口可以是光接口、电接口或其组合。无线通信接口可以为无线局域网(wireless local area networks,WLAN)接口、蜂窝网络通信接口或其组合等。Communication interface 2004 uses any transceiver-like device for communicating with other devices or a communication network. The communication interface 2004 includes a wired communication interface, and may also include a wireless communication interface. Wherein, the wired communication interface may be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (wireless local area network, WLAN) interface, a cellular network communication interface, or a combination thereof, and the like.
在具体实现中,作为一种实施例,处理器2001可以包括一个或多个CPU,如图20中所示的CPU0和CPU1。In a specific implementation, as an embodiment, the processor 2001 may include one or more CPUs, such as CPU0 and CPU1 as shown in FIG. 20 .
在具体实现中,作为一种实施例,弱视训练设备可以包括多个处理器,如图20中所示的处理器2001和处理器2005。这些处理器中的每一个可以是一个单核处理器,也可以是一个多核处理器。这里的处理器可以指一个或多个设备、电路、和/或用于处理数据(如计算机程序指令)的处理核。In a specific implementation, as an embodiment, the amblyopia training device may include multiple processors, such as the processor 2001 and the processor 2005 as shown in FIG. 20 . Each of these processors can be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).
在具体实现中,作为一种实施例,弱视训练设备还可以包括输出设备2006和输入设备2007。输出设备2006和处理器2001通信,可以以多种方式来显示信息。例如,输出设备2006可以是液晶显示器(liquid crystal display,LCD)、发光二级管(light emitting diode,LED)显示设备、阴极射线管(cathode ray tube,CRT)显示设备或投影仪(projector)等。输入设备2007和处理器2001通信,可以以多种方式接收用户的输入。例如,输入设备2007可以是鼠标、键盘、触摸屏设备或传感设备等。In a specific implementation, as an embodiment, the amblyopia training device may further include an output device 2006 and an input device 2007 . The output device 2006 is in communication with the processor 2001 and can display information in a variety of ways. For example, the output device 2006 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, a projector, or the like . The input device 2007 is in communication with the processor 2001 and can receive user input in a variety of ways. For example, the input device 2007 may be a mouse, a keyboard, a touch screen device, a sensor device, or the like.
在一些实施例中,存储器2003用于存储执行本申请方案的程序代码2010,处理器2001可以执行存储器2003中存储的程序代码2010。该程序代码2010中可以包括一个或多个软件模块,该弱视训练设备可以通过处理器2001以及存储器2003中的程序代码2010,来实现上文实施例提供的方法。In some embodiments, the memory 2003 is used to store the program code 2010 for executing the solutions of the present application, and the processor 2001 can execute the program code 2010 stored in the memory 2003 . The program code 2010 may include one or more software modules, and the amblyopia training device may implement the methods provided by the above embodiments through the processor 2001 and the program code 2010 in the memory 2003 .
请参考图21,图21是本申请实施例提供的一种终端设备的结构示意图。该终端设备包括传感器单元1110、计算单元1120、存储单元1140和交互单元1130。Please refer to FIG. 21 , which is a schematic structural diagram of a terminal device provided by an embodiment of the present application. The terminal device includes a sensor unit 1110 , a computing unit 1120 , a storage unit 1140 and an interaction unit 1130 .
传感器单元1110,通常包括视觉传感器(如相机)、深度传感器、IMU、激光传感器等;The sensor unit 1110 usually includes a vision sensor (such as a camera), a depth sensor, an IMU, a laser sensor, etc.;
计算单元1120,通常包括CPU、GPU、缓存、寄存器等,主要用于运行操作系统;The computing unit 1120, usually including CPU, GPU, cache, registers, etc., is mainly used to run the operating system;
存储单元1140,主要包括内存和外部存储,主要用于用户本地和临时数据的读写等;The storage unit 1140 mainly includes memory and external storage, and is mainly used for reading and writing local and temporary data of users;
交互单元1130,主要包括显示屏、触摸板、扬声器、麦克风等,主要用于和用户进行交互,获取用于输入,并实施呈现算法效果等。比如,可以显示弱视训练图像,并投屏到弱视训练设备上。The interaction unit 1130 mainly includes a display screen, a touch panel, a speaker, a microphone, etc., and is mainly used for interacting with the user, obtaining input for input, and implementing the presentation algorithm effect and the like. For example, an amblyopia training image can be displayed and projected onto the amblyopia training device.
为便于理解,下面将对本申请实施例提供的一种终端设备100的结构进行示例说明。参见图22,图22是本申请实施例提供的一种终端设备的结构示意图。For ease of understanding, the following will illustrate the structure of a terminal device 100 provided by an embodiment of the present application. Referring to FIG. 22, FIG. 22 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.
如图22所示,终端设备100可以包括处理器110,外部存储器接口120,内部存储器121,通用串行总线(universal serial bus,USB)接口130,充电管理模块140,电源管理模块141,电 池142,天线1,天线2,移动通信模块150,无线通信模块160,音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,传感器模块180,按键190,马达191,指示器192,摄像头193,显示屏194,以及用户标识模块(subscriber identification module,SIM)卡接口195等。其中传感器模块180可以包括压力传感器180A,陀螺仪传感器180B,气压传感器180C,磁传感器180D,加速度传感器180E,距离传感器180F,接近光传感器180G,指纹传感器180H,温度传感器180J,触摸传感器180K,环境光传感器180L等。As shown in FIG. 22 , the terminal device 100 may include a processor 110 , an external memory interface 120 , an internal memory 121 , a universal serial bus (USB) interface 130 , a charging management module 140 , a power management module 141 , and a battery 142 , Antenna 1, Antenna 2, Mobile Communication Module 150, Wireless Communication Module 160, Audio Module 170, Speaker 170A, Receiver 170B, Microphone 170C, Headphone Interface 170D, Sensor Module 180, Key 190, Motor 191, Indicator 192, Camera 193 , a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, etc.
可以理解的是,本申请实施例示意的结构并不构成对终端设备100的具体限定。在本申请另一些实施例中,终端设备100可以包括比图示更多或更少的部件,或者组合某些部件,或者拆分某些部件,或者不同的部件布置。图示的部件可以以硬件,软件或软件和硬件的组合实现。It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 100 . In other embodiments of the present application, the terminal device 100 may include more or less components than those shown in the drawings, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
处理器110可以包括一个或多个处理单元,例如:处理器110可以包括应用处理器(application processor,AP),调制解调处理器,图形处理器(graphics processing unit,GPU),图像信号处理器(image signal processor,ISP),控制器,存储器,视频编解码器,数字信号处理器(digital signal processor,DSP),基带处理器,和/或神经网络处理器(neural-network processing unit,NPU)等。其中,不同的处理单元可以是独立的器件,也可以集成在一个或多个处理器中。处理器110可以执行计算机程序,以实现本申请实施例中任一种弱视训练方法。The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The processor 110 may execute a computer program to implement any amblyopia training method in the embodiments of the present application.
其中,控制器可以是终端设备100的神经中枢和指挥中心。控制器可以根据指令操作码和时序信号,产生操作控制信号,完成取指令和执行指令的控制。The controller may be the nerve center and command center of the terminal device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.
处理器110中还可以设置存储器,用于存储指令和数据。在一些实施例中,处理器110中的存储器为高速缓冲存储器。该存储器可以保存处理器110刚用过或循环使用的指令或数据。如果处理器110需要再次使用该指令或数据,可从所述存储器中直接调用,避免了重复存取,减少了处理器110的等待时间,因而提高了系统的效率。A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be directly called from the memory, which avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
在一些实施例中,处理器110可以包括一个或多个接口。接口可以包括集成电路(inter-integrated circuit,I1C)接口,集成电路内置音频(inter-integrated circuit sound,I1S)接口,脉冲编码调制(pulse code modulation,PCM)接口,通用异步收发传输器(universal asynchronous receiver/transmitter,UART)接口,移动产业处理器接口(mobile industry processor interface,MIPI),通用输入输出(general-purpose input/output,GPIO)接口,用户标识模块(subscriber identity module,SIM)接口,和/或通用串行总线(universal serial bus,USB)接口等。In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I1C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I1S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.
可以理解的是,本申请实施例示意的各模块间的接口连接关系,只是示意性说明,并不构成对终端设备100的结构限定。在本申请另一些实施例中,终端设备100也可以采用上述实施例中不同的接口连接方式,或多种接口连接方式的组合。It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the terminal device 100 . In other embodiments of the present application, the terminal device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.
充电管理模块140用于从充电器接收充电输入。其中,充电器可以是无线充电器,也可以是有线充电器。在一些有线充电的实施例中,充电管理模块140可以通过USB接口130接收有线充电器的充电输入。The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 .
电源管理模块141用于连接电池142,充电管理模块140与处理器110。电源管理模块141接收电池142和/或充电管理模块140的输入,为处理器110,内部存储器121,外部存储器,显示屏194,摄像头193,和无线通信模块160等供电。The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .
终端设备100的无线通信功能可以通过天线1,天线2,移动通信模块150,无线通信模块160,调制解调处理器以及基带处理器等实现。The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.
在一些可行的实施方式中,终端设备100可以使用无线通信功能和其他设备通信。例如,终端设备100可以和第二电子设备通信,终端设备100与第二电子设备建立投屏连接,终端设备100输出投屏数据至第二电子设备等。其中,终端设备100输出的投屏数据可以为音视频数据。In some possible implementations, the terminal device 100 may communicate with other devices using a wireless communication function. For example, the terminal device 100 may communicate with the second electronic device, the terminal device 100 establishes a screen projection connection with the second electronic device, and the terminal device 100 outputs screen projection data to the second electronic device, and so on. The screen projection data output by the terminal device 100 may be audio and video data.
天线1和天线2用于发射和接收电磁波信号。终端设备100中的每个天线可用于覆盖单个或多个通信频带。不同的天线还可以复用,以提高天线的利用率。例如:可以将天线1复用为无线局域网的分集天线。在另外一些实施例中,天线可以和调谐开关结合使用。Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.
移动通信模块150可以提供应用在终端设备100上的包括1G/3G/4G/5G等无线通信的解决方案。移动通信模块150可以包括至少一个滤波器,开关,功率放大器,低噪声放大器(low noise amplifier,LNA)等。移动通信模块150可以由天线1接收电磁波,并对接收的电磁波进行滤波,放大等处理,传送至调制解调处理器进行解调。移动通信模块150还可以对经调制解调处理器调制后的信号放大,经天线2转为电磁波辐射出去。在一些实施例中,移动通信模块150的至少部分功能模块可以被设置于处理器110中。在一些实施例中,移动通信模块150的至少部分功能模块可以与处理器110的至少部分模块被设置在同一个器件中。The mobile communication module 150 may provide wireless communication solutions including 1G/3G/4G/5G etc. applied on the terminal device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then convert it into electromagnetic waves and radiate it out through the antenna 2 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .
调制解调处理器可以包括调制器和解调器。其中,调制器用于将待发送的低频基带信号调制成中高频信号。解调器用于将接收的电磁波信号解调为低频基带信号。随后解调器将解调得到的低频基带信号传送至基带处理器处理。低频基带信号经基带处理器处理后,被传递给应用处理器。应用处理器通过音频设备(不限于扬声器170A,受话器170B等)输出声音信号,或通过显示屏194显示图像或视频。在一些实施例中,调制解调处理器可以是独立的器件。在另一些实施例中,调制解调处理器可以独立于处理器110,与移动通信模块150或其他功能模块设置在同一个器件中。The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.
无线通信模块160可以提供应用在终端设备100上的包括无线局域网(wireless local area networks,WLAN)(如无线保真(wireless fidelity,Wi-Fi)网络),蓝牙(bluetooth,BT),全球导航卫星系统(global navigation satellite system,GNSS),调频(frequency modulation,FM),近距离无线通信技术(near field communication,NFC),红外技术(infrared,IR)等无线通信的解决方案。无线通信模块160可以是集成至少一个通信处理模块的一个或多个器件。无线通信模块160经由天线1接收电磁波,将电磁波信号调频以及滤波处理,将处理后的信号发送到处理器110。无线通信模块160还可以从处理器110接收待发送的信号,对其进行调频,放大,经天线2转为电磁波辐射出去。The wireless communication module 160 can provide applications on the terminal device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 1 , modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .
在一些实施例中,终端设备100的天线1和移动通信模块150耦合,天线2和无线通信模块160耦合,使得终端设备100可以通过无线通信技术与网络以及其他设备通信。所述无线通信技术可以包括全球移动通讯系统(global system for mobile communications,GSM),通用分组无线服务(general packet radio service,GPRS),码分多址接入(code division multiple access,CDMA),宽带码分多址(wideband code division multiple access,WCDMA),时分码分多址(time-division code division multiple access,TD-SCDMA),长期演进(long term evolution,LTE),BT,GNSS,WLAN,NFC,FM,和/或IR技术等。所述GNSS可以包括全球卫星定位系统(global positioning system,GPS),全球导航卫星系统(global navigation satellite system,GLONASS),北斗卫星导航系统(beidou navigation satellite system,BDS),准天顶卫星系统(quasi-zenith satellite system,QZSS)和/或星基增强系统(satellite based augmentation systems, SBAS)。In some embodiments, the antenna 1 of the terminal device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).
终端设备100通过GPU,显示屏194,以及应用处理器等实现显示功能。GPU为图像处理的微处理器,连接显示屏194和应用处理器。GPU用于执行数学和几何计算,用于图形渲染。处理器110可包括一个或多个GPU,其执行程序指令以生成或改变显示信息。The terminal device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.
显示屏194用于显示图像,视频等。显示屏194包括显示面板。显示面板可以采用液晶显示屏(liquid crystal display,LCD),有机发光二极管(organic light-emitting diode,OLED),有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的,AMOLED),柔性发光二极管(flex light-emitting diode,FLED),Miniled,MicroLed,Micro-oLed,量子点发光二极管(quantum dot light emitting diodes,QLED)等。在一些实施例中,终端设备100可以包括1个或N个显示屏194,N为大于1的正整数。Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the terminal device 100 may include one or N display screens 194 , where N is a positive integer greater than one.
在一些可行的实施方式中,显示屏194可用于显示终端设备100的系统输出的各个界面。In some feasible implementations, the display screen 194 may be used to display various interfaces of the system output of the terminal device 100 .
终端设备100可以通过ISP,摄像头193,视频编解码器,GPU,显示屏194以及应用处理器等实现拍摄功能。The terminal device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.
ISP用于处理摄像头193反馈的数据。例如,拍照时,打开快门,光线通过镜头被传递到摄像头感光元件上,光信号转换为电信号,摄像头感光元件将所述电信号传递给ISP处理,转化为肉眼可见的图像。ISP还可以对图像的噪点,亮度,肤色进行算法优化。ISP还可以对拍摄场景的曝光,色温等参数优化。在一些实施例中,ISP可以设置在摄像头193中。The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .
摄像头193用于捕获静态图像或视频。物体通过镜头生成光学图像投射到感光元件。感光元件可以是电荷耦合器件(charge coupled device,CCD)或互补金属氧化物半导体(complementary metal-oxide-semiconductor,CMOS)光电晶体管。感光元件把光信号转换成电信号,之后将电信号传递给ISP转换成数字图像信号。ISP将数字图像信号输出到DSP加工处理。DSP将数字图像信号转换成标准的RGB,YUV等格式的图像信号。在一些实施例中,终端设备100可以包括1个或N个摄像头193,N为大于1的正整数。Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the terminal device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.
数字信号处理器用于处理数字信号,除了可以处理数字图像信号,还可以处理其他数字信号。A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals.
视频编解码器用于对数字视频压缩或解压缩。终端设备100可以支持一种或多种视频编解码器。这样,终端设备100可以播放或录制多种编码格式的视频,例如:动态图像专家组(moving picture experts group,MPEG)1,MPEG1,MPEG3,MPEG4等。Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record videos in various encoding formats, for example, moving picture experts group (moving picture experts group, MPEG) 1, MPEG1, MPEG3, MPEG4, and so on.
NPU为神经网络(neural-network,NN)计算处理器,通过借鉴生物神经网络结构,例如借鉴人脑神经元之间传递模式,对输入信息快速处理,还可以不断的自学习。通过NPU可以实现终端设备100的智能认知等应用,例如:图像识别,人脸识别,语音识别,文本理解等。The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the terminal device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.
外部存储器接口120可以用于连接外部存储卡,例如Micro SD卡,实现扩展终端设备100的存储能力。外部存储卡通过外部存储器接口120与处理器110通信,实现数据存储功能。例如将音乐,视频等文件保存在外部存储卡中。The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.
内部存储器121可以用于存储计算机可执行程序代码,所述可执行程序代码包括指令。处理器110通过运行存储在内部存储器121的指令,从而执行终端设备100的各种功能应用以及数据处理。内部存储器121可以包括存储程序区和存储数据区。其中,存储程序区可存储操作系统,至少一个功能所需的应用程序(比如本申请实施例中的室内定位方法等)等。存储数据区可存储终端设备100使用过程中所创建的数据(比如音频数据,电话本等)等。此外, 内部存储器121可以包括高速随机存取存储器,还可以包括非易失性存储器,例如至少一个磁盘存储器件,闪存器件,通用闪存存储器(universal flash storage,UFS)等。Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an operating system, an application program required for at least one function (such as the indoor positioning method in the embodiment of the present application, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the terminal device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.
终端设备100可以通过音频模块170,扬声器170A,受话器170B,麦克风170C,耳机接口170D,以及应用处理器等实现音频功能。例如音乐播放,录音等。在一些可行的实施方式中,音频模块170可用于播放视频对应的声音。例如,显示屏194显示视频播放画面时,音频模块170输出视频播放的声音。The terminal device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc. In some feasible implementations, the audio module 170 can be used to play the sound corresponding to the video. For example, when the display screen 194 displays a video playing screen, the audio module 170 outputs the sound of the video playing.
音频模块170用于将数字音频信息转换成模拟音频信号输出,也用于将模拟音频输入转换为数字音频信号。The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal.
扬声器170A,也称“喇叭”,用于将音频电信号转换为声音信号。Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals.
受话器170B,也称“听筒”,用于将音频电信号转换成声音信号。The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals.
麦克风170C,也称“话筒”,“传声器”,用于将声音信号转换为电信号。The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.
耳机接口170D用于连接有线耳机。耳机接口170D可以是USB接口130,也可以是3.5mm的开放移动电子设备平台(open mobile terminal platform,OMTP)标准接口,美国蜂窝电信工业协会(cellular telecommunications industry association of the USA,CTIA)标准接口。The earphone jack 170D is used to connect wired earphones. The earphone interface 170D may be the USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.
压力传感器180A用于感受压力信号,可以将压力信号转换成电信号。在一些实施例中,压力传感器180A可以设置于显示屏194。陀螺仪传感器180B可以用于确定终端设备100的运动姿态。气压传感器180C用于测量气压。The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . The gyro sensor 180B may be used to determine the motion attitude of the terminal device 100 . The air pressure sensor 180C is used to measure air pressure.
加速度传感器180E可检测终端设备100在各个方向上(包括三轴或六轴)加速度的大小。当终端设备100静止时可检测出重力的大小及方向。还可以用于识别终端设备姿态,应用于横竖屏切换,计步器等应用。The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in various directions (including three axes or six axes). The magnitude and direction of gravity can be detected when the terminal device 100 is stationary. It can also be used to identify the posture of terminal devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.
距离传感器180F,用于测量距离。Distance sensor 180F for measuring distance.
环境光传感器180L用于感知环境光亮度。The ambient light sensor 180L is used to sense ambient light brightness.
指纹传感器180H用于采集指纹。The fingerprint sensor 180H is used to collect fingerprints.
温度传感器180J用于检测温度。The temperature sensor 180J is used to detect the temperature.
触摸传感器180K,也称“触控面板”。触摸传感器180K可以设置于显示屏194,由触摸传感器180K与显示屏194组成触摸屏,也称“触控屏”。触摸传感器180K用于检测作用于其上或附近的触摸操作。触摸传感器可以将检测到的触摸操作传递给应用处理器,以确定触摸事件类型。可以通过显示屏194提供与触摸操作相关的视觉输出。在另一些实施例中,触摸传感器180K也可以设置于终端设备100的表面,与显示屏194所处的位置不同。Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal device 100 , which is different from the position where the display screen 194 is located.
按键190包括开机键,音量键等。按键190可以是机械按键。也可以是触摸式按键。终端设备100可以接收按键输入,产生与终端设备100的用户设置以及功能控制有关的键信号输入。The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100 .
马达191可以产生振动提示。Motor 191 can generate vibrating cues.
指示器192可以是指示灯,可以用于指示充电状态,电量变化,也可以用于指示消息,未接来电,通知等。The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.
SIM卡接口195用于连接SIM卡。The SIM card interface 195 is used to connect a SIM card.
在上述实施例中,可以全部或部分地通过软件、硬件、固件或者其任意结合来实现。当使用软件实现时,可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包 括一个或多个计算机指令。在计算机上加载和执行所述计算机指令时,全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络或其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中,或者从一个计算机可读存储介质向另一个计算机可读存储介质传输,例如,所述计算机指令可以从一个网站站点、计算机、服务器或数据中心通过有线(例如:同轴电缆、光纤、数据用户线(digital subscriber line,DSL))或无线(例如:红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质,或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质(例如:软盘、硬盘、磁带)、光介质(例如:数字通用光盘(digital versatile disc,DVD))或半导体介质(例如:固态硬盘(solid state disk,SSD))等。值得注意的是,本申请实施例提到的计算机可读存储介质可以为非易失性存储介质,换句话说,可以是非瞬时性存储介质。In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. that includes one or more available media integrated. The available media may be magnetic media (eg: floppy disk, hard disk, magnetic tape), optical media (eg: digital versatile disc (DVD)) or semiconductor media (eg: solid state disk (SSD)) Wait. It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, may be a non-transitory storage medium.
应当理解的是,本文提及的“多个”是指两个或两个以上。在本申请实施例的描述中,除非另有说明,“/”表示或的意思,例如,A/B可以表示A或B;本文中的“和/或”仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,为了便于清楚描述本申请实施例的技术方案,在本申请实施例中,采用了“第一”、“第二”等字样对功能和作用基本相同的相同项或相似项进行区分。本领域技术人员可以理解“第一”、“第二”等字样并不对数量和执行次序进行限定,并且“第一”、“第二”等字样也并不限定一定不同。It should be understood that references herein to "a plurality" means two or more. In the description of the embodiments of the present application, unless otherwise specified, "/" means or means, for example, A/B can mean A or B; "and/or" in this document is only an association that describes an associated object Relation, it means that there can be three kinds of relations, for example, A and/or B can mean that A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like are not necessarily different.
以上所述为本申请提供的实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。The above-mentioned examples provided for this application are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application. Inside.

Claims (27)

  1. 一种弱视训练方法,其特征在于,应用于弱视训练设备,所述方法包括:An amblyopia training method, characterized in that, applied to amblyopia training equipment, the method comprising:
    确定用户的主视眼和弱视眼;Determine the user's dominant eye and amblyopic eye;
    确定所述弱视眼的弱视类型,所述弱视类型包括斜视性弱视或屈光参差性弱视;determining the type of amblyopia of the amblyopic eye, the type of amblyopia including strabismus amblyopia or anisometropic amblyopia;
    当所述弱视眼的弱视类型被确定为斜视性弱视时,对弱视训练图像进行单应性变换处理来进行弱视训练;When the amblyopia type of the amblyopic eye is determined to be strabismus amblyopia, perform amblyopia training on the amblyopia training image by performing homography transformation processing;
    当所述弱视眼的弱视类型被确定为屈光参差性弱视时,对所述弱视训练图像进行图像尺寸调整处理来进行弱视训练。When the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia, image size adjustment processing is performed on the amblyopia training image to perform amblyopia training.
  2. 如权利要求1所述的方法,其特征在于,所述确定所述弱视眼的弱视类型,包括:The method of claim 1, wherein the determining the type of amblyopia of the amblyopic eye comprises:
    在所述主视眼对应的显示区域中显示第一测试图像,在所述弱视眼对应的显示区域中显示第二测试图像;displaying a first test image in a display area corresponding to the dominant eye, and displaying a second test image in a display area corresponding to the amblyopic eye;
    通过眼动跟踪的方式,确定所述主视眼在显示的所述第一测试图像中的凝视位置的坐标,以及所述弱视眼在显示的所述第二测试图像中的凝视位置的坐标,得到第一凝视位置坐标和第二凝视位置坐标;Determine the coordinates of the gaze position of the dominant eye in the displayed first test image and the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking, obtain the coordinates of the first gaze position and the coordinates of the second gaze position;
    根据所述第一凝视位置坐标和所述第二凝视位置坐标,确定所述弱视眼的弱视类型。The type of amblyopia of the amblyopic eye is determined according to the coordinates of the first gaze position and the coordinates of the second gaze position.
  3. 如权利要求2所述的方法,其特征在于,所述根据所述第一凝视位置坐标和所述第二凝视位置坐标,确定所述弱视眼的弱视类型,包括:The method of claim 2, wherein the determining the amblyopia type of the amblyopic eye according to the first gaze position coordinates and the second gaze position coordinates comprises:
    根据所述第一凝视位置坐标和所述第二凝视位置坐标,确定双眼偏差信息,所述双眼偏差信息是指所述弱视眼的视线方向与所述主视眼的视线方向之间的偏差信息;According to the coordinates of the first gaze position and the coordinates of the second gaze position, binocular deviation information is determined, where the binocular deviation information refers to deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye ;
    如果所述双眼偏差信息大于或等于第一阈值,则确定所述弱视眼的弱视类型为斜视性弱视;If the binocular deviation information is greater than or equal to the first threshold, determining that the amblyopia type of the amblyopic eye is strabismus amblyopia;
    如果所述双眼偏差信息小于所述第一阈值,则确定所述弱视眼的弱视类型为屈光参差性弱视。If the binocular deviation information is less than the first threshold, it is determined that the amblyopia type of the amblyopic eye is anisometropic amblyopia.
  4. 如权利要求1-3任一所述的方法,其特征在于,所述确定所述用户的主视眼和弱视眼,包括:The method according to any one of claims 1-3, wherein the determining the dominant eye and the amblyopic eye of the user comprises:
    检测所述用户的双眼的屈光度;detecting the diopter of the user's eyes;
    将所述用户的双眼中屈光度低的眼睛确定为主视眼,将所述用户的双眼中屈光度高的眼睛确定为弱视眼。The eye with low refractive power in both eyes of the user is determined as the dominant eye, and the eye with high refractive power in the user's both eyes is determined as the amblyopic eye.
  5. 如权利要求1-4任一所述的方法,其特征在于,所述对弱视训练图像进行单应性变换处理来进行弱视训练,包括:The method according to any one of claims 1-4, wherein the performing homography transformation processing on the amblyopia training image to perform amblyopia training, comprising:
    在所述主视眼对应的显示区域中显示所述弱视训练图像,对所述弱视训练图像进行单应性变换处理后显示在所述弱视眼对应的显示屏幕中,以使所述用户的双眼看到的图像能够合像来进行弱视训练。The amblyopic training image is displayed in the display area corresponding to the dominant eye, and the amblyopic training image is displayed in the display screen corresponding to the amblyopic eye after performing homography transformation processing, so that the user's eyes The images seen can be combined for amblyopia training.
  6. 如权利要求1-4任一所述的方法,其特征在于,所述对所述弱视训练图像进行图像尺寸调整处理来进行弱视训练,包括:The method according to any one of claims 1-4, characterized in that, performing image size adjustment processing on the amblyopia training image to perform amblyopia training, comprising:
    在所述主视眼对应的显示区域中显示所述弱视训练图像,对所述弱视训练图像进行图像尺寸调整处理后显示在所述弱视眼对应的显示区域中,以使所述用户的双眼看到的图像能够合像来进行弱视训练。Displaying the amblyopia training image in the display area corresponding to the dominant eye, performing image size adjustment processing on the amblyopia training image and displaying it in the display area corresponding to the amblyopic eye, so that both eyes of the user can see the amblyopia training image. The obtained images can be combined for amblyopia training.
  7. 如权利要求5或6所述的方法,其特征在于,在所述主视眼对应的显示区域中显示所述弱视训练图像之前,所述方法还包括:The method according to claim 5 or 6, wherein before displaying the amblyopia training image in the display area corresponding to the dominant eye, the method further comprises:
    确定所述主视眼对应的图像对比度和所述弱视眼对应的图像对比度,以使所述用户的双眼感知能力相同。The image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye are determined, so that the user's binocular perception ability is the same.
  8. 如权利要求7所述的方法,其特征在于,所述确定所述主视眼对应的图像对比度和所述弱视眼对应的图像对比度,包括:The method of claim 7, wherein the determining the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye comprises:
    在所述主视眼对应的显示区域中显示第三测试图像,在所述弱视眼对应的显示区域中显示第四测试图像;displaying a third test image in the display area corresponding to the dominant eye, and displaying a fourth test image in the display area corresponding to the amblyopic eye;
    降低显示的所述第三测试图像的对比度,增加显示的所述第四测试图像的对比度;reducing the contrast of the displayed third test image and increasing the displayed contrast of the fourth test image;
    当检测到对比度确定指令时,将降低后所述第三测试图像的对比度确定为所述主视眼对应的图像对比度,将增加后所述第四测试图像的对比度确定为所述弱视眼对应的图像对比度,所述对比度确定指令是所述用户根据显示的所述第三测试图像和所述第四测试图像反馈自身的双眼能够感知的对比度相同时触发的。When the contrast determination instruction is detected, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the dominant eye, and the contrast of the fourth test image after the increase is determined as the contrast corresponding to the amblyopic eye Image contrast, the contrast determination instruction is triggered when the user feeds back, according to the displayed third test image and the fourth test image, that the contrast that his own eyes can perceive is the same.
  9. 如权利要求7所述的方法,其特征在于,所述弱视训练图像中包括目标对象,所述目标对象用于进行弱视训练;The method of claim 7, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;
    所述在所述主视眼对应的显示区域中显示所述弱视训练图像,对所述弱视训练图像进行单应性变换处理后显示在所述弱视眼对应的显示区域中,包括:The displaying the amblyopic training image in the display area corresponding to the dominant eye, and performing homography transformation on the amblyopia training image and displaying it in the display area corresponding to the amblyopic eye, including:
    按照双眼偏差信息对所述弱视训练图像进行单应性变换,所述双眼偏差信息是指所述弱视眼的视线方向与所述主视眼的视线方向之间的偏差信息;Perform homography transformation on the amblyopia training image according to binocular deviation information, where the binocular deviation information refers to deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye;
    将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像,按照所述主视眼对应的图像对比度,在所述主视眼对应的显示区域中显示所述第一训练图像,按照所述弱视眼对应的图像对比度,在所述弱视眼对应的显示区域中显示所述第二训练图像;The amblyopia training image before the homography transformation is used as the first training image, and the amblyopia training image after the homography transformation is used as the second training image. The first training image is displayed in the display area of the amblyopic eye, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye;
    通过眼动跟踪的方式,确定所述主视眼在所述第一训练图像中的凝视位置,以及所述弱视眼在所述第二训练图像中的凝视位置,得到第一凝视位置和第二凝视位置;By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined, and the first gaze position and the second gaze position are obtained. gaze position;
    如果所述第一凝视位置与所述第一训练图像中所述目标对象的实际位置重合,且所述第二凝视位置与所述第二训练图像中所述目标对象的实际位置重合,则减小所述弱视训练图像的单应性变换量,返回将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像的步骤,直至弱视训练结束,或者所述弱视训练图像不进行单应性变换的情况下,所述第二凝视位置与所述第二训练图像中所述目标对象的实际位置重合为止。If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, then subtract the Small the amount of homography transformation of the amblyopia training image, returning the step of using the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia The training ends, or when the amblyopia training image does not undergo homography transformation, the second gaze position coincides with the actual position of the target object in the second training image.
  10. 如权利要求7所述的方法,其特征在于,所述弱视训练图像中包括目标对象,所述目标对象用于进行弱视训练;The method of claim 7, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;
    所述在所述主视眼对应的显示区域中显示所述弱视训练图像,对所述弱视训练图像进行图像尺寸调整处理后显示在所述弱视眼对应的显示区域中,包括:The displaying of the amblyopia training image in the display area corresponding to the dominant eye, and performing image size adjustment processing on the amblyopia training image and displaying it in the display area corresponding to the amblyopia eye, including:
    对所述弱视训练图像中所述目标对象的位置和/或尺寸进行多次调整,得到多个训练图像;Adjusting the position and/or size of the target object in the amblyopia training image multiple times to obtain multiple training images;
    确定所述弱视眼相对于所述主视眼的图像缩放比例;determining the image scaling ratio of the amblyopic eye relative to the dominant eye;
    按照所述图像缩放比例,对所述多个训练图像的尺寸进行缩放;scaling the sizes of the plurality of training images according to the image scaling ratio;
    将缩放前的所述多个训练图像作为多个第三训练图像,将缩放后的所述多个训练图像作为多个第四训练图像,按照所述主视眼对应的图像对比度,在所述主视眼对应的显示区域中依次显示所述多个第三训练图像,按照所述弱视眼对应的图像对比度,在所述弱视眼对应的显示区域中依次显示所述多个第四训练图像,且所述多个第三训练图像和所述多个第四训练图像的显示顺序和切换频率相同。Taking the plurality of training images before scaling as a plurality of third training images, and taking the plurality of training images after scaling as a plurality of fourth training images, according to the image contrast corresponding to the dominant eye, in the The plurality of third training images are sequentially displayed in the display area corresponding to the dominant eye, and the plurality of fourth training images are sequentially displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, And the display sequence and switching frequency of the plurality of third training images and the plurality of fourth training images are the same.
  11. 如权利要求10所述的方法,其特征在于,所述确定所述弱视眼相对于所述主视眼的图像缩放比例,包括:The method of claim 10, wherein the determining the image scaling ratio of the amblyopic eye relative to the dominant eye comprises:
    在所述主视眼对应的显示区域中显示第五测试图像,在所述弱视眼对应的显示区域中显示第六测试图像,所述第五测试图像包括第一测试目标,所述第六测试图像包括第二测试目标,所述第一测试目标和所述第二测试目标的比例相同;A fifth test image is displayed in the display area corresponding to the dominant eye, and a sixth test image is displayed in the display area corresponding to the amblyopic eye, the fifth test image includes the first test target, and the sixth test image the image includes a second test target, the first test target and the second test target are in the same proportion;
    缩放显示的所述第六测试图像的比例;scaling the ratio of the sixth test image displayed;
    当检测到比例确定指令时,确定所述第一测试目标与缩放后的所述第二测试目标之间的比例,得到所述图像缩放比例,所述比例确定指令是所述用户根据显示的所述第五测试图像和所述第六测试图像反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时触发的。When a scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user according to the displayed data. It is triggered when the fifth test image and the sixth test image feedback that their own eyes can see the first test target and the second test target with the same proportion.
  12. 如权利要求9-11任一所述的方法,其特征在于,所述方法还包括:The method according to any one of claims 9-11, wherein the method further comprises:
    通过眼动跟踪的方式,确定所述弱视眼在所述弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置;Determine the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye by means of eye tracking, and obtain the gaze position of the amblyopic eye;
    如果所述弱视眼凝视位置与所述弱视眼对应的显示区域显示的图像中所述目标对象的实际位置重合,则将所述弱视眼对应的显示区域显示的图像中所述目标对象的显示方式修改为参考显示方式,以指示弱视训练效果,和/或,根据所述弱视眼凝视位置和所述目标对象的实际位置,绘制弱视训练曲线,以指示弱视训练效果。If the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye is changed. Modified to a reference display mode to indicate the amblyopia training effect, and/or, according to the gaze position of the amblyopic eye and the actual position of the target object, draw an amblyopia training curve to indicate the amblyopia training effect.
  13. 一种弱视训练装置,其特征在于,应用于弱视训练设备,所述装置包括:An amblyopia training device, characterized in that it is applied to amblyopia training equipment, the device comprising:
    第一确定模块,用于确定用户的主视眼和弱视眼;a first determining module for determining the dominant eye and amblyopic eye of the user;
    第二确定模块,用于确定所述弱视眼的弱视类型,所述弱视类型包括斜视性弱视或屈光参差性弱视;a second determining module, configured to determine the type of amblyopia of the amblyopic eye, where the type of amblyopia includes strabismus amblyopia or anisometropic amblyopia;
    第一弱视训练模块,用于当所述弱视眼的弱视类型被确定为斜视性弱视时,对弱视训练图像进行单应性变换处理来进行弱视训练;a first amblyopia training module, used for performing amblyopia training on the amblyopia training image by performing homography transformation processing on the amblyopia training image when the amblyopia type of the amblyopic eye is determined to be strabismus amblyopia;
    第二弱视训练模块,用于当所述弱视眼的弱视类型被确定为屈光参差性弱视时,对所述弱视训练图像进行图像尺寸调整处理来进行弱视训练。The second amblyopia training module is configured to perform amblyopia training by performing image size adjustment processing on the amblyopia training image when the amblyopia type of the amblyopic eye is determined to be anisometropic amblyopia.
  14. 如权利要求13所述的装置,其特征在于,所述第二确定模块包括:The apparatus of claim 13, wherein the second determining module comprises:
    显示子模块,用于在所述主视眼对应的显示区域中显示第一测试图像,在所述弱视眼对应的显示区域中显示第二测试图像;a display submodule for displaying a first test image in a display area corresponding to the dominant eye, and displaying a second test image in a display area corresponding to the amblyopic eye;
    第一确定子模块,用于通过眼动跟踪的方式,确定所述主视眼在显示的所述第一测试图像中的凝视位置的坐标,以及所述弱视眼在显示的所述第二测试图像中的凝视位置的坐标,得到第一凝视位置坐标和第二凝视位置坐标;a first determination sub-module for determining, by means of eye tracking, the coordinates of the gaze position of the dominant eye in the displayed first test image, and the second test displayed by the amblyopic eye The coordinates of the gaze position in the image, to obtain the first gaze position coordinates and the second gaze position coordinates;
    第二确定子模块,用于根据所述第一凝视位置坐标和所述第二凝视位置坐标,确定所述弱视眼的弱视类型。The second determination submodule is configured to determine the type of amblyopia of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position.
  15. 如权利要求14所述的装置,其特征在于,所述第二确定子模块具体用于:The apparatus of claim 14, wherein the second determination submodule is specifically configured to:
    根据所述第一凝视位置坐标和所述第二凝视位置坐标,确定双眼偏差信息,所述双眼偏差信息是指所述弱视眼的视线方向与所述主视眼的视线方向之间的偏差信息;According to the coordinates of the first gaze position and the coordinates of the second gaze position, binocular deviation information is determined, where the binocular deviation information refers to deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye ;
    如果所述双眼偏差信息大于或等于第一阈值,则确定所述弱视眼的弱视类型为斜视性弱视;If the binocular deviation information is greater than or equal to the first threshold, determining that the amblyopia type of the amblyopic eye is strabismus amblyopia;
    如果所述双眼偏差信息小于所述第一阈值,则确定所述弱视眼的弱视类型为屈光参差性弱视。If the binocular deviation information is less than the first threshold, it is determined that the amblyopia type of the amblyopic eye is anisometropic amblyopia.
  16. 如权利要求13-15任一所述的装置,其特征在于,所述第一确定模块具体用于:The apparatus according to any one of claims 13-15, wherein the first determining module is specifically configured to:
    检测所述用户的双眼的屈光度;detecting the diopter of the user's eyes;
    将所述用户的双眼中屈光度低的眼睛确定为主视眼,将所述用户的双眼中屈光度高的眼睛确定为弱视眼。The eye with low refractive power in both eyes of the user is determined as the dominant eye, and the eye with high refractive power in the user's both eyes is determined as the amblyopic eye.
  17. 如权利要求13-16任一所述的装置,其特征在于,所述第一弱视训练模块包括:The device according to any one of claims 13-16, wherein the first amblyopia training module comprises:
    第一弱视训练子模块,用于在所述主视眼对应的显示区域中显示所述弱视训练图像,对所述弱视训练图像进行单应性变换处理后显示在所述弱视眼对应的显示区域中,以使所述用户的双眼看到的图像能够合像来进行弱视训练。The first amblyopia training sub-module is used for displaying the amblyopia training image in the display area corresponding to the dominant eye, and performing homography transformation on the amblyopia training image and displaying it in the display area corresponding to the amblyopia eye , so that the images seen by the user's eyes can be combined to perform amblyopia training.
  18. 如权利要求13-16任一所述的装置,其特征在于,所述第二弱视训练模块包括:The device according to any one of claims 13-16, wherein the second amblyopia training module comprises:
    第二弱视训练子模块,用于在所述主视眼对应的显示区域中显示所述弱视训练图像,对所述弱视训练图像进行图像尺寸调整处理后显示在所述弱视眼对应的显示区域中,以使所述用户的双眼看到的图像能够合像来进行弱视训练。The second amblyopia training sub-module is configured to display the amblyopia training image in the display area corresponding to the dominant eye, and display the amblyopia training image in the display area corresponding to the amblyopia after performing image size adjustment processing on the amblyopia training image , so that the images seen by the user's eyes can be combined to perform amblyopia training.
  19. 如权利要求17或18所述的装置,其特征在于,所述装置还包括:The apparatus of claim 17 or 18, wherein the apparatus further comprises:
    第三确定模块,用于确定所述主视眼对应的图像对比度和所述弱视眼对应的图像对比度,以使所述用户的双眼感知能力相同。The third determining module is configured to determine the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye, so that the user's binocular perception ability is the same.
  20. 如权利要求19所述的装置,其特征在于,所述第三确定模块具体用于:The apparatus of claim 19, wherein the third determining module is specifically configured to:
    在所述主视眼对应的显示区域中显示第三测试图像,在所述弱视眼对应的显示区域中显示第四测试图像;displaying a third test image in the display area corresponding to the dominant eye, and displaying a fourth test image in the display area corresponding to the amblyopic eye;
    降低显示的所述第三测试图像的对比度,增加显示的所述第四测试图像的对比度;reducing the contrast of the displayed third test image and increasing the displayed contrast of the fourth test image;
    当检测到对比度确定指令时,将降低后所述第三测试图像的对比度确定为所述主视眼对应的图像对比度,将增加后所述第四测试图像的对比度确定为所述弱视眼对应的图像对比度,所述对比度确定指令是所述用户根据显示的所述第三测试图像和所述第四测试图像反馈自身的双眼能够感知的对比度相同时触发的。When the contrast determination instruction is detected, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the dominant eye, and the contrast of the fourth test image after the increase is determined as the contrast corresponding to the amblyopic eye Image contrast, the contrast determination instruction is triggered when the user feeds back, according to the displayed third test image and the fourth test image, that the contrast that his own eyes can perceive is the same.
  21. 如权利要求19所述的装置,其特征在于,所述弱视训练图像中包括目标对象,所述目标对象用于进行弱视训练;The apparatus of claim 19, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;
    所述第一弱视训练子模块具体用于:The first amblyopia training submodule is specifically used for:
    按照双眼偏差信息对所述弱视训练图像进行单应性变换,所述双眼偏差信息是指所述弱视眼的视线方向与所述主视眼的视线方向之间的偏差信息;Perform homography transformation on the amblyopia training image according to binocular deviation information, where the binocular deviation information refers to deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye;
    将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像,按照所述主视眼对应的图像对比度,在所述主视眼对应的显示区域中显示所述第一训练图像,按照所述弱视眼对应的图像对比度,在所述弱视眼对应的显示区域中显示所述第二训练图像;The amblyopia training image before the homography transformation is used as the first training image, and the amblyopia training image after the homography transformation is used as the second training image. The first training image is displayed in the display area of the amblyopic eye, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye;
    通过眼动跟踪的方式,确定所述主视眼在所述第一训练图像中的凝视位置,以及所述弱视眼在所述第二训练图像中的凝视位置,得到第一凝视位置和第二凝视位置;By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined, and the first gaze position and the second gaze position are obtained. gaze position;
    如果所述第一凝视位置与所述第一训练图像中所述目标对象的实际位置重合,且所述第二凝视位置与所述第二训练图像中所述目标对象的实际位置重合,则减小所述弱视训练图像的单应性变换量,返回将单应性变换前的弱视训练图像作为第一训练图像,将单应性变换后的弱视训练图像作为第二训练图像的步骤,直至弱视训练结束,或者所述弱视训练图像不进行单应性变换的情况下,所述第二凝视位置与所述第二训练图像中所述目标对象的实际位置重合为止。If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, then subtract the Small the amount of homography transformation of the amblyopia training image, returning the step of using the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia The training ends, or when the amblyopia training image does not undergo homography transformation, the second gaze position coincides with the actual position of the target object in the second training image.
  22. 如权利要求19所述的装置,其特征在于,所述弱视训练图像中包括目标对象,所述目标对象用于进行弱视训练;The apparatus of claim 19, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;
    所述第二弱视训练子模块具体用于:The second amblyopia training submodule is specifically used for:
    对所述弱视训练图像中所述目标对象的位置和/或尺寸进行多次调整,得到多个训练图像;Adjusting the position and/or size of the target object in the amblyopia training image multiple times to obtain multiple training images;
    确定所述弱视眼相对于所述主视眼的图像缩放比例;determining the image scaling ratio of the amblyopic eye relative to the dominant eye;
    按照所述图像缩放比例,对所述多个训练图像的尺寸进行缩放;scaling the sizes of the plurality of training images according to the image scaling ratio;
    将缩放前的所述多个训练图像作为多个第三训练图像,将缩放后的所述多个训练图像作为多个第四训练图像,按照所述主视眼对应的图像对比度,在所述主视眼对应的显示区域中依次显示所述多个第三训练图像,按照所述弱视眼对应的图像对比度,在所述弱视眼对应的显示区域中依次显示所述多个第四训练图像,且所述多个第三训练图像和所述多个第四训练图像的显示顺序和切换频率相同。Taking the plurality of training images before scaling as a plurality of third training images, and taking the plurality of training images after scaling as a plurality of fourth training images, according to the image contrast corresponding to the dominant eye, in the The plurality of third training images are sequentially displayed in the display area corresponding to the dominant eye, and the plurality of fourth training images are sequentially displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, And the display sequence and switching frequency of the plurality of third training images and the plurality of fourth training images are the same.
  23. 如权利要求22所述的装置,其特征在于,所述第二弱视训练子模块还用于:The device of claim 22, wherein the second amblyopia training submodule is further used for:
    在所述主视眼对应的显示区域中显示第五测试图像,在所述弱视眼对应的显示区域中显示第六测试图像,所述第五测试图像包括第一测试目标,所述第六测试图像包括第二测试目标,所述第一测试目标和所述第二测试目标的比例相同;A fifth test image is displayed in the display area corresponding to the dominant eye, and a sixth test image is displayed in the display area corresponding to the amblyopic eye, the fifth test image includes the first test target, and the sixth test image the image includes a second test target, the first test target and the second test target are in the same proportion;
    缩放显示的所述第六测试图像的比例;scaling the ratio of the sixth test image displayed;
    当检测到比例确定指令时,确定所述第一测试目标与缩放后的所述第二测试目标之间的比例,得到所述图像缩放比例,所述比例确定指令是所述用户根据显示的所述第五测试图像和所述第六测试图像反馈自身的双眼能够看到比例相同的第一测试目标和第二测试目标时触发的。When a scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user according to the displayed data. It is triggered when the fifth test image and the sixth test image feedback that their own eyes can see the first test target and the second test target with the same proportion.
  24. 如权利要求21-23任一所述的装置,其特征在于,所述装置还包括:The device according to any one of claims 21-23, wherein the device further comprises:
    第四确定模块,用于通过眼动跟踪的方式,确定所述弱视眼在所述弱视眼对应的显示区域显示的图像中的凝视位置,得到弱视眼凝视位置;a fourth determining module, configured to determine the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye by means of eye tracking, so as to obtain the gaze position of the amblyopic eye;
    训练效果指示模块,用于如果所述弱视眼凝视位置与所述弱视眼对应的显示区域显示的图像中所述目标对象的实际位置重合,则将所述弱视眼对应的显示区域显示的图像中所述目标对象的显示方式修改为参考显示方式,以指示弱视训练效果,和/或,根据所述弱视眼凝视位置和所述目标对象的实际位置,绘制弱视训练曲线,以指示弱视训练效果。The training effect indication module is used for, if the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, then the image displayed in the display area corresponding to the amblyopic eye will be displayed in the image. The display mode of the target object is modified to a reference display mode to indicate amblyopia training effect, and/or an amblyopia training curve is drawn according to the gaze position of the amblyopic eye and the actual position of the target object to indicate amblyopia training effect.
  25. 一种弱视训练设备,其特征在于,所述弱视训练设备包括存储器和处理器,所述存储器用于存储计算机程序,所述处理器用于执行所述计算机程序,以实现权利要求1-12任一所述方法的步骤。An amblyopia training device, characterized in that the amblyopia training device comprises a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program to realize any one of claims 1-12 the steps of the method.
  26. 一种计算机可读存储介质,其特征在于,所述存储介质内存储有计算机程序,所述计算机程序被处理器执行时实现权利要求1-12任一所述的方法的步骤。A computer-readable storage medium, characterized in that, a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of any one of the methods of claims 1-12 are implemented.
  27. 一种计算机程序产品,其特征在于,当所述计算机程序产品在计算机上运行时,使得所述计算机执行权利要求1-12任一所述的方法的步骤。A computer program product, characterized in that, when the computer program product runs on a computer, the computer is made to execute the steps of the method according to any one of claims 1-12.
PCT/CN2021/095234 2020-09-24 2021-05-21 Amblyopia training method, apparatus and device, storage medium, and program product WO2022062436A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202011019488.4A CN114255204A (en) 2020-09-24 2020-09-24 Amblyopia training method, device, equipment and storage medium
CN202011019488.4 2020-09-24

Publications (1)

Publication Number Publication Date
WO2022062436A1 true WO2022062436A1 (en) 2022-03-31

Family

ID=80790144

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/095234 WO2022062436A1 (en) 2020-09-24 2021-05-21 Amblyopia training method, apparatus and device, storage medium, and program product

Country Status (2)

Country Link
CN (1) CN114255204A (en)
WO (1) WO2022062436A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101224151A (en) * 2007-01-19 2008-07-23 北京同仁验光配镜中心 Amblyopia vision increasing treatment method and system thereof
CN103876886A (en) * 2014-04-09 2014-06-25 合肥科飞视觉科技有限公司 Amblyopia treatment system
CN110123594A (en) * 2019-05-04 2019-08-16 吴登智 A kind of VR for amblyopia training and intelligent terminal synchronous display system
CN110381810A (en) * 2016-09-23 2019-10-25 诺瓦赛特有限公司 Screening apparatus and method
CN110856686A (en) * 2018-08-25 2020-03-03 广州联海信息科技有限公司 VR amblyopia patient training system

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101224151A (en) * 2007-01-19 2008-07-23 北京同仁验光配镜中心 Amblyopia vision increasing treatment method and system thereof
CN103876886A (en) * 2014-04-09 2014-06-25 合肥科飞视觉科技有限公司 Amblyopia treatment system
CN110381810A (en) * 2016-09-23 2019-10-25 诺瓦赛特有限公司 Screening apparatus and method
CN110856686A (en) * 2018-08-25 2020-03-03 广州联海信息科技有限公司 VR amblyopia patient training system
CN110123594A (en) * 2019-05-04 2019-08-16 吴登智 A kind of VR for amblyopia training and intelligent terminal synchronous display system

Also Published As

Publication number Publication date
CN114255204A (en) 2022-03-29

Similar Documents

Publication Publication Date Title
US9946393B2 (en) Method of controlling display of electronic device and electronic device
US10365882B2 (en) Data processing method and electronic device thereof
CN107767333B (en) Method and equipment for beautifying and photographing and computer storage medium
KR20160026584A (en) A wearable electronic devcie
EP3961358A1 (en) False touch prevention method for curved screen, and eletronic device
US20220197033A1 (en) Image Processing Method and Head Mounted Display Device
CN110708533B (en) Visual assistance method based on augmented reality and intelligent wearable device
EP3222035B1 (en) Method for controlling image display and apparatus supporting same
US20210271321A1 (en) Method and apparatus for training gaze tracking model, and method and apparatus for gaze tracking
KR20180099026A (en) Photographing method using external electronic device and electronic device supporting the same
KR20180071012A (en) Electronic apparatus and controlling method thereof
KR20160146037A (en) Method and apparatus for changing focus of camera
KR20170001430A (en) Display apparatus and image correction method thereof
US11379960B2 (en) Image processing method, image processing apparatus, and wearable device
US11076100B2 (en) Displaying images on a smartglasses device based on image data received from external camera
US20150194125A1 (en) Method and system for adjusting output of display
US20200041761A1 (en) Optical lens assembly and electronic device comprising same
WO2022062436A1 (en) Amblyopia training method, apparatus and device, storage medium, and program product
US11335090B2 (en) Electronic device and method for providing function by using corneal image in electronic device
KR20180109217A (en) Method for enhancing face image and electronic device for the same
KR20180046543A (en) Electronic device and method for acquiring omnidirectional image
US11263790B2 (en) Method for correcting image processing region corresponding to skin and electronic device
WO2020015149A1 (en) Wrinkle detection method and electronic device
CN108604128B (en) Processing method and mobile device
US10969865B2 (en) Method for transmission of eye tracking information, head mounted display and computer device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21870823

Country of ref document: EP

Kind code of ref document: A1