WO2022062436A1

WO2022062436A1 - Amblyopia training method, apparatus and device, storage medium, and program product

Info

Publication number: WO2022062436A1
Application number: PCT/CN2021/095234
Authority: WO
Inventors: 何庭波; 李江; 张朋; 李瑞华; 刘迎春; 臧磊; 郭帮辉
Original assignee: 华为技术有限公司
Priority date: 2020-09-24
Filing date: 2021-05-21
Publication date: 2022-03-31
Also published as: CN114255204A

Abstract

An amblyopia training method, apparatus and device, a storage medium, and a program product, relating to the technical field of image processing. The method comprises: determining the dominant eye and the amblyopic eye of the eyes of a user; determining an amblyopia type of the amblyopic eye, the amblyopia type comprising strabismic amblyopia or anisometropic amblyopia; and when the amblyopia type of the amblyopic eye is determined as the strabismic amblyopia, performing homography transformation processing on an amblyopia training image to perform an amblyopia training, and when the amblyopia type of the amblyopic eye is determined as the anisometropic amblyopia, performing image size adjustment processing on the amblyopia training image to perform the amblyopia training. In the present method, the amblyopia type of the amblyopic eye is determined, and according to the amblyopia type of the amblyopic eye, an amblyopia training is performed after the amblyopia training image is processed. That is to say, by distinguishing between the amblyopia types, a targeted training can be performed according to different symptoms, thus improving the effect of the amblyopia training.

Description

Amblyopia training method, device, equipment, storage medium and program product

This application claims the priority of the Chinese patent application with the application number 202011019488.4 and the invention titled "Amblyopia training method, device, equipment and storage medium" filed on September 24, 2020, the entire contents of which are incorporated into this application by reference .

technical field

The embodiments of the present application relate to the technical field of image processing, and in particular, to an amblyopia training method, apparatus, device, storage medium, and program product.

Background technique

Amblyopia is an eye disease in which there is no organic disease in the eyeball but the best corrected visual acuity is lower than normal vision. It is usually caused by monocular strabismus, anisometropia, high refractive error, visual deprivation, etc. If not treated in time, it can lead to aggravation of amblyopia and even blindness. Therefore, an amblyopia training method is urgently needed to correct the vision of amblyopic eyes.

The related art proposes an amblyopia training device, which can display binocular visual field content, and the binocular visual field content includes a moving target. The amblyopia training instrument can track the gaze trajectories of the user's left eye and right eye to the moving target respectively, and determine the dominant eye and the Amblyopic eye. Then, for the amblyopic eye, the content of the field of view containing the moving object is displayed repeatedly, and the closeness of the gaze track of the amblyopic eye to the moving object's actual movement track is recorded.

However, for patients with different causes or degrees of amblyopia, after training according to the above-mentioned amblyopia training device, the training effect is not very satisfactory.

SUMMARY OF THE INVENTION

The embodiments of the present application provide an amblyopia training method, device, equipment, storage medium and program product, which can solve the problem of unsatisfactory training effect in the related art. The technical solution is as follows:

In a first aspect, an amblyopia training method is provided, which is applied to an amblyopia training device, the method comprising: determining a user's dominant eye and amblyopic eye, and determining amblyopia type of the amblyopic eye, the amblyopia type includes strabismus amblyopia and anisometropia Amblyopia, when the type of amblyopia of the amblyopic eye is determined to be strabismus amblyopia, the amblyopia training image is subjected to homography transformation processing for amblyopia training, when the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia, the amblyopia training The images are subjected to image resizing processing for amblyopia training.

In the embodiment of the present application, by determining the amblyopia type of the amblyopic eye, the amblyopia training image is processed and then amblyopia training is performed according to the amblyopia type of the amblyopic eye. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved.

Wherein, the realization process of determining the type of amblyopia of the amblyopic eye includes: displaying the first test image in the display area corresponding to the dominant eye, displaying the second test image in the display area corresponding to the amblyopic eye, and determining by eye tracking The coordinates of the gaze position of the dominant eye in the displayed first test image and the coordinates of the gaze position of the amblyopic eye in the displayed second test image are used to obtain the first gaze position coordinates and the second gaze position coordinates. According to the coordinates of the first gaze position and the coordinates of the second gaze position, the type of amblyopia of the amblyopic eye is determined.

As an example, the realization process of determining the coordinates of the gaze position of the dominant eye in the displayed first test image by means of eye tracking is as follows: determining a position transformation matrix, and determining the dominant eye by means of eye tracking Then multiply the eyeball position coordinates of the dominant eye with the position transformation matrix to obtain the coordinates of the gaze position of the dominant eye in the first test image, that is, the first gaze position coordinates. Similarly, the realization process of determining the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking is as follows: determining the position transformation matrix, and determining the eyeball position coordinates of the amblyopic eye by means of eye tracking. , and then multiply the eyeball position coordinates of the amblyopic eye by the position transformation matrix to obtain the coordinates of the gaze position of the amblyopic eye in the second test image, that is, the second gaze position coordinates.

The position transformation matrix refers to the transformation matrix between the position of the eyeball and the position of the target point in the image.

As an example, the realization process of determining the amblyopia type of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position is as follows: according to the coordinates of the first gaze position and the coordinates of the second gaze position, determining the binocular deviation information, the binocular deviation information It refers to the deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye. If the binocular deviation information is greater than or equal to the first threshold, it is determined that the amblyopia type of the amblyopic eye is strabismus amblyopia. If the binocular deviation information is less than the first threshold, it is determined that the type of amblyopia of the amblyopic eye is anisometropic amblyopia.

It should be noted that under normal circumstances, the binocular deviation information should be at least close to 0. That is to say, the larger the binocular deviation information, the more serious the degree of strabismus, and the smaller the binocular deviation information, the lesser the degree of strabismus. Therefore, when it is determined that there is an amblyopic eye in the user's eyes, it can be determined whether the binocular deviation information is greater than or equal to the first threshold, and if the binocular deviation information is greater than or equal to the first threshold, it indicates that the amblyopia type of the user's amblyopic eye is strabismus amblyopia. If the binocular deviation information is less than the first threshold, the amblyopia type representing the amblyopic eye of the user is anisometropic amblyopia.

That is, when it is determined that there is an amblyopic eye in both eyes of the user, the amblyopia type of the amblyopic eye can be determined as strabismus amblyopia or anisometropic amblyopia through the first threshold.

The first threshold refers to any value within a reference distance range, and the reference distance range refers to a distance range used to distinguish strabismus amblyopia and anisometropic amblyopia. Furthermore, the first threshold is related to the angular resolution of the amblyopia training device. For example, in the case where the average angular resolution of the display screen of the amblyopia training device is 20, the first threshold is 100 pixels (pixels). When the average angular resolution of the display screen of the amblyopia training device is 30, the first threshold is 150 pixels.

It is worth noting that, based on the above description, amblyopia is usually caused by monocular strabismus, anisometropia, high refractive error, visual deprivation, etc. However, for the two types of high refractive error and visual deprivation, due to the physiological It has already caused damage to the eyes, and the vision of amblyopic eyes cannot be corrected through amblyopia training. Therefore, in the embodiments of the present application, amblyopia training is performed by distinguishing two types of strabismus amblyopia and anisometropic amblyopia, and does not involve the two types of amblyopia, high refractive error and visual deprivation.

Wherein, the realization process of determining the dominant eye and the amblyopic eye in the user's two eyes is: detecting the diopter of the user's two eyes, determining the eye with a low refractive index in the user's two eyes as the dominant eye, and determining the user's eye with a high refractive power in both eyes. Determined to be amblyopic.

Diopter generally refers to the degree of myopia, farsightedness or astigmatism. Visual acuity mainly refers to the ability to image the retina of the fundus. As an example, an eye chart is displayed in the two display areas of the amblyopia training device respectively. When the user's eyes are looking at the eye chart respectively, the diopter of the user's eyes is detected by adjusting the virtual image distance. The eye chart may be projected by the terminal device to the amblyopia training device. Of course, the eye chart may also be stored by the amblyopia training device itself, which is not limited in this embodiment of the present application.

In some embodiments, the implementation process of performing homography transformation on the amblyopia training image is as follows: displaying the amblyopia training image in the display area corresponding to the dominant eye, performing homography transformation on the amblyopia training image and displaying it on the amblyopic eye In the corresponding display area, amblyopia training is performed so that the images seen by the user's eyes can be combined. Similarly, the realization process of performing amblyopia training by adjusting the image size of the amblyopia training image is as follows: displaying the amblyopia training image in the display area corresponding to the dominant eye, and adjusting the image size of the amblyopia training image and displaying it on the display corresponding to the amblyopia eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.

In other embodiments, in order to improve the imaging ability of the amblyopic eye, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye may also be determined, so that the user's binocular perception ability is the same. In this way, the amblyopic training image can be displayed in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and the amblyopia training image can be subjected to homography transformation processing according to the image contrast corresponding to the amblyopic eye and the amblyopia type of the amblyopic eye Then, it is displayed in the display area corresponding to the amblyopic eye, so that the images seen by the user's eyes can be combined to perform amblyopia training. Or, display the amblyopic training image in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and perform image size adjustment processing on the amblyopic training image according to the image contrast corresponding to the amblyopic eye and display it on the display corresponding to the amblyopic eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.

For the above two embodiments, the operation of processing the amblyopia training image according to the type of amblyopia of the amblyopic eye and the operation of performing amblyopia training are the same, the difference is only whether it is necessary to determine the image contrasts corresponding to the eyes, and then in the amblyopia training operation. The process of displaying amblyopia training images according to the respective image contrasts of the eyes. Next, the second embodiment is taken as an example to introduce the amblyopia training process provided by the embodiment of the present application.

In some embodiments, the implementation process of determining the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye includes: displaying a third test image in a display area corresponding to the dominant eye, displaying a third test image in a display area corresponding to the amblyopic eye Fourth test image. After that, the contrast of the displayed third test image is decreased, and the contrast of the displayed fourth test image is increased. When the contrast determination instruction is detected, the contrast of the reduced third test image is determined as the image contrast corresponding to the dominant eye, and the contrast of the increased fourth test image is determined as the image contrast corresponding to the amblyopic eye, and the contrast determination instruction is It is triggered when the user feeds back that the contrasts that his eyes can perceive are the same according to the displayed third test image and the fourth test image.

That is, by displaying a test image on the two display areas of the amblyopia training device respectively, the contrast of the two test images is adjusted through the user's subjective feedback, so that the user's eyes can perceive the same contrast. Furthermore, in this embodiment of the present application, the contrast ratio of the third test image is reduced and the contrast ratio of the fourth test image is increased to ensure that the user's eyes can perceive the same contrast ratio. That is to say, by reducing the contrast of the dominant eye and increasing the contrast of the amblyopic eye, in the subsequent amblyopic training process, the dominant eye and the amblyopic eye are displayed with different contrast images respectively to strengthen the amblyopic eye and suppress the dominant eye. Imaging ability to achieve the purpose of amblyopia training.

In the above implementation process, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye are determined according to the user's subjective feedback, but the contrast of the user's subjective feedback may often be biased. Therefore, in some other embodiments, multiple third test images are displayed in the display area corresponding to the dominant eye, and multiple fourth test images are displayed in the display area corresponding to the amblyopic eye. A first moving object is included, and a second moving object is included in the plurality of fourth test images. In this way, before determining the contrast of the reduced third test image as the image contrast corresponding to the dominant eye, and determining the increased contrast of the fourth test image as the image contrast corresponding to the amblyopic eye, the method further includes: by means of eye tracking. , determine the gaze trajectory of the dominant eye to the first moving target in a plurality of third test images, and the gaze trajectory of the amblyopic eye to the second moving target in a plurality of fourth test images, and obtain the first gaze trajectory and the second Gaze track. The actual motion trajectory of the first moving target and the actual motion trajectory of the second moving target are obtained, and the first actual motion trajectory and the second actual motion trajectory are obtained. If the first gaze trajectory matches the first actual motion trajectory, and the second gaze trajectory matches the second actual motion trajectory, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the main eye, and the contrast of the third test image after the decrease The contrast of the four test images was determined as the image contrast corresponding to the amblyopic eye.

That is, by means of eye tracking, it can be further accurately determined whether the image contrast corresponding to the eyes determined by the user's subjective feedback is accurate, so as to avoid errors caused by the subjective feedback.

Based on the above description, the types of amblyopia include strabismus amblyopia and anisometropic amblyopia. For different types of amblyopia, the operations for processing amblyopia training images are different, which will be introduced separately next.

The type of amblyopia in the amblyopic eye is strabismus amblyopia

In this case, the target object is included in the amblyopia training image, and the target object is used for amblyopia training. At this time, homography transformation is performed on the amblyopia training image according to the binocular deviation information. The amblyopia training image before homography transformation is used as the first training image, and the amblyopia training image after homography transformation is used as the second training image. According to the image contrast corresponding to the dominant eye, in the display area corresponding to the dominant eye The first training image is displayed, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye. By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined to obtain the first gaze position and the second gaze position. If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the amount of homography transformation of the amblyopia training image, and return The step of taking the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia training ends, or the amblyopia training image is not subjected to homography transformation. In this case, the second gaze position coincides with the actual position of the target object in the second training image.

A homography transformation refers to the mapping relationship of one image to another, usually including translation and/or rotation. Since eyes are generally not sensitive enough to rotation, in this embodiment of the present application, the homography transformation mainly refers to translation. In addition, based on the operation process of the above-mentioned binocular deviation, the binocular deviation information can be divided into deviation information in the horizontal direction and deviation information in the vertical direction. In this way, the realization process of performing homography transformation on the amblyopia training image according to the binocular deviation information is: The deviation information in the horizontal direction in the binocular deviation information is determined as the translation amount in the horizontal direction, and the deviation information in the vertical direction in the binocular deviation information is determined as the translation amount in the vertical direction. Then, the amblyopia training image is translated according to the translation amount in the horizontal direction and the translation amount in the vertical direction, so as to realize the homography transformation of the amblyopia training image.

For strabismus amblyopia, due to the deviation between the line of sight of the amblyopic eye and the line of sight of the dominant eye, it is necessary to perform homography transformation on the amblyopia training image to adjust the position of the target object in the amblyopia training image. Then, the unadjusted amblyopia training image is displayed in the display area corresponding to the dominant eye, and the adjusted amblyopia training image is displayed in the display area corresponding to the amblyopic eye. The images can be combined, that is, both eyes can see the same picture. Moreover, the two eyes simultaneously watch the amblyopia training image in the corresponding display area, which can ensure the stereoscopic vision of the two eyes.

In addition, based on the above description, the image contrast corresponding to the dominant eye may be different from the image contrast corresponding to the amblyopic eye. In this way, the first training image is displayed according to the image contrast corresponding to the dominant eye, and the second training image is displayed according to the image contrast corresponding to the amblyopic eye. The training images not only improve the imaging ability of the amblyopic eye, but also ensure the comfort of the user, make it easier for the user to accept the training method and training process, improve the training effect, and increase the user's viscosity.

The type of amblyopia in the amblyopic eye is anisometropic amblyopia

In this case, the target object is included in the amblyopia training image, and the target object is used for amblyopia training. At this time, the position and/or size of the target object in the amblyopic training image is adjusted multiple times to obtain multiple training images, the image scaling ratio of the amblyopic eye relative to the dominant eye is determined, and according to the image scaling ratio, the multiple training images are obtained. The dimensions of the training images are scaled. Taking the multiple training images before scaling as multiple third training images, and using the multiple training images after scaling as multiple fourth training images, in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye Display a plurality of third training images in sequence, and sequentially display a plurality of fourth training images in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, and display a plurality of third training images and a plurality of fourth training images The sequence and switching frequency are the same.

For anisometropic amblyopia, since the diopter of the amblyopic eye is higher than that of the dominant eye, the size of the image seen by the amblyopic eye may be different from that seen by the dominant eye. Therefore, in the training images for amblyopia After adjusting the position and/or size of the target object multiple times to obtain multiple training images, it is necessary to determine the image scaling ratio of the amblyopic eye relative to the dominant eye, and then scale the sizes of the multiple training images according to the image scaling ratio. , so that the images seen by the dominant eye and the amblyopic eye can be combined, that is, both eyes can see the same picture.

Wherein, the realization process of determining the image scaling ratio of the amblyopic eye relative to the dominant eye is: displaying the fifth test image in the display area corresponding to the dominant eye, displaying the sixth test image in the display area corresponding to the amblyopic eye, and displaying the fifth test image in the display area corresponding to the dominant eye. The test image includes a first test target, the sixth test image includes a second test target, and the ratio of the first test target and the second test target is the same. Scale the sixth test image. When the scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio. Fired when the target and the second test target.

That is, for the same eye, the first test object in the fifth test image and the second test object in the sixth test image are initially in the same proportion. Then, when the dominant eye is looking at the fifth test image and the amblyopic eye is looking at the sixth test image, by scaling the ratio of the sixth test image, the user feedbacks that both eyes of the user can see the first test target and the sixth test image with the same proportions. During the second test target, the image scaling ratio is determined by the size of the first test target and the scaled size of the second test target.

After performing amblyopia training according to different types of amblyopia in the embodiment of the present application, the effect of amblyopia training can also be fed back, so as to facilitate the user to adjust the training plan in a targeted manner. Next, two of the ways to feedback the effect of amblyopia training will be introduced.

In the first manner, the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye is determined by means of eye tracking, and the gaze position of the amblyopic eye is obtained. If the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, modify the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye to the reference display mode to indicate amblyopia training Effect.

The reference display mode may refer to modes such as highlighting, discoloration, and the like, which are not limited in this embodiment of the present application.

In the second method, the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye is determined by means of eye tracking, and the gaze position of the amblyopic eye is obtained. According to the gaze position of the amblyopic eye and the actual position of the target object, the amblyopia training curve is drawn to indicate the amblyopia training effect.

As an example, count the total training times in this amblyopia training cycle, and the percentage of the times that the gaze position of the amblyopic eye coincides with the actual position of the target object in the current amblyopia training cycle and the total training times. Then, the amblyopia training curve is drawn by combining the data of multiple amblyopia training cycles in history. That is, taking multiple amblyopia training cycles as the horizontal axis, and taking the percentage determined in each amblyopia training cycle as the vertical axis, the amblyopia training curve is drawn.

It should be noted that the above two manners may be used alone or in combination, which are not limited in the embodiments of the present application. Of course, in practical applications, the effect of amblyopia training can also be fed back in other ways. For example, after the amblyopia training is performed in each amblyopia training cycle, the diopter of the amblyopic eye can be detected, and then the amblyopia training curve is drawn with multiple amblyopic training cycles as the horizontal axis and the diopter of the amblyopic eye as the vertical axis.

In a second aspect, an amblyopia training device is provided, and the amblyopia training device has the function of implementing the behavior of the amblyopia training method in the first aspect. The amblyopia training device includes at least one module, and the at least one module is used to implement the amblyopia training method provided in the first aspect.

In a third aspect, an amblyopia training device is provided, the amblyopia training device includes a processor and a memory, and the memory is used for storing a program for executing the amblyopia training method provided in the first aspect, and for implementing the amblyopia training method provided in the first aspect. The data involved in the amblyopia training method. The processor is configured to execute programs stored in the memory. The amblyopia training device may also include a communication bus for establishing a connection between the processor and the memory.

In a fourth aspect, a computer-readable storage medium is provided, where instructions are stored in the computer-readable storage medium, when the computer-readable storage medium runs on a computer, the computer executes the amblyopia training method described in the first aspect.

In a fifth aspect, there is provided a computer program product comprising instructions, which, when run on a computer, cause the computer to execute the amblyopia training method of the above-mentioned first aspect.

The technical effects obtained by the second aspect, the third aspect, the fourth aspect and the fifth aspect are similar to the technical effects obtained by the corresponding technical means in the first aspect, and will not be repeated here.

The technical solutions provided in the embodiments of the present application can at least bring the following beneficial effects:

In the embodiment of the present application, by determining the type of amblyopia of the amblyopic eye, the amblyopia training image is processed in different ways according to the type of amblyopia of the amblyopic eye, and then amblyopia training is performed. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved.

Description of drawings

Fig. 1 is the architecture diagram of a kind of amblyopia training system provided by the embodiment of the present application;

2 is a flowchart of a method for amblyopia training provided by an embodiment of the present application;

3 is a schematic diagram of a plurality of binocular deviation information determined according to an embodiment of the present application;

4 is a schematic diagram of training strabismus amblyopia provided by an embodiment of the present application;

Fig. 5 is a kind of schematic diagram of strabismus amblyopia returning to front view provided by an embodiment of the present application;

6 is a schematic diagram of adjusting amblyopia training images according to the first method provided by an embodiment of the present application;

7 is a schematic diagram of a change in amblyopia training images displayed according to the first method provided by an embodiment of the present application;

8 is a schematic diagram of adjusting an amblyopia training image according to a second method provided by an embodiment of the present application;

9 is a schematic diagram of a change in amblyopia training images displayed in a second manner provided by an embodiment of the present application;

10 is a schematic diagram of adjusting amblyopia training images according to a third method provided by an embodiment of the present application;

11 is a schematic diagram of a change in amblyopia training images displayed in a third manner provided by an embodiment of the present application;

12 is a schematic diagram of adjusting an amblyopia training image according to a fourth method provided by an embodiment of the present application;

13 is a schematic diagram of a change in an amblyopia training image displayed in a fourth manner according to an embodiment of the present application;

14 is a schematic diagram of determining an image scaling ratio by displaying a test image on both eyes according to an embodiment of the present application;

15 is a schematic diagram of a binocular combined image provided by an embodiment of the present application;

16 is a schematic diagram of the first indication of amblyopia training effect provided by an embodiment of the present application;

17 is a schematic diagram of a second indication of amblyopia training effect provided by an embodiment of the present application;

18 is a schematic diagram of a third indication amblyopia training effect provided by an embodiment of the present application;

19 is a schematic structural diagram of an amblyopia training device provided by an embodiment of the present application;

20 is a schematic structural diagram of an amblyopia training device provided by an embodiment of the present application;

FIG. 21 is a schematic structural diagram of a terminal provided by an embodiment of the present application;

FIG. 22 is a schematic structural diagram of another terminal provided by an embodiment of the present application.

detailed description

In order to make the objectives, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

Before the amblyopia training method provided by the embodiments of the present application is explained in detail, the terms and implementation environments involved in the embodiments of the present application are introduced.

First, the terms involved in the embodiments of the present application are introduced.

Amblyopia: refers to an eye disease in which the eyeball has no organic lesions but the best corrected visual acuity is lower than normal vision. It is usually caused by monocular strabismus, anisometropia, high refractive error, and visual deprivation.

Monocular strabismus: refers to the imbalance of central control, the imbalance of extraocular muscle strength, the inability of both eyes to look at the target at the same time, the visual axis is in a state of separation, one of the eyes is looking at the target, and the other eye is deviating from the target. It can also be understood that due to the abnormal binocular interaction caused by the deviation of the eye position, the different objects (confusion) received by the fovea of the strabismus are inhibited, resulting in the best corrected visual acuity of the strabismus being lower than normal vision. Phenomenon.

anisometropia: refers to the difference in the refractive power of the eyes. Among them, amblyopia is formed in the eye with the higher diopter. Typically, anisometropia is considered monocular amblyopia.

High refractive error: refers to the fact that when the eye does not use adjustment, after the parallel light passes through the refraction of the eye, it cannot form a clear image on the retina, but images in front or behind the retina. Mainly refers to hyperopic refractive error or high astigmatism, which can cause form deprivation due to blurred objects in both eyes. Often, high refractive errors cause binocular amblyopia.

Visual deprivation: refers to a phenomenon that prevents the transmission of visual information to humans without damaging the tissue structure of the eye. It can also be understood as a phenomenon in which complete ptosis is caused due to the confusion of refractive interstitials (leukoplakia, cataract, vitreous inflammation or blood accumulation) and complete ptosis during the critical period of vision.

Dominant eye: refers to the eye with a lower refractive power, which is a relative concept to the amblyopic eye. That is, among the eyes of the user, the eye with a relatively low refractive power is the dominant eye.

Amblyopic eye: refers to the eye with higher refractive power, which is a relative concept to the dominant eye. That is, among the eyes of the user, the eye with relatively high refractive power is the amblyopic eye.

Eye tracking: refers to the tracking of eye movement by measuring the gaze position of the eye or the movement of the eye relative to the head. The most commonly used method is to obtain the position of the eye through a video shooting device.

Next, the implementation environment involved in the embodiments of the present application is introduced.

Please refer to FIG. 1 . FIG. 1 is an architecture diagram of an amblyopia training system provided by an embodiment of the present application. The system includes a terminal device 101 and an amblyopia training device 102, and communication between the terminal device 101 and the amblyopia training device 102 is performed in a wireless or wired manner.

The terminal device 101 is configured to display the amblyopia training image, and project the displayed amblyopia training image to the amblyopia training device 102 . The amblyopia training device 102 includes two display areas, the two display areas correspond to the eyes of the user, respectively, a display area corresponding to the main eye and a display area corresponding to the amblyopic eye. The two display areas of the amblyopia training device 102 are used to display the amblyopia training image projected by the terminal device 101, and the amblyopia training device 102 is also used to determine the amblyopia type of the user's amblyopic eye, and process the amblyopia training image according to the amblyopia type, thereby Amblyopia training is performed through the amblyopia training images displayed on the two display areas to correct the vision of the amblyopic eye.

In some embodiments, the amblyopia training device 102 includes a display module, an eye tracking module, and an image processing module. The display module is used to display the amblyopia training image projected by the terminal device 101 . The eye tracking module is used to determine the type of amblyopia of the user's amblyopic eye. The image processing module is used to process the amblyopia training images according to the type of amblyopia.

Optionally, the eye tracking module is further configured to determine the gaze position of the user's eyes, so as to determine the amblyopia training effect.

It should be noted that the terminal device 101 is any electronic product that can perform human-computer interaction with the user in one or more ways, such as a keyboard, a touch panel, a touch screen, a remote control, a voice interaction or a handwriting device. For example, personal computer (PC), mobile phone, smart phone, personal digital assistant (PDA), wearable device, Pocket PC (Pocket PC), tablet computer, smart car, smart TV, smart speaker Wait. The amblyopia training device 102 is any kind of head-mounted device with two display areas. For example, virtual reality (VR) glasses, VR helmets, movie viewing glasses, augmented reality (AR) glasses, AR helmets, mixed reality (mix reality, MR) glasses, MR helmets and other devices. The two display areas of the amblyopia training device 102 are two display screens of the amblyopia training device 102 , or different display areas in the same display screen of the amblyopia training device 102 .

It is worth noting that in the above system architecture, the terminal device 101 is used to project the amblyopia training image to the amblyopia training device 102 , and then the amblyopia training device 102 performs amblyopia training on the user's amblyopic eyes. In some other embodiments, the above system architecture does not include the terminal device 101, that is, the amblyopia training device 102 can store amblyopia training images, and directly display the stored amblyopia training images during the amblyopia training process.

Next, the amblyopia training method provided by the embodiment of the present application is explained in detail.

Please refer to FIG. 2. FIG. 2 is a flowchart of an amblyopia training method provided by an embodiment of the present application. The method is applied to an amblyopia training device. The amblyopia training device includes two display areas, and the two display areas correspond to user eyes. The method includes the following steps.

Step 201: Determine the dominant eye and the amblyopic eye of the user.

In some embodiments, the process of determining the dominant eye and the amblyopic eye of the user is as follows: detecting the diopter of the user's both eyes respectively, determining the eye with a low diopter in the user's two eyes as the dominant eye, and determining the user's binocular with a high diopter. eyes were identified as amblyopic.

Step 202: Determine the type of amblyopia of the amblyopic eye, where the type of amblyopia includes strabismus amblyopia or anisometropic amblyopia.

In some embodiments, the realization process of determining the amblyopia type of the amblyopic eye is: displaying the first test image in the display area corresponding to the dominant eye, and displaying the second test image in the display area corresponding to the amblyopic eye. By means of eye tracking, determine the coordinates of the gaze position of the dominant eye in the displayed first test image, and the coordinates of the gaze position of the amblyopic eye in the displayed second test image, and obtain the coordinates of the first gaze position and the 2. Gaze position coordinates. According to the coordinates of the first gaze position and the coordinates of the second gaze position, the type of amblyopia of the amblyopic eye is determined.

The first test image and the second test image may be the same or different. The first test image and the second test image may be projected by the terminal device to the amblyopia training device. Of course, the first test image and the second test image may also be stored by the amblyopia training device itself, which is not the case in the embodiment of the present application. Do limit.

The position transformation matrix refers to the transformation matrix between the position of the eyeball and the position of the target point in the image. The determination method of the position transformation matrix may be as follows: the amblyopia training device sequentially displays multiple target points, and then, when the user's eyes are gazing at the multiple target points, determines the user's eyeball position coordinates by means of eye tracking, and obtains the corresponding Multiple eye position coordinates. Next, the position transformation matrix is determined by the following formula.

P ₁ =T*P ₂

Among them, in the above formula, P ₁ refers to the matrix formed by the coordinates of multiple target points, T refers to the position transformation matrix, and P ₂ refers to the user's eyes looking at the multiple target points, determined by eye tracking. of multiple eye position coordinates.

The binocular deviation information is also called the distance between the binocular gaze positions. Therefore, in some embodiments, the amblyopia training device determines the binocular deviation information by the following formula according to the first gaze position coordinates and the second gaze position coordinates.

Wherein, in the above formula, ED is the binocular deviation information, (L _x , _Ly ) are the coordinates of the first gaze position, and (R _x , R _y ) are the coordinates of the second gaze position.

That is, when it is determined that there is an amblyopic eye in both eyes of the user, it can be determined whether the amblyopia type of the amblyopic eye is strabismus amblyopia or anisometropic amblyopia through the first threshold.

The above implementation process is to perform a test to determine the binocular deviation information by means of eye tracking, so as to determine the type of amblyopia of the amblyopic eye. In other embodiments, the binocular deviation information can be determined by performing multiple tests by means of eye tracking, so as to determine the type of amblyopia of the amblyopic eye. That is, in other embodiments, the realization process of determining the amblyopia type of the amblyopic eye is: displaying the first test image in the display area corresponding to the dominant eye, and displaying the second test image in the display area corresponding to the amblyopic eye. . By means of eye tracking, the coordinates of multiple gaze positions of the dominant eye in the displayed first test image and the coordinates of multiple gaze positions of the amblyopic eye in the displayed second test image are determined to obtain multiple first A gaze position coordinate and a plurality of second gaze position coordinates. The amblyopia type of the amblyopic eye is determined according to the plurality of first gaze position coordinates and the plurality of second gaze position coordinates.

Wherein, the realization process of determining the amblyopia type of the amblyopic eye according to a plurality of first gaze position coordinates and a plurality of second gaze position coordinates is as follows: according to the first gaze position coordinates and the second gaze position coordinates determined at the same time, determine a pair of eyes The deviation information is obtained, thereby obtaining a plurality of binocular deviation information, and the binocular deviation information refers to the deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye. The statistical values of the plurality of binocular deviation information are determined, and if the statistical values of the plurality of binocular deviation information are greater than or equal to the first threshold, it is determined that the amblyopia type of the amblyopic eye is strabismus amblyopia. If the statistical value of the plurality of binocular deviation information is less than the first threshold, it is determined that the amblyopia type of the amblyopic eye is anisometropic amblyopia.

For example, it is assumed that the amblyopia training device displays the first test image in the display area corresponding to the dominant eye, and after displaying the second test image in the display area corresponding to the amblyopic eye, determines that the dominant eye is in the first test image by means of eye tracking The coordinates of the seven gaze positions in the test image and the coordinates of the seven gaze positions of the amblyopic eye in the second test image are used to obtain seven first gaze position coordinates and seven second gaze position coordinates. According to the 7 first gaze position coordinates and the 7 second gaze position coordinates, the determined 7 binocular deviation information is shown in FIG. 3 . By determining the statistical values of the seven binocular deviation information, the type of amblyopia of the amblyopic eye can be determined.

It should be noted that the statistical value of the plurality of binocular deviation information refers to an average value, a median value, and the like of the plurality of binocular deviation information, which is not limited in this embodiment of the present application.

Step 203: when the type of amblyopia of the amblyopic eye is determined to be strabismus amblyopia, perform homography transformation on the amblyopic training image to perform amblyopia training, and when the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia The training images are subjected to image resizing processing for amblyopia training.

In other embodiments, in order to improve the imaging ability of the amblyopic eye, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye may also be determined, so that the user's binocular perception ability is the same. In this way, the amblyopic training image can be displayed in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and the amblyopic training image can be displayed in the corresponding amblyopic eye after homography transformation according to the image contrast corresponding to the amblyopic eye. Amblyopia training is performed in the display area of the user so that the images seen by the user's eyes can be combined. Or, display the amblyopic training image in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye, and perform image size adjustment processing on the amblyopic training image according to the image contrast corresponding to the amblyopic eye and display it on the display corresponding to the amblyopic eye. In the region, amblyopia training is performed so that the images seen by the user's eyes can be combined.

For the above two embodiments, the operation of processing the amblyopia training image according to the type of amblyopia of the amblyopic eye and the operation of performing amblyopia training are the same, the difference is only whether it is necessary to determine the image contrasts corresponding to the eyes, and then in the amblyopia training operation. The process of displaying amblyopia training images according to the respective image contrasts of the eyes. Next, taking the second embodiment as an example, the following steps (1)-(2) are used to introduce the amblyopia training process provided by the embodiment of the present application.

(1) Determine the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye, so that the user's binocular perception ability is the same.

In some embodiments, the amblyopia training device displays the third test image in the display area corresponding to the dominant eye, and displays the fourth test image in the display area corresponding to the amblyopic eye. After that, the contrast of the displayed third test image is decreased, and the contrast of the displayed fourth test image is increased. When the contrast determination instruction is detected, the contrast of the reduced third test image is determined as the image contrast corresponding to the dominant eye, and the contrast of the increased fourth test image is determined as the image contrast corresponding to the amblyopic eye, and the contrast determination instruction is It is triggered when the user feeds back that the contrasts that his eyes can perceive are the same according to the displayed third test image and the fourth test image.

The process of reducing the contrast of the displayed third test image and increasing the contrast of the displayed fourth test image is as follows: reducing the contrast of the third test image according to the adjustment step size, and increasing the contrast of the fourth test image according to the adjustment step size . The adjusted third and fourth test images are displayed. When the contrast adjustment instruction is detected, continue to reduce the contrast of the third test image according to the adjustment step size, and increase the contrast of the fourth test image according to the adjustment step size, and display the contrast-adjusted third test image and the fourth test image, until a contrast determination command is detected. Wherein, the contrast adjustment instruction is triggered by the user according to the displayed third test image and the fourth test image when the contrast between the two eyes is different.

It should be noted that in the process of adjusting the contrast of the third test image by adjusting the step size, the display area corresponding to the dominant eye can be blocked, and then displayed after adjustment, and the user determines whether the contrast of the two eyes is the same. Of course, in the process of adjusting the contrast of the third test image by adjusting the step size, the display area corresponding to the dominant eye may not be blocked. Similarly, in the process of adjusting the contrast of the fourth test image by adjusting the step length, the display area corresponding to the amblyopic eye can be occluded, and then displayed after the adjustment, and the user determines whether the contrast of the two eyes is the same. Of course, in the process of adjusting the contrast of the fourth test image by adjusting the step size, the display area corresponding to the amblyopic eye may not be blocked, which is not limited in this embodiment of the present application.

In addition, the contrast of the third test image and the contrast of the fourth test image can be adjusted in sequence. For example, the contrast of the third test image is first decreased according to the adjustment step, and then the adjusted third test image and the fourth test image are displayed. If the user feedbacks that the contrast between the eyes is different according to the displayed third test image and the fourth test image, then increase the contrast of the fourth test image according to the adjustment step, and display the adjusted third test image and the fourth test image, if The user feeds back that the contrast of the eyes is different according to the displayed third test image and the fourth test image, then continues to reduce the contrast of the third test image according to the adjustment step, until the user feeds back the eyes according to the displayed third test image and the fourth test image. the contrast is the same.

Of course, reducing the contrast of the third test image first, and then increasing the contrast of the fourth test image is just an example. Of course, the contrast of the fourth test image can also be increased first, and then the contrast of the third test image can be reduced. This embodiment of the present application The adjustment order of the third test image and the fourth test image is not limited.

The third test image and the fourth test image may be the same or different. The third test image and the fourth test image may be projected by the terminal device to the amblyopia training device. Of course, the third test image and the fourth test image may also be stored by the amblyopia training device itself, which is not the case in the embodiment of the present application. Do limit.

When the third test image and the fourth test image are stored by the amblyopia training device itself, the contrast of the test images can be adjusted by manually adjusting the amblyopia training device according to the adjustment step. However, when the third test image and the fourth test image are projected from the terminal device to the amblyopia training device, not only can the amblyopia training device be manually adjusted according to the adjustment step to adjust the contrast of the test image, but also the contrast of the test image can be adjusted according to the adjustment step. Adjust the contrast of the test image by manually adjusting the terminal equipment for a long time.

Wherein, the adjustment step size for each contrast adjustment may be the same or different. The adjustment step size may be a preset step size, or may be calculated according to the contrast between the third test image and the fourth test image.

As an example, the contrast difference between the third test image and the fourth test image is determined, and the contrast difference is divided by a preset number of times to obtain the adjustment step size. That is to say, in this way, after adjusting the third test image and the fourth test image a preset number of times, the contrast seen by the user's eyes will be the same.

As another example, the contrast difference between the third test image and the fourth test image is determined, and the corresponding adjustment step is obtained from the stored correspondence between the contrast difference and the adjustment step according to the contrast difference. Wherein, the corresponding relationship between the contrast difference and the adjustment step size may be determined empirically in advance.

In the above implementation process, the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye are determined according to the user's subjective feedback, but the contrast of the user's subjective feedback may often be biased. Therefore, in some other embodiments, multiple third test images are displayed in the display area corresponding to the dominant eye, and multiple fourth test images are displayed in the display area corresponding to the amblyopic eye. A first moving object is included, and a second moving object is included in the plurality of fourth test images. In this way, before determining the contrast of the reduced third test image as the image contrast corresponding to the dominant eye, and determining the increased contrast of the fourth test image as the image contrast corresponding to the amblyopic eye, the method further includes: by means of eye tracking. , determine the gaze trajectory of the dominant eye to the first moving target in a plurality of third test images, and the gaze trajectory of the amblyopic eye to the second moving target in a plurality of fourth test images, and obtain the first gaze trajectory and the second Gaze track. The actual motion trajectory of the first moving target and the actual motion trajectory of the second moving target are acquired, and the first actual motion trajectory and the second actual motion trajectory are obtained. If the first gaze trajectory matches the first actual motion trajectory, and the second gaze trajectory matches the second actual motion trajectory, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the main eye, and the contrast of the third test image after the increase The contrast of the four test images was determined as the image contrast corresponding to the amblyopic eye.

As an example, matching the first gaze trajectory with the first actual motion trajectory means that the degree of coincidence between the first gaze trajectory and the first actual motion trajectory is greater than a certain threshold. Similarly, the matching of the second gaze trajectory with the second actual motion trajectory is It means that the degree of coincidence between the second gaze trajectory and the second actual motion trajectory is greater than a certain threshold.

The first moving target and the second moving target may be the same or different. In addition, the manner of determining the first gaze trajectory and the second gaze trajectory is similar to the manner in which the coordinates of the two gaze positions are determined when the binocular deviation is determined, which is not repeated in this embodiment of the present application.

Further, if the first gaze trajectory does not match the first actual motion trajectory, the contrast of the third test image needs to be re-adjusted, and during the adjustment process, the first gaze trajectory is re-determined by means of eye tracking. , if the re-determined first gaze trajectory does not match the first actual motion trajectory, continue to adjust until the re-determined first gaze trajectory matches the first actual motion trajectory to obtain the image contrast corresponding to the dominant eye. Similarly, if the second gaze trajectory does not match the second actual motion trajectory, then the contrast of the fourth test image needs to be re-adjusted, and during the adjustment process, the second gaze trajectory is re-determined by eye tracking. , if the re-determined second gaze trajectory does not match the second actual motion trajectory, continue to adjust until the re-determined second gaze trajectory matches the second actual motion trajectory to obtain the image contrast corresponding to the amblyopic eye.

(2) According to the image contrast corresponding to the dominant eye, display the amblyopia training image in the display area corresponding to the dominant eye, and perform homography transformation processing on the amblyopia training image according to the image contrast corresponding to the amblyopic eye and the amblyopia type of the amblyopic eye Or the image size is adjusted and displayed in the display area corresponding to the amblyopic eye, so that the images seen by the user's eyes can be combined to perform amblyopia training.

The type of amblyopia in the amblyopic eye is strabismus amblyopia

In addition, the subsequent reduction of the homography transformation amount of the amblyopia training image may refer to reducing the translation amount in the horizontal direction and/or the translation amount in the vertical direction of the amblyopia training image, and the reduction amount in the horizontal direction and the vertical direction The same or different, and this reduction can be set as desired.

For example, as shown in Figure 4, it is assumed that the user's left eye is strabismus amblyopia, that is to say, the user's left eye is strabismus, and the right eye is emmetropic. At this time, there is a deviation between the line of sight of the left eye and the line of sight of the right eye . In order to perform amblyopia training on the user's left eye, the position of the target object in the amblyopia training image needs to be shifted to the left. Afterwards, the amblyopia training image before translation is used as the first training image, and the amblyopia training image after translation is used as the second training image, and the first training image is displayed in the display area corresponding to the right eye according to the image contrast corresponding to the right eye, According to the image contrast corresponding to the left eye, the second training image is displayed in the display area corresponding to the left eye.

By means of eye tracking, the gaze position of the right eye in the first training image and the gaze position of the left eye in the second training image are determined to obtain the first gaze position and the second gaze position. If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the leftward translation of the target object in the amblyopia training image , and continue to display the corresponding amblyopia training image in the display area of both eyes. If the second gaze position coincides with the actual position of the target object in the second training image without the target object in the amblyopia training image being translated, then it can be determined that the user's left eye has returned to emmetropia, that is, as shown in Figure 5.

The manner of determining the first gaze position and the second gaze position is similar to the manner in which the coordinates of the two gaze positions are determined when the binocular deviation is determined, which is not repeated in this embodiment of the present application.

It should be noted that the whole process of amblyopia training can be carried out in multiple cycles, but after each cycle of amblyopia training, the image contrast corresponding to the eyes may change. Therefore, under normal circumstances, before each cycle of amblyopia training , it is necessary to determine the image contrast corresponding to the user's eyes. For the specific implementation process, refer to the foregoing description, which will not be repeated here.

The type of amblyopia in the amblyopic eye is anisometropic amblyopia

Among them, there are many ways to adjust the position and/or size of the target object in the amblyopia training image multiple times, and the following four ways will be introduced.

In the first method, the position of the target object in the amblyopia training image is adjusted in the order of up, down, left, and right, and four training images are obtained. That is, for the four training images, the position of the target object is located in the upper position, the lower position, the left position and the right position in the image in sequence.

For example, as shown in Figure 6, the position of the target object in the amblyopia training image is adjusted in the order of up, down, left and right, and the four training images obtained are training image 1 to training image 4 respectively. In this way, after the four training images are displayed in sequence in the display area, the user will see the images as shown in Figure 7, that is, the target objects are in the

positions

1, 2, 3, and 4 in Figure 7. Transform.

In the second method, the position of the target object in the amblyopia training image is adjusted in a rotating manner to obtain multiple training images.

For example, as shown in FIG. 8 , after adjusting the position of the target object in the amblyopia training image in a rotating manner, eight training images are obtained, namely training image 1 to training image 8 . In this way, after the eight training images are displayed in sequence in the display area, the user will see the images shown in FIG. The positions of 5, 6, 7, and 8 are transformed.

In the third method, the position of the target object in the amblyopia training image is randomly adjusted to obtain multiple training images.

For example, as shown in FIG. 10 , after randomly adjusting the position of the target object in the amblyopia training image, nine training images are obtained, which are respectively training image 1 to training image 9 . In this way, after the nine training images are displayed in sequence in the display area, the user will see the images as shown in Figure 11, that is, the target object will follow

steps

1, 2, 3, 4, and 4 in Figure 11. The positions of 5, 6, 7, 8, and 9 are transformed.

In the fourth manner, the size of the target object in the amblyopia training image is adjusted multiple times to obtain multiple training images.

For example, as shown in FIG. 12 , the size of the target object in the amblyopia training image is adjusted four times to obtain four training images, which are respectively training image 1 to training image 4 . In this way, after the four training images are displayed in sequence in the display area, the user will see the images shown in FIG. 13 , that is, the target object presents a shape from far to near.

It should be noted that the first three methods are to adjust the position of the target object, and the last method is to adjust the size of the target object. In practical applications, the first three methods can also adjust the size of the target object on the basis of the position adjustment, that is, the position and the size are superimposed to adjust. Similarly, for the last method, on the basis of adjusting the size of the target object, the position of the target object may also be adjusted, which will not be repeated in this embodiment of the present application.

The embodiment of the present application trains the extraocular muscles and improves the imaging ability of the amblyopic eye through the movement modes such as up and down, left and right, rotation, random, far and near, so as to achieve the purpose of training the amblyopic eye.

Wherein, the realization process of determining the image scaling ratio of the amblyopic eye relative to the dominant eye is: displaying the fifth test image in the display area corresponding to the dominant eye, displaying the sixth test image in the display area corresponding to the amblyopic eye, and displaying the fifth test image in the display area corresponding to the dominant eye. The test image includes a first test target, the sixth test image includes a second test target, and the ratio of the first test target and the second test target is the same. Scales the ratio of the sixth test image displayed. When the scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user feedback according to the displayed fifth test image and the sixth test image Triggered when your own eyes can see the first test target and the second test target with the same proportions.

That is, for the same eye, the first test object in the fifth test image and the second test object in the sixth test image are initially in the same proportion. Then, in the process that the dominant eye is watching the fifth test image and the amblyopic eye is watching the sixth test image, the ratio of the sixth test image is adjusted, and the user feeds back his own When both eyes can see the first test target and the second test target with the same scale, the image scaling ratio is determined by the size of the first test target and the size of the scaled second test target.

As an example, the ratio between the height of the scaled second test target and the height of the unscaled first test target is determined as the scaling ratio in the vertical direction, and the width of the scaled second test target is The ratio between the widths of the unscaled first test targets is determined as the scaling ratio in the horizontal direction. Then, the scaling ratio in the vertical direction and the scaling ratio in the horizontal direction are determined as the image scaling ratios of the amblyopic eye relative to the dominant eye.

In this example, according to the image scaling ratio, the realization process of scaling the sizes of the multiple training images is: multiplying the heights of the multiple training images by the scaling ratio in the vertical direction, and then scaling the multiple training images. The width is multiplied by the horizontal scaling ratio to obtain multiple training images after scaling.

As another example, the ratio between the height of the unscaled first test target and the height of the scaled second test target is determined as the scaling ratio in the vertical direction, and the width of the unscaled first test target is The ratio between the scaled width of the second test target and the scaled width of the second test target is determined as the scaling ratio in the horizontal direction. Then, the scaling ratio in the vertical direction and the scaling ratio in the horizontal direction are determined as the image scaling ratios of the amblyopic eye relative to the dominant eye.

In this example, according to the image scaling ratio, the realization process of scaling the sizes of the multiple training images is: dividing the heights of the multiple training images by the scaling ratio in the vertical direction, and then dividing the multiple training images by the scaling ratio in the vertical direction. The width is divided by the horizontal scaling ratio to obtain multiple training images after scaling.

It should be noted that the above is to determine the image scaling ratio by scaling the ratio of the sixth test image, of course, the image scaling ratio can also be determined by scaling the ratio of the fifth test image. In addition, in order to facilitate binocular image formation, the backgrounds of the fifth test image and the sixth test image may be the same.

For example, as shown in FIG. 14 , assuming that the left eye is the dominant eye and the right eye is the amblyopic eye, a fifth test image is displayed in the display area of the left eye, the fifth test image includes the first test object, and the first test object’s The height and width are both 2 cm, and a sixth test image is displayed in the display area of the right eye. The sixth test image includes a second test target, and the height and width of the second test target are also 2 cm. After that, the scale of the sixth test image is scaled. When the image seen by the user is the image shown in FIG. 15 , that is, when the outer contour of the second test target is inscribed with the outer contour of the first test target, it is determined that the user’s eyes can see the first test target with the same proportion. A test target and a second test target. At this time, it is determined that the height and width of the scaled second test target are both 3 cm, then the image scaling ratio is determined to be 3:2. In this way, for amblyopic eyes, the size of multiple training images needs to be multiplied by the image scaling ratio to perform amblyopia training subsequently.

It should be noted that the embodiment of the present application can not only determine the image scaling ratio of the amblyopic eye relative to the dominant eye, but also determine the rotation angle of the amblyopic eye relative to the dominant eye. However, since eyes are usually not sensitive enough to rotation, the embodiment of the present application mainly determines the image scaling ratio, and then performs scaling processing on the image for amblyopia training for amblyopic eyes.

In addition, when performing amblyopia training according to different types of amblyopia, the diopter of the amblyopic eye can also be corrected, and then the amblyopia training is performed in the above manner.

For example, as shown in FIG. 16 , if the gaze position of the amblyopic eye coincides with the actual position of the target object, then the target object is subjected to discoloration processing. In this way, for the user, after several times of training, the number of times the gaze position coincides with the actual position can be known, so as to determine the training effect of amblyopia, which is convenient for adjusting the training plan at any time.

For example, as shown in Figure 17, the amblyopia training device has performed amblyopia training in 7 amblyopia training cycles. During these 7 amblyopia training cycles, the number of times that the gaze position of the amblyopic eye coincides with the actual position of the target object and the percentage of the total number of training times are respectively 50%, 60%, 70%, 80%, 90%, 95%, 100%. At this time, the drawn amblyopia training curve is shown in FIG. 17 .

It should be noted that the above two manners may be used alone or in combination, which are not limited in the embodiments of the present application. Of course, in practical applications, the effect of amblyopia training can also be fed back in other ways. For example, after performing amblyopia training in each amblyopic training cycle, the diopter of the amblyopic eye can be detected, and then, with multiple amblyopic training cycles as the horizontal axis and the diopter of the amblyopic eye as the vertical axis, the amblyopia training curve is drawn. As shown in Figure 18, the amblyopia training equipment performed amblyopia training in 11 amblyopia training cycles. During these 11 amblyopia training cycles, the diopter of amblyopic eyes were 275, 275, 275. At this time, the drawn amblyopia training curve is shown in FIG. 18 .

In addition, in the embodiment of the present application, the amblyopia training effect can be fed back through the amblyopia training device, and the amblyopia training effect can also be fed back through the terminal device. That is, after the amblyopia training device determines the effect of the amblyopia training, it can directly feed back to the user. Of course, the amblyopia training device can also send the amblyopia training effect to the terminal device, and the terminal device directly feeds it back to the user.

The above implementation process is used to determine the gaze position of the amblyopic eye, thereby feeding back the effect of amblyopia training. Of course, in other embodiments, the gaze position of the dominant eye can also be determined, so as to feed back the training effect of amblyopia. The method of determining the gaze position of the dominant eye and feeding back the training effect of amblyopia is the same as the method for the amblyopic eye above, which will not be repeated in this embodiment of the present application.

In the embodiment of the present application, amblyopia training is performed by determining the amblyopia type of the amblyopic eye, and processing the amblyopia training image according to the amblyopia type of the amblyopic eye. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved. Moreover, during the amblyopia training process in the embodiment of the present application, the amblyopia training images are displayed in the display areas corresponding to both eyes, so that the images seen by the two eyes can be combined and the stereoscopic vision of the eyes can be improved. Furthermore, after the amblyopia training is performed in the embodiment of the present application, the effect of the amblyopia training can also be fed back, so that the user can adjust the amblyopia training plan in a targeted manner and improve the user's viscosity. Finally, the weak training device provided by the embodiment of the present application integrates amblyopia detection, training, and effect feedback, so that the user can perform targeted training more flexibly, and prompt training efficiency and training effect.

19 is a schematic structural diagram of an amblyopia training device provided in an embodiment of the present application. The amblyopia training device can be realized by software, hardware or a combination of the two to become part or all of the amblyopia training device, and the amblyopia training device can be shown in FIG. 1 Amblyopia training device shown. Referring to FIG. 19 , the apparatus includes: a first determination module 1901 , a second determination module 1902 , a first amblyopia training module 1903 and a second amblyopia training module 1904 .

The first determination module 1901 is used to determine the dominant eye and the amblyopic eye of the user;

The second determining module 1902 is configured to determine the type of amblyopia of the amblyopic eye, where the type of amblyopia includes strabismus amblyopia or anisometropic amblyopia;

The first amblyopia training module 1903 is used to perform amblyopia training on the amblyopia training image by performing homography transformation processing on the amblyopia type when the amblyopia type of the amblyopic eye is determined as strabismus amblyopia;

The second amblyopia training module 1904 is configured to perform amblyopia training by performing image size adjustment processing on the amblyopia training image when the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia.

Optionally, the second determining module 1902 includes:

a display sub-module for displaying the first test image in the display area corresponding to the dominant eye, and displaying the second test image in the display area corresponding to the amblyopic eye;

The first determination submodule is used to determine the coordinates of the gaze position of the dominant eye in the displayed first test image and the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking, obtain the coordinates of the first gaze position and the coordinates of the second gaze position;

The second determination submodule is configured to determine the type of amblyopia of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position.

Optionally, the second determination submodule is specifically used for:

According to the coordinates of the first gaze position and the coordinates of the second gaze position, the binocular deviation information is determined, and the binocular deviation information refers to the deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye;

If the binocular deviation information is greater than or equal to the first threshold, then determine that the amblyopia type of the amblyopic eye is strabismus amblyopia;

If the binocular deviation information is less than the first threshold, it is determined that the type of amblyopia of the amblyopic eye is anisometropic amblyopia.

Optionally, the first determining module is specifically used for:

detecting the diopter of the user's eyes;

An eye with a low refractive power in both eyes of the user is determined as a dominant eye, and an eye with a high refractive power in both eyes of the user is determined as an amblyopic eye.

Optionally, the first amblyopia training module 1903 includes:

The first amblyopia training sub-module is used for displaying the amblyopia training image in the display area corresponding to the dominant eye, and performing homography transformation on the amblyopia training image and displaying it in the display area corresponding to the amblyopia eye, so that the user's eyes can see the amblyopia training image. The obtained images can be combined for amblyopia training.

Optionally, the second amblyopia training module 1904 includes:

The second amblyopia training sub-module is used to display the amblyopia training image in the display area corresponding to the dominant eye, adjust the image size of the amblyopia training image and display it in the display area corresponding to the amblyopia eye, so that the user's eyes can see the amblyopia training image The images can be combined for amblyopia training.

Optionally, the device also includes:

The third determining module is configured to determine the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye, so that the user's binocular perception ability is the same.

Optionally, the third determining module is specifically used for:

displaying the third test image in the display area corresponding to the dominant eye, and displaying the fourth test image in the display area corresponding to the amblyopic eye;

reducing the contrast of the displayed third test image and increasing the contrast of the displayed fourth test image;

The third determination sub-module is configured to, when the contrast determination instruction is detected, determine the contrast of the reduced third test image as the image contrast corresponding to the dominant eye, and determine the contrast of the increased fourth test image as the image contrast corresponding to the amblyopic eye Image contrast, the contrast determination instruction is triggered when the user feeds back that the contrast that can be perceived by both eyes of the user is the same according to the displayed third test image and the fourth test image.

Optionally, a target object is included in the amblyopia training image, and the target object is used for amblyopia training;

The first amblyopia training sub-module is specifically used for:

Perform homography transformation on the amblyopic training image according to the binocular deviation information, where the binocular deviation information refers to the deviation information between the sight direction of the amblyopic eye and the sight direction of the dominant eye;

The amblyopia training image before homography transformation is used as the first training image, and the amblyopia training image after homography transformation is used as the second training image. According to the image contrast corresponding to the dominant eye, in the display area corresponding to the dominant eye Display the first training image, and display the second training image in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye;

Determine the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image by means of eye tracking, to obtain the first gaze position and the second gaze position;

If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, reduce the amount of homography transformation of the amblyopia training image, and return The step of taking the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia training ends, or the amblyopia training image is not subjected to homography transformation. In this case, the second gaze position coincides with the actual position of the target object in the second training image.

The second amblyopia training sub-module is specifically used for:

If the type of amblyopia of the amblyopic eye is anisometropic amblyopia, the position and/or size of the target object in the amblyopia training image is adjusted multiple times to obtain multiple training images;

Determine the image scaling ratio of the amblyopic eye relative to the dominant eye;

Scale the size of multiple training images according to the image scaling ratio;

Taking the multiple training images before scaling as multiple third training images, and using the multiple training images after scaling as multiple fourth training images, in the display area corresponding to the dominant eye according to the image contrast corresponding to the dominant eye Display a plurality of third training images in sequence, and sequentially display a plurality of fourth training images in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, and display a plurality of third training images and a plurality of fourth training images The sequence and switching frequency are the same.

Optionally, the second amblyopia training submodule is also used for:

A fifth test image is displayed in the display area corresponding to the dominant eye, and a sixth test image is displayed in the display area corresponding to the amblyopic eye. The fifth test image includes the first test object, the sixth test image includes the second test object, and the sixth test image includes the second test object. The ratio of the first test target and the second test target is the same;

scaling the ratio of the sixth test image displayed;

When the scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user feedback according to the displayed fifth test image and the sixth test image Triggered when your own eyes can see the first test target and the second test target with the same proportions.

Optionally, the device also includes:

a fourth determining module, configured to determine the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye by means of eye tracking, so as to obtain the gaze position of the amblyopic eye;

The training effect indication module is used to modify the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye as a reference if the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye The display mode is used to indicate the amblyopia training effect, and/or, according to the gaze position of the amblyopic eye and the actual position of the target object, an amblyopia training curve is drawn to indicate the amblyopia training effect.

In the embodiment of the present application, amblyopia training is performed by determining the amblyopia type of the amblyopic eye, and processing the amblyopia training image according to the amblyopia type of the amblyopic eye. That is to say, by distinguishing the types of amblyopia, so as to carry out targeted training according to different symptoms, the training effect of amblyopia can be improved. Moreover, during the amblyopia training process in the embodiment of the present application, the amblyopia training images are displayed in the display areas corresponding to both eyes, so that the images seen by the two eyes can be combined and the stereoscopic vision of the eyes can be improved.

It should be noted that: when the amblyopia training device provided in the above-mentioned embodiment performs amblyopia training, only the division of the above-mentioned functional modules is used for illustration. That is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the amblyopia training device and the amblyopia training method embodiments provided by the above embodiments belong to the same concept, and the specific implementation process thereof is detailed in the method embodiments, which will not be repeated here.

Please refer to FIG. 20 , which is a schematic structural diagram of an amblyopia training device according to an embodiment of the present application. The amblyopia training device includes at least one processor 2001 , a communication bus 2002 , a memory 2003 and at least one communication interface 2004 .

The processor 2001 may be a general-purpose central processing unit (CPU), a network processor (NP), a microprocessor, or may be one or more integrated circuits for implementing the solution of the present application, such as , an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above-mentioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a general array logic (generic array logic, GAL) or any combination thereof.

The communication bus 2002 is used to transfer information between the aforementioned components. The communication bus 2002 can be divided into an address bus, a data bus, a control bus, and the like. For ease of presentation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

The memory 2003 may be read-only memory (ROM), random access memory (RAM), or electrically erasable programmable read-only memory (electrically erasable programmable read-only memory). , EEPROM), optical disc (including compact disc read-only memory, CD-ROM), compact disc, laser disc, digital versatile disc, Blu-ray disc, etc.), magnetic disk storage media or other magnetic storage devices, or capable of Any other medium for carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but not limited thereto. The memory 2003 may exist independently and be connected to the processor 2001 through the communication bus 2002 . The memory 2003 may also be integrated with the processor 2001 .

Communication interface 2004 uses any transceiver-like device for communicating with other devices or a communication network. The communication interface 2004 includes a wired communication interface, and may also include a wireless communication interface. Wherein, the wired communication interface may be, for example, an Ethernet interface. The Ethernet interface can be an optical interface, an electrical interface, or a combination thereof. The wireless communication interface may be a wireless local area network (wireless local area network, WLAN) interface, a cellular network communication interface, or a combination thereof, and the like.

In a specific implementation, as an embodiment, the processor 2001 may include one or more CPUs, such as CPU0 and CPU1 as shown in FIG. 20 .

In a specific implementation, as an embodiment, the amblyopia training device may include multiple processors, such as the processor 2001 and the processor 2005 as shown in FIG. 20 . Each of these processors can be a single-core processor or a multi-core processor. A processor herein may refer to one or more devices, circuits, and/or processing cores for processing data (eg, computer program instructions).

In a specific implementation, as an embodiment, the amblyopia training device may further include an output device 2006 and an input device 2007 . The output device 2006 is in communication with the processor 2001 and can display information in a variety of ways. For example, the output device 2006 may be a liquid crystal display (LCD), a light emitting diode (LED) display device, a cathode ray tube (CRT) display device, a projector, or the like . The input device 2007 is in communication with the processor 2001 and can receive user input in a variety of ways. For example, the input device 2007 may be a mouse, a keyboard, a touch screen device, a sensor device, or the like.

In some embodiments, the memory 2003 is used to store the program code 2010 for executing the solutions of the present application, and the processor 2001 can execute the program code 2010 stored in the memory 2003 . The program code 2010 may include one or more software modules, and the amblyopia training device may implement the methods provided by the above embodiments through the processor 2001 and the program code 2010 in the memory 2003 .

Please refer to FIG. 21 , which is a schematic structural diagram of a terminal device provided by an embodiment of the present application. The terminal device includes a sensor unit 1110 , a computing unit 1120 , a storage unit 1140 and an interaction unit 1130 .

The sensor unit 1110 usually includes a vision sensor (such as a camera), a depth sensor, an IMU, a laser sensor, etc.;

The computing unit 1120, usually including CPU, GPU, cache, registers, etc., is mainly used to run the operating system;

The storage unit 1140 mainly includes memory and external storage, and is mainly used for reading and writing local and temporary data of users;

The interaction unit 1130 mainly includes a display screen, a touch panel, a speaker, a microphone, etc., and is mainly used for interacting with the user, obtaining input for input, and implementing the presentation algorithm effect and the like. For example, an amblyopia training image can be displayed and projected onto the amblyopia training device.

For ease of understanding, the following will illustrate the structure of a terminal device 100 provided by an embodiment of the present application. Referring to FIG. 22, FIG. 22 is a schematic structural diagram of a terminal device provided by an embodiment of the present application.

As shown in FIG. 22 , the terminal device 100 may include a processor 110 , an external memory interface 120 , an internal memory 121 , a universal serial bus (USB) interface 130 , a charging management module 140 , a power management module 141 , and a battery 142 , Antenna 1, Antenna 2, Mobile Communication Module 150, Wireless Communication Module 160, Audio Module 170, Speaker 170A, Receiver 170B, Microphone 170C, Headphone Interface 170D, Sensor Module 180, Key 190, Motor 191, Indicator 192, Camera 193 , a display screen 194, and a subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, etc.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the terminal device 100 . In other embodiments of the present application, the terminal device 100 may include more or less components than those shown in the drawings, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. The processor 110 may execute a computer program to implement any amblyopia training method in the embodiments of the present application.

The controller may be the nerve center and command center of the terminal device 100 . The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be directly called from the memory, which avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

In some embodiments, the processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I1C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I1S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transceiver (universal asynchronous transmitter) receiver/transmitter, UART) interface, mobile industry processor interface (MIPI), general-purpose input/output (GPIO) interface, subscriber identity module (SIM) interface, and / or universal serial bus (universal serial bus, USB) interface, etc.

It can be understood that the interface connection relationship between the modules illustrated in the embodiments of the present application is only a schematic illustration, and does not constitute a structural limitation of the terminal device 100 . In other embodiments of the present application, the terminal device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from the wired charger through the USB interface 130 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .

The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

In some possible implementations, the terminal device 100 may communicate with other devices using a wireless communication function. For example, the terminal device 100 may communicate with the second electronic device, the terminal device 100 establishes a screen projection connection with the second electronic device, and the terminal device 100 outputs screen projection data to the second electronic device, and so on. The screen projection data output by the terminal device 100 may be audio and video data.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in terminal device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, the antenna 1 can be multiplexed as a diversity antenna of the wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide wireless communication solutions including 1G/3G/4G/5G etc. applied on the terminal device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then convert it into electromagnetic waves and radiate it out through the antenna 2 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the terminal device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 1 , modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2 .

In some embodiments, the antenna 1 of the terminal device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the terminal device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), broadband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC , FM, and/or IR technology, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).

The terminal device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the terminal device 100 may include one or N display screens 194 , where N is a positive integer greater than one.

In some feasible implementations, the display screen 194 may be used to display various interfaces of the system output of the terminal device 100 .

The terminal device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .

Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the terminal device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals.

Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record videos in various encoding formats, for example, moving picture experts group (moving picture experts group, MPEG) 1, MPEG1, MPEG3, MPEG4, and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the terminal device 100 can be implemented through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the terminal device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example to save files like music, video etc in external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the terminal device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area may store an operating system, an application program required for at least one function (such as the indoor positioning method in the embodiment of the present application, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the terminal device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The terminal device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc. In some feasible implementations, the audio module 170 can be used to play the sound corresponding to the video. For example, when the display screen 194 displays a video playing screen, the audio module 170 outputs the sound of the video playing.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal.

Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals.

The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals.

The earphone jack 170D is used to connect wired earphones. The earphone interface 170D may be the USB interface 130, or may be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 180A may be provided on the display screen 194 . The gyro sensor 180B may be used to determine the motion attitude of the terminal device 100 . The air pressure sensor 180C is used to measure air pressure.

The acceleration sensor 180E can detect the magnitude of the acceleration of the terminal device 100 in various directions (including three axes or six axes). The magnitude and direction of gravity can be detected when the terminal device 100 is stationary. It can also be used to identify the posture of terminal devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

Distance sensor 180F for measuring distance.

The ambient light sensor 180L is used to sense ambient light brightness.

The fingerprint sensor 180H is used to collect fingerprints.

The temperature sensor 180J is used to detect the temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 . In other embodiments, the touch sensor 180K may also be disposed on the surface of the terminal device 100 , which is different from the position where the display screen 194 is located.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The terminal device 100 may receive key input and generate key signal input related to user settings and function control of the terminal device 100 .

Motor 191 can generate vibrating cues.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network or other programmable device. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be downloaded from a website site, computer, server, or data center Transmission to another website site, computer, server, or data center by wire (eg, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg, infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server, a data center, etc. that includes one or more available media integrated. The available media may be magnetic media (eg: floppy disk, hard disk, magnetic tape), optical media (eg: digital versatile disc (DVD)) or semiconductor media (eg: solid state disk (SSD)) Wait. It should be noted that the computer-readable storage medium mentioned in the embodiments of the present application may be a non-volatile storage medium, in other words, may be a non-transitory storage medium.

It should be understood that references herein to "a plurality" means two or more. In the description of the embodiments of the present application, unless otherwise specified, "/" means or means, for example, A/B can mean A or B; "and/or" in this document is only an association that describes an associated object Relation, it means that there can be three kinds of relations, for example, A and/or B can mean that A exists alone, A and B exist at the same time, and B exists alone. In addition, in order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, words such as "first" and "second" are used to distinguish the same or similar items with basically the same function and effect. Those skilled in the art can understand that the words "first", "second" and the like do not limit the quantity and execution order, and the words "first", "second" and the like are not necessarily different.

The above-mentioned examples provided for this application are not intended to limit this application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application. Inside.

Claims

An amblyopia training method, characterized in that, applied to amblyopia training equipment, the method comprising:

Determine the user's dominant eye and amblyopic eye;

determining the type of amblyopia of the amblyopic eye, the type of amblyopia including strabismus amblyopia or anisometropic amblyopia;

When the amblyopia type of the amblyopic eye is determined to be strabismus amblyopia, perform amblyopia training on the amblyopia training image by performing homography transformation processing;

When the type of amblyopia of the amblyopic eye is determined to be anisometropic amblyopia, image size adjustment processing is performed on the amblyopia training image to perform amblyopia training.
The method of claim 1, wherein the determining the type of amblyopia of the amblyopic eye comprises:

displaying a first test image in a display area corresponding to the dominant eye, and displaying a second test image in a display area corresponding to the amblyopic eye;

Determine the coordinates of the gaze position of the dominant eye in the displayed first test image and the coordinates of the gaze position of the amblyopic eye in the displayed second test image by means of eye tracking, obtain the coordinates of the first gaze position and the coordinates of the second gaze position;

The type of amblyopia of the amblyopic eye is determined according to the coordinates of the first gaze position and the coordinates of the second gaze position.
The method of claim 2, wherein the determining the amblyopia type of the amblyopic eye according to the first gaze position coordinates and the second gaze position coordinates comprises:

According to the coordinates of the first gaze position and the coordinates of the second gaze position, binocular deviation information is determined, where the binocular deviation information refers to deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye ;

If the binocular deviation information is greater than or equal to the first threshold, determining that the amblyopia type of the amblyopic eye is strabismus amblyopia;

If the binocular deviation information is less than the first threshold, it is determined that the amblyopia type of the amblyopic eye is anisometropic amblyopia.
The method according to any one of claims 1-3, wherein the determining the dominant eye and the amblyopic eye of the user comprises:

detecting the diopter of the user's eyes;

The eye with low refractive power in both eyes of the user is determined as the dominant eye, and the eye with high refractive power in the user's both eyes is determined as the amblyopic eye.
The method according to any one of claims 1-4, wherein the performing homography transformation processing on the amblyopia training image to perform amblyopia training, comprising:

The amblyopic training image is displayed in the display area corresponding to the dominant eye, and the amblyopic training image is displayed in the display screen corresponding to the amblyopic eye after performing homography transformation processing, so that the user's eyes The images seen can be combined for amblyopia training.
The method according to any one of claims 1-4, characterized in that, performing image size adjustment processing on the amblyopia training image to perform amblyopia training, comprising:

Displaying the amblyopia training image in the display area corresponding to the dominant eye, performing image size adjustment processing on the amblyopia training image and displaying it in the display area corresponding to the amblyopic eye, so that both eyes of the user can see the amblyopia training image. The obtained images can be combined for amblyopia training.
The method according to claim 5 or 6, wherein before displaying the amblyopia training image in the display area corresponding to the dominant eye, the method further comprises:

The image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye are determined, so that the user's binocular perception ability is the same.
The method of claim 7, wherein the determining the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye comprises:

displaying a third test image in the display area corresponding to the dominant eye, and displaying a fourth test image in the display area corresponding to the amblyopic eye;

reducing the contrast of the displayed third test image and increasing the displayed contrast of the fourth test image;

When the contrast determination instruction is detected, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the dominant eye, and the contrast of the fourth test image after the increase is determined as the contrast corresponding to the amblyopic eye Image contrast, the contrast determination instruction is triggered when the user feeds back, according to the displayed third test image and the fourth test image, that the contrast that his own eyes can perceive is the same.
The method of claim 7, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;

The displaying the amblyopic training image in the display area corresponding to the dominant eye, and performing homography transformation on the amblyopia training image and displaying it in the display area corresponding to the amblyopic eye, including:

Perform homography transformation on the amblyopia training image according to binocular deviation information, where the binocular deviation information refers to deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye;

The amblyopia training image before the homography transformation is used as the first training image, and the amblyopia training image after the homography transformation is used as the second training image. The first training image is displayed in the display area of the amblyopic eye, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye;

By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined, and the first gaze position and the second gaze position are obtained. gaze position;

If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, then subtract the Small the amount of homography transformation of the amblyopia training image, returning the step of using the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia The training ends, or when the amblyopia training image does not undergo homography transformation, the second gaze position coincides with the actual position of the target object in the second training image.
The method of claim 7, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;

The displaying of the amblyopia training image in the display area corresponding to the dominant eye, and performing image size adjustment processing on the amblyopia training image and displaying it in the display area corresponding to the amblyopia eye, including:

Adjusting the position and/or size of the target object in the amblyopia training image multiple times to obtain multiple training images;

determining the image scaling ratio of the amblyopic eye relative to the dominant eye;

scaling the sizes of the plurality of training images according to the image scaling ratio;

Taking the plurality of training images before scaling as a plurality of third training images, and taking the plurality of training images after scaling as a plurality of fourth training images, according to the image contrast corresponding to the dominant eye, in the The plurality of third training images are sequentially displayed in the display area corresponding to the dominant eye, and the plurality of fourth training images are sequentially displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, And the display sequence and switching frequency of the plurality of third training images and the plurality of fourth training images are the same.
The method of claim 10, wherein the determining the image scaling ratio of the amblyopic eye relative to the dominant eye comprises:

A fifth test image is displayed in the display area corresponding to the dominant eye, and a sixth test image is displayed in the display area corresponding to the amblyopic eye, the fifth test image includes the first test target, and the sixth test image the image includes a second test target, the first test target and the second test target are in the same proportion;

scaling the ratio of the sixth test image displayed;

When a scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user according to the displayed data. It is triggered when the fifth test image and the sixth test image feedback that their own eyes can see the first test target and the second test target with the same proportion.
The method according to any one of claims 9-11, wherein the method further comprises:

Determine the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye by means of eye tracking, and obtain the gaze position of the amblyopic eye;

If the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, the display mode of the target object in the image displayed in the display area corresponding to the amblyopic eye is changed. Modified to a reference display mode to indicate the amblyopia training effect, and/or, according to the gaze position of the amblyopic eye and the actual position of the target object, draw an amblyopia training curve to indicate the amblyopia training effect.
An amblyopia training device, characterized in that it is applied to amblyopia training equipment, the device comprising:

a first determining module for determining the dominant eye and amblyopic eye of the user;

a second determining module, configured to determine the type of amblyopia of the amblyopic eye, where the type of amblyopia includes strabismus amblyopia or anisometropic amblyopia;

a first amblyopia training module, used for performing amblyopia training on the amblyopia training image by performing homography transformation processing on the amblyopia training image when the amblyopia type of the amblyopic eye is determined to be strabismus amblyopia;

The second amblyopia training module is configured to perform amblyopia training by performing image size adjustment processing on the amblyopia training image when the amblyopia type of the amblyopic eye is determined to be anisometropic amblyopia.
The apparatus of claim 13, wherein the second determining module comprises:

a display submodule for displaying a first test image in a display area corresponding to the dominant eye, and displaying a second test image in a display area corresponding to the amblyopic eye;

a first determination sub-module for determining, by means of eye tracking, the coordinates of the gaze position of the dominant eye in the displayed first test image, and the second test displayed by the amblyopic eye The coordinates of the gaze position in the image, to obtain the first gaze position coordinates and the second gaze position coordinates;

The second determination submodule is configured to determine the type of amblyopia of the amblyopic eye according to the coordinates of the first gaze position and the coordinates of the second gaze position.
The apparatus of claim 14, wherein the second determination submodule is specifically configured to:

According to the coordinates of the first gaze position and the coordinates of the second gaze position, binocular deviation information is determined, where the binocular deviation information refers to deviation information between the gaze direction of the amblyopic eye and the gaze direction of the dominant eye ;

If the binocular deviation information is greater than or equal to the first threshold, determining that the amblyopia type of the amblyopic eye is strabismus amblyopia;

If the binocular deviation information is less than the first threshold, it is determined that the amblyopia type of the amblyopic eye is anisometropic amblyopia.
The apparatus according to any one of claims 13-15, wherein the first determining module is specifically configured to:

detecting the diopter of the user's eyes;

The eye with low refractive power in both eyes of the user is determined as the dominant eye, and the eye with high refractive power in the user's both eyes is determined as the amblyopic eye.
The device according to any one of claims 13-16, wherein the first amblyopia training module comprises:

The first amblyopia training sub-module is used for displaying the amblyopia training image in the display area corresponding to the dominant eye, and performing homography transformation on the amblyopia training image and displaying it in the display area corresponding to the amblyopia eye , so that the images seen by the user's eyes can be combined to perform amblyopia training.
The device according to any one of claims 13-16, wherein the second amblyopia training module comprises:

The second amblyopia training sub-module is configured to display the amblyopia training image in the display area corresponding to the dominant eye, and display the amblyopia training image in the display area corresponding to the amblyopia after performing image size adjustment processing on the amblyopia training image , so that the images seen by the user's eyes can be combined to perform amblyopia training.
The apparatus of claim 17 or 18, wherein the apparatus further comprises:

The third determining module is configured to determine the image contrast corresponding to the dominant eye and the image contrast corresponding to the amblyopic eye, so that the user's binocular perception ability is the same.
The apparatus of claim 19, wherein the third determining module is specifically configured to:

displaying a third test image in the display area corresponding to the dominant eye, and displaying a fourth test image in the display area corresponding to the amblyopic eye;

reducing the contrast of the displayed third test image and increasing the displayed contrast of the fourth test image;

When the contrast determination instruction is detected, the contrast of the third test image after the reduction is determined as the image contrast corresponding to the dominant eye, and the contrast of the fourth test image after the increase is determined as the contrast corresponding to the amblyopic eye Image contrast, the contrast determination instruction is triggered when the user feeds back, according to the displayed third test image and the fourth test image, that the contrast that his own eyes can perceive is the same.
The apparatus of claim 19, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;

The first amblyopia training submodule is specifically used for:

Perform homography transformation on the amblyopia training image according to binocular deviation information, where the binocular deviation information refers to deviation information between the line of sight direction of the amblyopic eye and the line of sight direction of the dominant eye;

The amblyopia training image before the homography transformation is used as the first training image, and the amblyopia training image after the homography transformation is used as the second training image. The first training image is displayed in the display area of the amblyopic eye, and the second training image is displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye;

By means of eye tracking, the gaze position of the dominant eye in the first training image and the gaze position of the amblyopic eye in the second training image are determined, and the first gaze position and the second gaze position are obtained. gaze position;

If the first gaze position coincides with the actual position of the target object in the first training image, and the second gaze position coincides with the actual position of the target object in the second training image, then subtract the Small the amount of homography transformation of the amblyopia training image, returning the step of using the amblyopia training image before the homography transformation as the first training image, and using the amblyopia training image after the homography transformation as the second training image, until the amblyopia The training ends, or when the amblyopia training image does not undergo homography transformation, the second gaze position coincides with the actual position of the target object in the second training image.
The apparatus of claim 19, wherein the amblyopia training image includes a target object, and the target object is used for amblyopia training;

The second amblyopia training submodule is specifically used for:

Adjusting the position and/or size of the target object in the amblyopia training image multiple times to obtain multiple training images;

determining the image scaling ratio of the amblyopic eye relative to the dominant eye;

scaling the sizes of the plurality of training images according to the image scaling ratio;

Taking the plurality of training images before scaling as a plurality of third training images, and taking the plurality of training images after scaling as a plurality of fourth training images, according to the image contrast corresponding to the dominant eye, in the The plurality of third training images are sequentially displayed in the display area corresponding to the dominant eye, and the plurality of fourth training images are sequentially displayed in the display area corresponding to the amblyopic eye according to the image contrast corresponding to the amblyopic eye, And the display sequence and switching frequency of the plurality of third training images and the plurality of fourth training images are the same.
The device of claim 22, wherein the second amblyopia training submodule is further used for:

A fifth test image is displayed in the display area corresponding to the dominant eye, and a sixth test image is displayed in the display area corresponding to the amblyopic eye, the fifth test image includes the first test target, and the sixth test image the image includes a second test target, the first test target and the second test target are in the same proportion;

scaling the ratio of the sixth test image displayed;

When a scale determination instruction is detected, the ratio between the first test target and the scaled second test target is determined to obtain the image scaling ratio, and the ratio determination instruction is the user according to the displayed data. It is triggered when the fifth test image and the sixth test image feedback that their own eyes can see the first test target and the second test target with the same proportion.
The device according to any one of claims 21-23, wherein the device further comprises:

a fourth determining module, configured to determine the gaze position of the amblyopic eye in the image displayed in the display area corresponding to the amblyopic eye by means of eye tracking, so as to obtain the gaze position of the amblyopic eye;

The training effect indication module is used for, if the gaze position of the amblyopic eye coincides with the actual position of the target object in the image displayed in the display area corresponding to the amblyopic eye, then the image displayed in the display area corresponding to the amblyopic eye will be displayed in the image. The display mode of the target object is modified to a reference display mode to indicate amblyopia training effect, and/or an amblyopia training curve is drawn according to the gaze position of the amblyopic eye and the actual position of the target object to indicate amblyopia training effect.
An amblyopia training device, characterized in that the amblyopia training device comprises a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program to realize any one of claims 1-12 the steps of the method.
A computer-readable storage medium, characterized in that, a computer program is stored in the storage medium, and when the computer program is executed by a processor, the steps of any one of the methods of claims 1-12 are implemented.
A computer program product, characterized in that, when the computer program product runs on a computer, the computer is made to execute the steps of the method according to any one of claims 1-12.