DE102021104127A1 - Method for operating a mobile computing device, computer program and mobile computing device - Google Patents

Abstract

The invention relates to a method for operating a mobile computing device, which has at least one display device, one input device and one camera, with the mobile computing device or an additional device connected to it being used to capture the face of a user of the mobile computing device and the distance between at least one reference point in the The user's face and the mobile computing device is determined, with a test character to be visually detected by the user being displayed on the display device in a size that is determined as a function of the determined distance.

Description

The invention relates to a method for operating a mobile computing device which has at least one display device, one input device and one camera. The invention also relates to a computer program for carrying out such a method and such a mobile computing device.

There are already computer programs, e.g. in the form of apps, with which the user can check his own eyesight, e.g. B. with the Android app "Eye test". However, current computer programs are not suitable for replacing an eye test, such as that carried out in an ophthalmologist's practice or at an optician's.

The invention is based on the object of specifying an improved method and a corresponding computer program for this, as well as a corresponding mobile computer device.

This object is achieved by a method for operating a mobile computing device which has at least one display device, one input device and one camera, the face of a user of the mobile computing device being recorded by means of the mobile computing device or an additional device connected thereto and the distance between at least one Reference point is determined in the face of the user and the mobile computing device, wherein on the display device at least one test symbol to be detected visually by the user is displayed in a size that is determined as a function of the determined distance. The invention allows standardized ophthalmological examinations, in particular standardized examinations and documentation of ophthalmological examinations, using mobile computer devices.

The invention allows a standardized implementation of an eye test, which is comparable to an eye test at an ophthalmologist or optician, by the user himself using the mobile computer device. The mobile computer device can also create a documentation of the results of the eye test.

The implementation of ophthalmological examinations can be carried out with mobile devices at any location, especially at home (home care environment), documented and displayed in the course.

The previous examinations are carried out under standardized conditions in an ophthalmological practice or at an optician's with defined distances between the eye of the person to be examined and the ophthalmological parameters. An example is the visual acuity test, where the optotypes have to be presented at a defined distance in order to determine the visual angle and thus the eyesight. The distance between the eye and the test symbol is usually 5 meters during testing, for example, and the visual acuity is defined as the inverse resolution in arc minutes.

The patient must always go to the ophthalmologist or optician for the documentation of findings, enter a defined test environment and determine these ophthalmological parameters with specific individual devices. This requires the provision of premises, the arrangement of appointments, and older people in particular need an accompanying person. Specific devices for determining the ophthalmological parameters must be available

The invention makes it possible to carry out standard ophthalmological examinations, such as determining eyesight, on mobile computer devices, such as smartphones, with the help of the distance measurement and face recognition options available in the devices, to document these findings accordingly and in the to present the documentation to the ophthalmologist.

In general, many mobile devices based on smartphone technologies offer the possibility of displaying ophthalmological test objects and thus carrying out corresponding ophthalmological examinations. Ophthalmological examinations can be: Checking the visual acuity, measuring and determining metamorphopsia, determining the visual field, determining the scotoma, determining the squint position and documenting the findings. For this purpose, corresponding test objects, such as optotypes, Amsler network, fixation on the mobile devices must be displayed.

However, in order to be able to compare the results with the standardized examinations that are carried out by an ophthalmologist or optician or that are defined on the basis of standards, it is advantageous to record the precise distance between the test objects. In addition, it is advantageous to carry out the examinations for the eyes individually or for both eyes simultaneously.

With the help of the distance measurements and face recognition available in mobile devices or by means of additional mobile devices that are connected to the mobile device, it is possible to record the distance to each eye individually or to a reference point.

By measuring the distance between the display device (display of the mobile device or smartphone) and the eye or reference points on the face, the size of the test variables can be continuously changed by software in such a way that the visual angle between the eye and the test object remains constant to the extent that the corresponding Visual angle corresponds to the corresponding DIN norms or the ophthalmological standards, even if the distance between the reference point on the face and the display on the mobile computing device changes.

This invention should make it possible to carry out ophthalmological examinations not only at an ophthalmologist, but also at any desired location.

This technique is intended to be used to determine and control ophthalmological parameters in the home setting or outside of standardized settings such as those found at the ophthalmologist's, optometrist's or appraiser's. Examples of ophthalmological parameters are visual acuity. Squint angle, visual field, metamorphopsia.

This technique should be used both for self-monitoring and for therapy monitoring of ophthalmological diseases in which therapy decisions depend on the recording of standardized ophthalmological examinations. This includes, for example, therapy control using intravitreal injections for retinal vascular diseases such as age-related macular degeneration, vascular occlusions or diabetic maculopathy.

The mobile computing device can e.g. B. be a mobile computing device, such as a smartphone, a tablet, a laptop or a notepad. Where a computer is mentioned, it may be the main processor or any processor of the mobile computing device. The display device can, for. B. be a display of the mobile computing device, in particular a touch screen. As an input device, the mobile computing device z. B. have one or more of the following components: touch screen, conventional buttons or switches, microphone. The camera of the mobile computing device itself can also be used as an input device. The camera of the mobile computing device can be any camera, e.g. B. the front camera or rear camera of a smartphone, or in a laptop or notebook the built-in webcam.

According to an advantageous embodiment of the invention, it is provided that the size of the test symbol displayed on the display device is automatically and continuously adapted as a function of the distance determined in such a way that it can be recognized by the user with the same size. This has the advantage that the function of the mobile computer device can ensure that an eye test is carried out in a standardized manner. The size of the check symbol automatically adapts to a defined, specified size. For example, the mobile computing device can have a control function by which the viewing angle with which the user perceives the check symbol on the display device with his eye is automatically controlled to a constant value depending on the determined distance. The viewing angle then remains constant regardless of movements that the user makes with the mobile computing device.

The method can also have a warning function. The warning function checks whether a maximum value is reached or exceeded when the size of the test symbol displayed on the display device changes and whether a minimum value is reached or fallen below. For example, the maximum value may be defined by the physical size of the display device. The minimum value may be defined by the resolution of the display device, e.g. B. such that if the minimum size is not reached at the given resolution, a meaningful representation of the test symbol is no longer possible. If the warning function detects that the size of the displayed check symbol is at or above the maximum value or at or below the minimum value, a warning signal can be issued. The warning signal can B. be an optical and / or acoustic and / or haptic signal.

According to an advantageous embodiment of the invention, it is provided that an input from the user via the input device is awaited, and when the user has entered an input, at least one other test symbol and/or one test symbol in a different size is displayed on the display device. In this way, the typical course of an eye test can be carried out through interactions with the user. For example, the vision test may begin by displaying relatively large test symbols, and with each test symbol correctly recognized, the test symbol may be displayed at a reduced size.

The method may include querying for user input. As long as the user does not make any input, the current test symbol is still displayed. If an entry is made, e.g. B. a manual input, an acoustic input, z. B. a voice input, or an optical input, z. B. a certain gesture, this input is analyzed in the mobile computing device and checked for validity. If it is determined that the mobile computing device is informed by the input that the user has recognized the test symbol correctly, an eye test can e.g. B. the next stage can be carried out. After the final stage of the eye test, documentation of the user's eyesight can then be created.

According to an advantageous embodiment of the invention, the distance between the at least one reference point on the user's face and the mobile computing device is determined using at least one optical sensor device of the mobile computing device. The reference point can e.g. B. be the center of an eye of the user. For example, the user's face can be automatically analyzed using the camera of the mobile computing device, e.g. B. with known methods of face recognition, and thereby the at least one reference point in the face of the user can be determined.

According to an advantageous embodiment of the invention, it is provided that the spatial position or change in position of the mobile computing device is detected gyroscopically in at least one spatial direction, with a check symbol to be visually detected by the user being displayed in a size and/or in a position and/or with a change in shape is represented, which is determined as a function of the detected spatial position or change in position. In this way, a distortion in the perception of the test symbol resulting from a change in position of the mobile computing device can be automatically compensated for. In addition, changes in position, which result, for example, from the user not holding the mobile computing device unsteadily, can be compensated for automatically. For example, gyroscopic sensors present in the mobile computing device, such as acceleration sensors, can be used for this function.

In an independent aspect of the invention, which can also be advantageously combined with the above-mentioned embodiments of the invention, it is provided that the fixation behavior of at least one eye of the user is recorded by means of the mobile computer device or an additional device connected to it, with at least one of the User to visually detect test symbol is displayed in a size and / or in a position that is determined depending on the fixation behavior. This has the advantage that the fixation behavior of the user's eye can be automatically taken into account when carrying out the ophthalmological examination. In this way, the examination results are not negatively influenced by unexpected peculiarities in the individual fixation behavior.

The determination of the distance between at least one reference point on the user's face and the mobile computing device can, for. B. be carried out on a smartphone by means of a distance measurement function that is already present in the smartphone. In the method, only the distance already determined in the operating system by the integrated function has to be evaluated and used as the distance variable for determining the size of the test symbol. The mobile computing device can be used for distance measurement z. B. be equipped with an infrared diode, which is operated with a specific modulation. Reflections of the light emitted by the infrared diode can then be received again on the mobile computing device, e.g. B. by means of the camera, and by evaluating the transit time, the distance can be determined. Another way to determine the distance is to intentionally create a certain amount of blurriness in the image recorded by the camera, e.g. B. by specifically varying the focus of the camera. A distance measurement can also be carried out using the unsharpness information.

The fixation behavior can B. are automatically detected by the mobile computing device by specifically a light signal such. B. a flash of light, is radiated onto the user. The light signal creates a reflex on the cornea of the user's eye, which can be automatically captured by the camera of the mobile computing device. For example, this corneal reflex can be localized using face recognition methods and automatically evaluated accordingly. For this purpose, the mobile computing device can be equipped with a light signal emitting device.

According to an advantageous embodiment of the invention, it is provided that the size and/or position of the test symbol shown on the display device is automatically and continuously adapted as a function of the fixation behavior. This can z. B. the test symbol can be reproduced on the display device in such a way that it can be recognized by the user in the same size and/or in the same position, so that the fixation behavior of the user is automatically compensated.

The object mentioned at the outset is also achieved by a computer program with program code means, set up to carry out a method of the type explained above if the computer program is executed on a computer of a mobile computing device. The advantages explained above can also be realized in this way. The computer program can B. be designed as an app.

The object mentioned at the outset is also achieved by a mobile computing device. The advantages explained above can also be realized in this way.

According to an advantageous embodiment of the invention, it is provided that the mobile computer device has a distance measuring device or at least one distance measuring function, by means of which the distance between at least one reference point on the user's face and the mobile computer device can be determined. The distance measurement function can e.g. B. be an existing function of the operating system of the mobile computing device.

A special feature of the invention is the combination of distance measurement, face recognition and corresponding adjustment of the size of the test objects that are presented on the mobile computer devices.

The distance between the mobile computer device and the corresponding reference points on the face of the person to be examined can thus be measured by means of the distance measurements installed in the mobile computer devices or smartphones. Distance measurements can be carried out by analyzing images that are recorded with the cameras used in the mobile computing devices, by specific optical sensors or by lidar (light detection and ranging) techniques. Cameras and/or LIDAR sensors are particularly suitable for this. With the help of this data, a 3D facial matrix is determined using face recognition techniques and augmented reality methods. The distance to various reference points within this matrix, for example to the eye to be examined, can thus be determined. The reference points on the face for the distance measurement to the output unit (e.g. display of a smartphone) are determined using face recognition techniques and augmented reality methods.

In addition, the fixation behavior or the fixation point of the person to be examined can be detected and recorded. The measured distance can then be used to directly convert the size of the test object to be determined to the ophthalmological or international nomens and to adjust it accordingly on the display of the mobile computing device. The fixation behavior can thus be recognized on the basis of face recognition and recognition of the eye position. This is particularly suitable for examining strabismus. In addition, the presented test object can be shifted in its position and, if necessary, its size in relation to the patient's fixation point. This technology is necessary, for example, for valid visual field measurements, microperimetry, Amsler grid testing and metamorphopsia detection.

If necessary, these methods can be increased in their accuracy with the help of gyroscopic sensors, which indicate the inclination of the device display.

In addition, the sensors can be used to determine if the patient is squinting or covering one eye, for example with a hand, to ensure that only one eye is being tested at a time. It can also be ensured that both eyes are open and not covered when binocular functions are to be measured.

The algorithm, which uses distance measurement to adjust the test variables, can be stored as a program (app) in software in the mobile computer device or in the form of a connection to external computing power.

The test symbol can be any graphic or other symbol, e.g. a character or other test object. The test symbol can also be a combination of several individual symbols or characters. For example, the verification symbol may be selected depending on at least one characteristic of the viewer, such as age. For example, pictures of animals can be used as test symbols for children.

The invention is based on an embodiment using the 1 and 2 explained.

the 1 shows a mobile computing device 1 performing an eye test that a user 2 performs himself.

The mobile computing device 1 has a display device 10, an input device 11 and a camera 12. The mobile computing device 1 uses the camera 12 to capture the face of the user 2 and carries out face recognition. With face recognition, e.g. B. a continuous creation of a face matrix can be carried out, are recognized in the reference points in the face. Face recognition detects certain prominent points on the face, e.g. B. the eyes. The mobile computing device detects a reference point in the face recognition result 3 on the user's face, e.g. B. the position of the pupil of an eye.

An eye test is then carried out by the user 2 himself using the mobile computing device 1 . The mobile computing device 1 determines z. B. via the camera 12 or another distance sensor, the distance d between the reference point 3 and the mobile computing device 1. In the course of the vision test, a test symbol 13 with a size a is first displayed on the display device 10. The distance measurement is carried out continuously, i. H. the distance d always reflects the current value. The mobile computing device 1 continuously automatically adjusts the size (a) of the test symbol 13 on the display device 10 as a function of the distance d, so that the viewing angle a for the user 2 remains constant.

The user 2 can now make an entry on the mobile computing device 1, e.g. by voice recognition, touching the touch screen or by capturing a gesture using the camera 12. The mobile computing device 1 evaluates the input and decides, depending on this, how the eye test is to be continued , or whether the eye test is finished. For example, the eye test can be continued after the entry and a test symbol 13 can be displayed in the next size, e.g. B. with a size a', which is smaller than the size a. After this, input from user 2 is again awaited. These steps are repeated until the eye test is completed. The results of the eye test are then documented and can be automatically made available to an ophthalmologist or optician.

In addition, the mobile computing device 1 can also automatically detect the fixation behavior of the user 2 and change the size of the check symbol 13 and/or the position of the check symbol 13 on the display device 10 depending on the fixation behavior, e.g. B. adapt such that the test symbol 13 always appears constant for the user 2 regardless of the fixation behavior.

the 2 FIG. 1 also shows a mobile computing device 1 carrying out an eye test that a user 2 carries out himself. The eye test runs as described above. In addition, gyroscopic signals 14 are taken into account in the process, which are determined, for example, by means of gyroscopic sensors of mobile computing device 1 . Depending on the gyroscopic signals 14, the test symbol size a and the position of the test symbol 13 on the display device 10 are adjusted to a gyroscopically determined position and/or location of the display device 10 in relation to the user 2.

Reference List

1

mobile computing device

2

user

3

reference point

10

display device

11

input device

12

camera

13

check icon

14

gyroscopic signals

a

size

i.e

Distance

Claims (10)

Method for operating a mobile computing device, which has at least one display device, one input device and one camera, characterized in that the face of a user of the mobile computing device and the distance between at least one reference point on the face are detected by means of the mobile computing device or an additional device connected to it of the user and the mobile computing device is determined, wherein at least one test symbol to be detected visually by the user is displayed on the display device in a size that is determined as a function of the determined distance. procedure after claim 1 , characterized in that the size of the test symbol shown on the display device is automatically and continuously adjusted as a function of the distance determined in each case in such a way that it can be recognized by the user with the same size. Method according to one of the preceding claims, characterized in that an input by the user via the input device is awaited, and when the user has entered an input, at least one other test symbol and/or one test symbol in a different size is displayed on the display device. Method according to one of the preceding claims, characterized in that the determination of the distance between the at least one reference point on the user's face and the mobile computing device is carried out by means of at least one optical sensor device of the mobile computing device. Method according to one of the preceding claims, characterized in that the spatial position or change in position of the mobile computing device is detected gyroscopically in at least one spatial direction, with a test symbol to be visually detected by the user in a size and/or in a position and/or is represented with a change in shape, which is determined as a function of the detected spatial position or change in position. Method for operating a mobile computer device which has at least one display device, one input device and one camera, in particular method according to one of the preceding claims, characterized in that the fixation behavior of at least one eye of the user is recorded by means of the mobile computer device or an additional device connected to it, wherein at least one test symbol to be visually detected by the user is displayed on the display device in a size and/or in a position that is determined as a function of the fixation behavior. procedure after claim 6 , characterized in that the size and/or position of the test symbol shown on the display device is automatically and continuously adjusted as a function of the fixation behavior. Computer program with program code means, set up to carry out a method according to one of the preceding claims, when the computer program is executed on a computer of a mobile computing device. Mobile computing device having at least one display device, one input device and one camera, and having at least one memory in which a computer program claim 8 is stored, wherein the computing device for performing a method according to any one of Claims 1 until 6 is set up when the computer program is executed on a computer of the mobile computing device. Mobile computing device claim 9 , characterized in that mobile computing device has a distance measuring device or at least one distance measuring function, by which the distance between at least one reference point on the user's face and the mobile computing device can be determined.

Images