KR102686346B1 - System of measureing user sight and changing display panel setting - Google Patents

Abstract

The present invention includes a smart terminal including a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality; And by operating the vision measurement game on the smart terminal according to the user's operation, obtaining information about the user's refractive power abnormality in the process of playing the vision measurement game, and displaying the display corresponding to the degree of the user's refractive power abnormality. An application that changes the settings of; Includes, wherein the application plays the visual acuity measurement game using a video, still image, animation, or screen corresponding to a plurality of refractive powers to the user, and Characterized in that, by utilizing the results of the user's refractive power abnormality, information on the user's refractive power abnormality is obtained and the settings of the display panel are changed to the settings of the display panel corresponding to the refractive power at which the result of the game was best. Initiate measurement and response system.

Description

User vision measurement and response system {SYSTEM OF MEASUREING USER SIGHT AND CHANGING DISPLAY PANEL SETTING}

The present invention relates to a system for measuring a user's visual acuity. More specifically, the present invention relates to a system for measuring a user's visual acuity. More specifically, the present invention relates to a system for measuring a user's visual acuity, which shows the user a video, still image, animation, or screen corresponding to a plurality of refractive powers through the visual acuity measurement game. Using the results of the game, information about the user's refractive power abnormality is obtained, and the settings of the display panel used by the user are changed to the settings of the display panel corresponding to the refractive power for which the game result was the best, so that the user can enjoy the game while also enjoying the game. The present invention relates to a user visual acuity measurement and response system that allows the screen settings of a display panel to be set in accordance with the user's refractive power abnormality.

Due to the coronavirus outbreak, many people are living at home rather than outside, and various content industries such as Netflix and YouTube are developing as the number of people living at home increases, but on the contrary, as people stay at home more, many people's eyesight is weakening. Deterioration is increasing.

According to recent research data from Korea's only pediatric and adolescent hospital, the visual acuity data of 1,472 1st and 2nd graders and 1,367 4th and 5th graders of elementary school who were examined for 3 months from May to July 2021 were analyzed, and the results showed that the lower grades of elementary school were myopic. The prevalence rate is 38%, and the prevalence of myopia in upper grades is close to 60%. It is believed that myopia has increased due to the increase in video classes and smartphone use due to social distancing. This is not just a problem for elementary school students, but the problem of vision loss is increasing among middle and high school students and even adults.

Meanwhile, as life expectancy increases, the problem of presbyopia is also increasing. Presbyopia is a phenomenon in which the elasticity of the lens in the eye is reduced, making it difficult to focus on close objects. It can be corrected with corrective lenses. In order to solve these problems of vision loss or presbyopia, an easy vision measurement tool is needed to conveniently and accurately measure the refractive power of patients suspected of having decreased vision or presbyopia without visiting a hospital.

In relation to this, according to Republic of Korea Patent No. 10-1667399 (registered on October 12, 2016) (Patent Document 1), “The present invention is a new technology that can prevent errors in visual acuity measurement due to differences in surrounding brightness. The digital vision measuring device according to the present invention relates to a monitor (30) using an ambient brightness measuring means (35) and a screen brightness measuring means (36) provided in the monitor (30). ) By measuring the surrounding brightness and the screen brightness of the display panel 31 provided on the monitor 30, and accurately adjusting the screen brightness of the display panel 31 according to the surrounding brightness, more accurate vision measurement results can be obtained. “There is an advantage in this technology.”

In addition, the present applicant's Korean Patent Publication No. 10-2017-0085309 (published on July 24, 2017) (Patent Document 2) states, “The present invention relates to a system and method for measuring visual acuity using a mobile device such as a smartphone. The mobile vision measurement system provided as an embodiment of the present invention includes an infrared irradiator that irradiates infrared rays to the user's eye, a sensor unit that detects the movement of the user's eye, and a clip unit for fixing the infrared irradiator to the sensor unit. , a visual acuity calculation unit that calculates visual acuity using the time irradiated by the infrared irradiator, the time when the sensor unit detects the movement of the user's eyes, and the moving speed of the infrared irradiated by the infrared irradiator as variables, the infrared irradiator is capable of wired and wireless communication, , the time at which infrared rays are irradiated is transmitted to the visual acuity calculation unit through communication, and the visual acuity calculation unit may correspond to a mobile device.”

However, in Patent Document 1 and Patent Document 2, unlike the present application, the user is shown the visual acuity measurement game using a video, still image, animation, or screen corresponding to a plurality of refractive powers, and the results of this game are shown. Utilizing the , obtain information about the user's refractive power abnormality, change the settings of the display panel used by the user to the settings of the display panel corresponding to the refractive power that gave the best game results, and enjoy the game while enjoying the display panel. There has been no disclosure of a user visual acuity measurement and response system that allows screen settings to be set in accordance with the user's refractive power abnormality.

Republic of Korea Patent No. 10-1667399 (registered on October 12, 2016), title of invention: digital vision meter Republic of Korea Patent Publication No. 10-2017-0085309 (published on July 24, 2017), Title of invention: Mobile vision measurement system and vision measurement method using the same {THE SYSTEM AND SYSTEM FOR INSPECTING VISION}

The present invention was created to solve the above-mentioned problems, and the present invention shows the user through the visual acuity measurement game using a video, still image, animation, or screen corresponding to a plurality of refractive powers, and provides Using the results, information about the user's refractive power abnormality is obtained, and the settings of the display panel used by the user are changed to the settings of the display panel corresponding to the refractive power that gave the best game results, so that the display panel is used while playing the game. The purpose is to provide a user vision measurement and response system that allows the screen settings to be set in response to the user's refractive power abnormality.

A user vision measurement and response system according to a preferred embodiment of the present invention for achieving the above object includes a smart terminal including a display panel having a multi-layer MLA structure that corrects the user's refractive power abnormality; And by operating the vision measurement game on the smart terminal according to the user's operation, obtaining information about the user's refractive power abnormality in the process of playing the vision measurement game, and displaying the display corresponding to the degree of the user's refractive power abnormality. Includes an application that changes the panel settings.

Here, the application plays the visual acuity measurement game using a video, still image, animation, or screen corresponding to a plurality of refractive powers to the user, uses the results of the game for the user, and determines the user's vision. It is desirable to obtain information about the refractive power abnormality and change the settings of the display panel to those corresponding to the refractive power at which the game result was best.

Additionally, in the vision measurement game, based on the difference in results, performance, or performance between advancing past the exact center of the display panel and advancing past the edge of the display panel in a mission in which the character must advance straight forward. Thus, the vision measurement game can be designed so that the macular degeneration suspicion score of the user playing the vision measurement game increases.

Additionally, in the vision measurement game, there is a difference in results, performance, or performance between a mine or treasure located at the exact center of the display panel and a mine or treasure located at the edge of the display panel in a minesweeper or treasure hunt mission. Based on this, the vision measurement game can be designed so that the macular degeneration suspicion score of the user playing the vision measurement game increases.

In addition, the multi-layer MLA structure is a multi-layer MLA structure that corrects the screen of the display panel to the degree of the user's refractive power abnormality for users with eye refractive power problems, including presbyopia, myopia, and hyperopia. a first MLA lens having a lower thickness of the first gap (gap_1), being spaced apart from the display panel by the first gap (gap_1), and having a height (lens_sag_height_1) of the first lens; a first gap layer on the first MLA lens having a first thickness (gap_thinckness_1); A second MLA lens formed on the first gap layer, has a lower thickness of the second gap (gap_2), is spaced apart from the first gap layer by the second gap (gap_2), and has a height (lens_sag_height_2) of the second lens. ; a second gap layer on the second MLA lens having a second thickness (gap_thinckness_2); It is formed on the (n-1)th gap layer, has a lower thickness of the nth gap (gap_n), is spaced apart from the (n-1)th gap layer by the nth gap (gap_n), and has a height of the nth lens (lens_sag_height_n) ) having the nth MLA lens (n is a natural number of 3 or more); and an n-th gap layer on the n-th MLA lens having an n-th thickness (gap_thinckness_n) (n is a natural number of 3 or more).

In addition, when processing images to correct a user's refractive error on the display panel, there are steps of capturing the entire image of the underlying app; Processing the entire image of the underlying app to generate an image prime corrected according to the degree of refractive error of the user; And processing the image, including displaying the image prime as the entire image of the front app.

Meanwhile, according to another embodiment of the present invention, as a method of measuring and responding to the user's visual acuity using the above-described user visual acuity measurement and response system, the smart terminal plays the visual acuity measurement game according to the user's operation. Initiating a; Obtaining information about the user's refractive power abnormality in the process of playing the game for measuring visual acuity; and changing settings in the game for measuring visual acuity based on information about the user's refractive power abnormality, thereby verifying the judgment regarding the refractive power abnormality. Providing a method for measuring and responding to a user's visual acuity, including. do.

In addition, when changing the settings in the vision measurement game based on information about the user's refractive power abnormality, the settings in the vision measurement game are changed using a multi-layer MLA structure that corrects the user's refractive power abnormality. .

In addition, according to another embodiment of the present invention, a medium stored in a smart terminal including a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality and readable by the smart terminal is used to measure the user's visual acuity. In the process of playing a game, obtaining information about the user's refractive power abnormality; and a step of verifying the judgment about the refractive power abnormality by changing settings in the vision measurement game based on information about the user's refractive power abnormality. .

In addition, according to another embodiment of the present invention, an application that is stored in a storage medium in a smart terminal and operates in conjunction with the smart terminal including a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality is provided to the user. In the process of playing a game for measuring eyesight, obtaining information about the user's refractive power abnormality; and changing settings in the vision measurement game based on information about the user's refractive power abnormality, and verifying the judgment about the refractive power abnormality.

According to the user vision measurement and response system according to the present invention,

First, it provides an easy visual acuity measurement tool that conveniently and accurately measures the refractive power of patients suspected of having poor visual acuity or presbyopia without visiting a hospital. At the same time, it changes the display settings in response to the measured visual acuity, allowing the display panel to be clearly visible even with the naked eye. It is possible to do so.

Second, by using a vision measurement game, it is possible to measure the user's vision naturally while playing the game without the user feeling that their vision is being measured.

In other words, the application (app) according to the present invention provides the user with a video, still image, animation, or screen corresponding to a plurality of refractive powers, and uses the results of the game to measure the user's refractive power. It is possible to obtain information about the abnormality and change the settings of the display panel to those corresponding to the refractive power at which the result of the game was best.

Third, by using a vision measurement game, it is more fun and enjoyable than a conventional vision test chart such as which direction is the C shape open or what is its shape? It is more fun and enjoyable, and provides accurate results, for example, early detection of diseases such as macular degeneration. It is possible to recognize.

1 is a schematic diagram showing the configuration of a user vision measurement and response system according to the present invention.
Figure 2 shows the difference between the grids seen when there is no macular degeneration and when macular degeneration occurs.
Figure 3 is a diagram for explaining analysis when two lenses are arranged.
Figure 4 shows that the retinal image formed on the retina is a combination of images 1 and 2 formed from an individual lens array. In this case, the same image is placed in the overlapping area of images 1 and 2 so that they are recognized as one image. This is a diagram showing that it can be processed.
Figure 5 shows an example of a cross-sectional side view of an MLA multilayer structure.
FIG. 6 illustrates an image processing method for an image formed through a multi-layered MLA portion (a in FIG. 6) formed in the same structure as in FIG. 5.
Figure 7 shows a basic image processing flowchart.
Figure 8 shows a set of transparent pixels of the front app UI, and is a diagram to explain detection of changes in the underlying app UI through this.
FIG. 9 is a diagram illustrating a method for capturing an underlying app UI image by processing the entire front app UI area to be transparent, that is, a method of obtaining an underlying app UI image.
Figure 10 is a block diagram showing a detailed configuration of the overall logic for processing captured images.
Figure 11 shows the case where the MLA is inside the eye area and the case where the MLA is outside the eye area, showing the angle of the image area seen from the eye through the MLA, the coordinates of the virtual image area seen from the eye through the MLA, and the virtual image This shows how to obtain the coordinates of the mapped display area.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. Prior to this, terms or words used in this specification and claims should not be construed as limited to their usual or dictionary meanings, and the inventor should appropriately use the concept of terms to explain his or her invention in the best way. Based on the principle of definability, it must be interpreted with meaning and concept consistent with the technical idea of the present invention.

Accordingly, the embodiments described in this specification and the configurations shown in the drawings are only one of the most preferred embodiments of the present invention and do not represent the entire technical idea of the present invention, so at the time of filing this application, various alternatives are available to replace them. It should be understood that equivalents and variations may exist.

(System Configuration)

1 is a schematic diagram showing the configuration of a user vision measurement and response system according to the present invention. Referring to FIG. 1, a user vision measurement and response system 10 according to a preferred embodiment of the present invention will be described.

As shown in FIG. 1, the user vision measurement and response system 10 according to a preferred embodiment of the present invention includes a smart terminal 100 and an application 200 that operates on the smart terminal.

First, the smart terminal 100 is a configuration that includes a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality, and is used for various electronic devices running today's applications, such as smart phones, iPads, Galaxy Pads, and laptops. These are all general concepts. The multi-layer MLA structure will be described in detail below, but basically, refer to the content disclosed in Republic of Korea No. 10-1976759, which the present applicant applied for and received patent registration for.

Next, the application 200 operates the vision measurement game on the above-described smart terminal 100 according to the user's operation, so that it is natural in the process of playing the vision measurement game, that is, without the user feeling that vision measurement is being performed. , It is a configuration that obtains information about the user's refractive power abnormality and, as a result, changes the settings of the display panel 300 in response to the degree of the user's refractive power abnormality. Here, the applicator can be changed to various other expressions such as app or program.

Such an application 200 provides the user with a video, still image, animation, or game for visual acuity measurement using a screen corresponding to a plurality of refractive powers, and utilizes the results of the user's game. It is possible to obtain information about the user's refractive power abnormality and change the settings of the display panel to those corresponding to the refractive power that gave the best game results.

Here, when playing a game and showing screens corresponding to multiple refractive indices, the applicant sets the screen that best matches the user's refractive power over the course of playing the game several times if the user's game performance is different even though it is the same game. It was designed on the assumption that the best grades, results, and performance will be achieved.

(Game for measuring eyesight)

The vision measurement game used in the present invention will be described. A vision measurement game is functionally an application for the purpose of measuring vision, but it is also possible to implement the functions used in the present invention into a game for the user's fun or pleasure. In other words, it is possible to obtain information about the user's refractive index abnormality by incorporating the core function of the present invention into a game that is widely used by users, and to change the settings of the user's display panel to correspond to the obtained refractive index abnormality. .

Meanwhile, it is also possible to use vision measurement games not only for measuring refractive index abnormalities, but also for other vision measurement purposes such as macular degeneration or light blur.

For example, macular degeneration is a disease that causes vision problems due to changes in the macula, located in the center of the retina inside the eye. The early symptoms of macular degeneration include deformity. When deformed, the object appears bent. This mainly appears as a symptom of early choroidal neovascularization. Choroidal neovascularization is when new blood vessels develop under the retina in the macular area. Proliferation of blood vessels under the retina and resulting hemorrhage causes the retina to become curved. In other words, the retina becomes convex and protrudes. Deformity is a vision problem that occurs when the retina, which should be flat, becomes bent.

Second, there is a central scotoma. A central scotoma is a case where an invisible area occurs in the central area of vision. In other words, there is a small area that appears relatively dark or not visible at all compared to other areas. These symptoms can be detected when examining one eye at a time while covering the other eye.

For reference, Figure 2 shows the difference between the grids seen when there is no macular degeneration and when macular degeneration occurs.

With this in mind, this is an example of determining macular degeneration in a vision measurement game, such as when a character in the game does not perform properly in a mission that requires moving forward straight, or in a mission such as a minesweeper or treasure hunt. It is possible to design a game for measuring visual acuity so that the suspicion score for macular degeneration increases when it is difficult to find mines or treasures in the exact center of the screen, but there is a difference in results or performance as you move closer to the edge of the screen.

That is, in the vision measurement game, based on the difference in results, performance, or performance between when the character advances past the exact center of the display panel and when the character advances past the edge of the display panel in a mission in which the character must advance straight forward. or, in a minesweeper or treasure hunt mission, based on the difference in results, performance, or performance between a mine or treasure hunt located in the exact center of the display panel and a mine or treasure hunt located at the edge of the display panel, the vision; It is possible for the vision measurement game to be designed so that the macular degeneration suspicion score of the user playing the measurement game increases.

By using this type of vision measurement game, it is possible to naturally obtain accurate results while being more fun and enjoyable than the conventional vision test charts such as which direction is the C shape open or what shape is it?

Likewise, it is possible to check the degree of light blur symptoms through a vision measurement game. For example, if the screen is configured according to the degree of light blur symptoms, and even though the mission is of the same difficulty, if there are differences in grades, results, and performance depending on the degree of light blur symptoms, the game for measuring vision is based on this. It is possible to design a game for visual acuity measurement to increase the suspicion score of light blur disease in users who are using it.

Likewise, it is possible to implement the same vision measurement game to check the degree of refractive index abnormality. For example, if there are differences in grades, results, and performance in various screen configurations depending on the refractive index despite missions of similar or similar difficulty, it is possible to check the refractive index of the user playing the game for measuring vision based on this. It is also possible to change the settings of the display panel using the corresponding result value to verify whether the judgment result of the vision measurement game is correct.

(Verification of judgment results of games for visual acuity measurement)

As described above, by playing a vision measurement game, the suspicion score of refractive index abnormality, macular degeneration, or light blur of the user playing the vision measurement game is determined, and the results are used to correct the settings of the display panel. By changing it, it is possible to verify whether the judgment of the vision measurement game for the user is correct, that is, by seeing whether the same difficulty mission with the changed display panel settings produces better performance, grades, and results.

Below, the multi-layer MLA structure will be described as an example of a method of changing the settings of the display panel corresponding to the refractive index when the refractive index is higher. As mentioned above, basically, reference is made to the contents disclosed in Republic of Korea No. 10-1976759, which the present applicant (Pixelo) applied for and received patent registration for.

(Multi-layer MLA structure)

The multi-layer MLA structure replaces the conventional single-layer lens array arrangement by arranging arrays with different lens characteristics into multiple layers, allowing screen recognition and correction effects to be felt at normal use distances without maintaining a large distance between the display and the MLA. It is what makes it possible.

That is, the multi-layer MLA structure is a multi-layer MLA structure that corrects the screen of the display panel to the degree of the user's refractive power abnormality for users with eye refractive power problems, including presbyopia, myopia, and hyperopia. The first gap of the display panel a first MLA lens having a lower thickness of (gap_1), spaced apart from the display panel by the first gap (gap_1), and having a height of the first lens (lens_sag_height_1); a first gap layer on the first MLA lens having a first thickness (gap_thinckness_1); a second MLA lens having a lower thickness of the second gap (gap_2), being spaced apart from the first gap layer by the second gap (gap_2), and having a height (lens_sag_height_2) of the second lens; an nth gap layer on the nth MLA lens having an nth thickness (gap_thinckness_n) (n is a natural number of 2 or more); and an n-th MLA lens, which has a lower thickness of the n-th gap (gap_n), is spaced apart from the n-th gap layer by the n-th gap (gap_n), and has a height (lens_sag_height_n) of the n-th lens.

In addition, such a multi-layer MLA structure can be made separately from the display panel and attached to the display panel, or it can be formed integrally with the display panel structure on the upper layer of the display panel.

Figure 3 is a diagram to explain the analysis when two lenses are arranged, and Figure 4 shows that the retinal image formed on the retina is a combination of images 1 and 2 formed from an individual lens array. In this case, the image It is a diagram showing that the same image can be processed to be recognized as one image by placing the same image in the overlapping area of images 1 and 2, and Figure 5 shows an example of a side cross-sectional view of the MLA multilayer structure according to the present invention. 6 shows an image processing method for an image formed through the multi-layered MLA part (a in FIG. 6) formed in the same structure as in FIG. 5.

Using FIG. 3, analysis in the case of arranging two lenses will be explained. When multiple lenses are arranged, the image position and magnification can be calculated through the lens formula. When two lenses are arranged as shown in Figure 3, the analysis is as follows.

As shown in FIG. 3, the image (S1') created by the first black lens (lens1) for the object S1 and the magnification M1 of this image are calculated as Equation 1.

formula 1

S1' becomes the object (S2) of the second green lens (lens2), and the image (S2') created by this object and the magnification M2 are calculated as in Equation 2 below.

formula 2

The final image magnification is created by multiplying M1 and M2. Here, when comparing the magnification of the object position S1 and the image position S2', the magnification of the image created by the actual two lenses is as shown in Equation 3. The object S1 is moved by the same amount of dmove with a single lens (lens3) and the image is transferred to S2'. It becomes smaller than S2'/S1, which is the magnification at the time of concluding.

Formula 3

Using this principle, it is possible to generate a shift in the image distance that produces a correction effect even at a small magnification. The movement distance effect through small magnification increases the resolution of the moved image, making it possible to clearly show the image even if the image is moved a long distance and to create the effect of expanding the area where the image is visible.

Expanding the content described above, if there is a first image (image_1) to be displayed on the display panel, a second image (image_2) by the first MLA lens for the first image (image_1), the second image A third image (image_3) is generated by the second MLA lens for (image_2), an n+1 image (image_n+1) is generated by the n MLA lens for the nth image (image_n), and the first The distance between the image (image_1) and the first MLA lens is S1, the distance between the second image (image_2) and the first MLA lens is S1', the focal length of the first MLA lens is f1, and the first When the magnification of the second image for the image is expressed as M1, , , the distance between the nth image (image_n) and the nth MLA lens is Sn, the distance between the n+1 image (image_n+1) and the nth MLA lens is Sn', and the nth MLA lens is If the focal length of is expressed as fn and the magnification of the n+1th image with respect to the nth image is expressed as Mn, , and improves resolution compared to a single lens with a magnification of M1M2..Mn.

The present invention is made up of the principle explained through FIG. 3, and the images generated by individual lenses in the multi-layer MLA are enlarged beyond the display and overlap each other at the position where the images are formed. By calculating the overlapping part of each image generated by each lens and arranging the same display image, it is possible to create the desired overall image.

As shown in FIG. 4, the retinal image formed on the retina is a combination of images 1 and 2 formed from an individual lens array. At this time, by placing the same image in the overlapping area between image 1 and image 2, the image can be recognized as one image.

Figure 5 shows an example of a cross-sectional side view of an MLA multilayer structure according to the present invention. As shown in Figure 5, a multi-layer MLA (Multi-Layered MLA) is constructed by placing an MLA Layer layer with a certain gap on the display and additionally placing an MLA Layer layer with a certain gap on the MLA layer. do. The lens characteristics of the gap and individual MLA are calculated by adjusting them to match the image movement distance. Here, it is possible to set the gap between the lens and the display, the height of lens layer 1, the gap between lens 1 and lens 2 (physical properties such as the material of the gap), and the height of lens layer 2 as needed.

FIG. 6 illustrates an image processing method for an image formed through a multi-layered MLA portion (a in FIG. 6) formed in the same structure as in FIG. 5. As shown in Figure 6c, depending on the characteristics of the MLA unit with a multi-layer structure, a partial area of the display unit (Figure 6b) is moved to a specific distance for each unit lens area to form an image. (c in Figure 6)

The image of the shifted image plane passes through the lens and is combined into a single combined image on the retina. The degree of image overlap varies depending on the position and direction of the user's eyes. This is calculated according to the distance value between the lens and the MLA part of the multi-layer structure to find the size and image values of the overlapping part of the image (c-1 in Figure 6, c-2) The image values obtained by processing the overlapping areas are extracted and extracted as the image values of the display unit in b in FIG. 6. (b-1 in Figure 6) When this process is repeatedly performed on individual lens areas of the entire MLA to generate an image of the display unit, the processed image can finally be moved to a certain distance and formed in the user's eyes.

Here, in order to process the image area moved by an individual lens, the center position of the moved image is found and the image is extracted according to the size of the magnification at which the image is enlarged based on this center position. In order to find the center position of an image moved by an individual lens, it can be calculated according to the angle (yan) of the eye looking at the moved image, as shown in FIG. 11. The angle of the eye (yan) is the angle of incidence of the image entering the eye through an individual lens, and is derived based on the distance (de) between the eye and the display and the direction between the lens and the user.

Calculate the center position (yc) where the center of the lens (mc) is moved in a state parallel to the eye in accordance with the derived eye angle (yan), and image image incident on the eye based on the moved center position (yc) The area is extracted.

The extracted area is adjusted to fit the size of the display area according to the magnification of the moved image and converted into a display image value. By extracting the image value for the entire lens, the display image value of the image moved at a certain distance is obtained.

Figure 7 shows a basic image processing flowchart, Figure 8 shows a transparent pixel set of the front app UI, and is a diagram to explain the detection of changes in the underlying app UI through this, and Figure 9 shows the entire area of the front app UI. This is a diagram to explain a method of acquiring an underlying app UI image, which allows capturing the underlying app UI by processing it transparently, and FIG. 10 is a block diagram showing a detailed configuration of the entire logic for processing the captured image. It is also a degree.

Hereinafter, using FIGS. 7 to 10, a technology applicable to an application that processes a specific image using a screen image currently output from a device with a display and then outputs the image will be described.

For example, in order to enable users with presbyopia disease to better see the image currently displayed on the screen on a smartphone, it is applied to an application that has a service function that processes the image and then outputs the image to the screen. can do.

When applying to a service application that has the function of using the image currently displayed on the screen to output on the screen an image prime created from that image through specific image processing, it is structurally impossible to obtain the source image as an independent channel. We present a method to extract source images from the system.

The underlying app UI is the source image, and the front app UI outputs an image prime obtained through image processing from the underlying app image.

However, the problem is that there is only one channel that captures the image currently displayed on the screen, and in that channel, two images, the source image and the image prime, coexist independently of time. Since the system (android framework, iOS framework, etc.) does not provide a method for obtaining only the source image from two images that coexist independently of time, an algorithm for obtaining the source image is needed.

In a system that does not provide a function for capturing the source image (Underlying App UI) from an independent channel, the method to obtain the source image is as follows.

Underlying App UI is the App UI that operates under the Front App UI, and is the source image of the image prime output from the Front App UI. Front App UI is an App UI that outputs an image prime processed from a source image. The Previous Image of Front App UI is the n-1th captured Front App UI image. The Current Image of Front App UI is the nth captured Front App UI image. Logic for detecting changed source image is logic that detects changes in the source image. Magic code for detecting dropped image prime occurs when the image prime is displayed on the screen, the image prime is dropped due to fense signal synchronization and is not displayed on the screen. Therefore, a magic code is injected into the image prime, the screen image is captured, and the magic code is used to check whether or not it is dropped.

Figure 7 shows a basic processing flow chart. As shown in Figure 7, the full screen image (source image) of the underlying app UI is captured, and the image (image prime) resulting from image processing of the captured image is output to the screen.

Next, the Front App UI is output in the overlay method, and events related to user interaction (screen touch) are transmitted to the underlying app. In other words, the user touches the Front App UI, but user interaction events are transmitted to the Underlying App, and the Underlying App processes the events. (Apply overlay of front app UI)

Next, in order to know changes in the Underlying App UI, a pixel set at a specific location in the Front App UI is processed transparently. The pixel set has the image pixel value of the underlying app UI, not the image pixel value of the front app UI. Therefore, if you compare the pixel set between the captured previous image of the front app UI and the current image of the front app UI, you can see that the underlying app UI image has changed. Figure 8 shows a set of transparent pixels of the front app UI (detecting changes in the underlying app UI).

Next, when the image change of the underlying app UI is known, the underlying app UI can be captured by processing the entire front app UI area to be transparent, as shown in Figure 9. (How to obtain the underlining app UI image)

Figure 10 is a block diagram showing a detailed configuration of the overall logic for processing captured images.

After image processing from the source image, the image prime for output may be a dropped image due to fence signal synchronization. This causes an error in the logic for detecting changed source image, so in order to detect drop image prime, the pixel value at a specific location in the output image prime is adjusted over time. Inject a unique magic code and check the drop by checking the pixel value from the captured image. (Magic code applied)

As described above, although the present invention has been described with limited examples and drawings, the present invention is not limited thereto, and the technical idea of the present invention and the following will be understood by those skilled in the art to which the present invention pertains. Of course, various modifications and variations are possible within the scope of equivalence of the claims to be described.

10...User vision measurement and response system
100...Smart terminal
200...Application
300...display panel

Claims (18)

A smart terminal including a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality; and
The smart terminal operates a game for visual acuity measurement according to the user's operation, obtains information about the user's refractive power abnormality in the process of playing the visual acuity measurement game, and displays the display corresponding to the degree of the user's refractive power abnormality. Includes an application that changes the settings of the panel,
The application allows the user to play the vision measurement game multiple times using a video, still image, animation, or screen corresponding to each of a plurality of refractive powers, and provides a plurality of results for the game played multiple times by the user. Based on each of the above, obtain information about the user's refractive power abnormality, and change the settings of the display panel to the settings of the display panel corresponding to the best refractive power among the results of the game played multiple times. ,
User vision measurement and response system.
delete According to claim 1,
In the above vision measurement game,
Based on the difference in results, performance, or performance between advancing past the exact center of the display panel and advancing past the edge of the display panel in a mission in which the character must advance straight forward,
Characterized in that the vision measurement game is designed to increase the macular degeneration suspicion score of the user playing the vision measurement game.
User vision measurement and response system.
According to claim 1,
In the above vision measurement game,
Based on the difference in results, performance, or performance between a mine or treasure hunt located in the exact center of the display panel and a mine or treasure hunt located at the edge of the display panel in a minesweeper or treasure hunt mission,
Characterized in that the vision measurement game is designed to increase the macular degeneration suspicion score of the user playing the vision measurement game,
User vision measurement and response system.
According to claim 3 or 4,
The multilayer MLA structure is,
A multi-layer MLA structure that corrects the screen of the display panel to the degree of the user's refractive power abnormality for users with eye refractive problems including presbyopia, myopia, and hyperopia,
a first MLA lens having a lower thickness of the first gap (gap_1) of the display panel, being spaced apart from the display panel by the first gap (gap_1), and having a height (lens_sag_height_1) of the first lens;
a first gap layer on the first MLA lens having a first thickness (gap_thinckness_1);
A second MLA lens formed on the first gap layer, has a lower thickness of the second gap (gap_2), is spaced apart from the first gap layer by the second gap (gap_2), and has a height (lens_sag_height_2) of the second lens. ;
a second gap layer on the second MLA lens having a second thickness (gap_thinckness_2);
It is formed on the (n-1)th gap layer, has a lower thickness of the nth gap (gap_n), is spaced apart from the (n-1)th gap layer by the nth gap (gap_n), and has a height of the nth lens (lens_sag_height_n) ) having the nth MLA lens (n is a natural number of 3 or more); and
Comprising: an nth gap layer on the nth MLA lens having an nth thickness (gap_thinckness_n) (n is a natural number of 3 or more);
User vision measurement and response system.
According to claim 5,
When processing images to correct the user's refractive power abnormality on the display panel,
Capturing an entire image of the underlying app;
Processing the entire image of the underlying app to generate an image prime corrected according to the degree of refractive error of the user; and
Processing the image, including displaying the image prime as the entire image of the front app,
User vision measurement and response system.
In the user vision measurement and response method,
A smart terminal starting a game for visual acuity measurement according to a user's operation;
Obtaining information about the user's refractive power abnormality in the process of playing the game for measuring visual acuity; and
Comprising: changing settings in the vision measurement game based on information about the user's refractive power abnormality, and verifying the judgment about the refractive power abnormality,
The starting step is,
Playing the visual acuity measurement game multiple times using videos, still images, animations, or screens corresponding to each of a plurality of refractive powers to the user,
The step of obtaining the information is,
Based on each of the plurality of results for the game played multiple times by the user, obtain information about the user's refractive power abnormality,
The step of verifying the judgment is,
Characterized by using a multi-layer MLA structure to change the settings of the display panel on which the vision measurement game is displayed to the settings of the display panel corresponding to the best refractive power among the results of the game played multiple times.
How to measure and respond to the user's vision.
delete According to claim 7,
The multilayer MLA structure is,
A multi-layer MLA structure that corrects the screen of the display panel to the degree of the user's refractive power abnormality for users with eye refractive problems including presbyopia, myopia, and hyperopia,
a first MLA lens having a lower thickness of the first gap (gap_1) of the display panel, being spaced apart from the display panel by the first gap (gap_1), and having a height (lens_sag_height_1) of the first lens;
a first gap layer on the first MLA lens having a first thickness (gap_thinckness_1);
A second MLA lens formed on the first gap layer, has a lower thickness of the second gap (gap_2), is spaced apart from the first gap layer by the second gap (gap_2), and has a height (lens_sag_height_2) of the second lens. ;
a second gap layer on the second MLA lens having a second thickness (gap_thinckness_2);
It is formed on the (n-1)th gap layer, has a lower thickness of the nth gap (gap_n), is spaced apart from the (n-1)th gap layer by the nth gap (gap_n), and has a height of the nth lens (lens_sag_height_n) ) having the nth MLA lens (n is a natural number of 3 or more); and
Comprising: an n-th gap layer on the n-th MLA lens having an n-th thickness (gap_thinckness_n) (n is a natural number of 3 or more);
How to measure and respond to the user's vision.
According to clause 9,
A method of processing an image for correcting a user's refractive power abnormality on the display panel,
Capturing an entire image of the underlying app;
Processing the entire image of the underlying app to generate an image prime corrected according to the degree of refractive error of the user; and
Processing the image, including displaying the image prime as the entire image of the front app,
How to measure and respond to the user's vision.
In the media stored in a smart terminal including a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality and readable by the smart terminal,
The smart terminal starting a game for visual acuity measurement according to the user's operation;
In the process of playing a game for measuring visual acuity for a user, obtaining information about the user's refractive power abnormality; and
Executing a step of verifying the determination of the refractive power abnormality by changing settings in the vision measurement game based on information about the user's refractive power abnormality,
The starting step is,
Playing the visual acuity measurement game multiple times using videos, still images, animations, or screens corresponding to each of a plurality of refractive powers to the user,
The step of obtaining the information is,
Based on each of the plurality of results for the game played multiple times by the user, obtain information about the user's refractive power abnormality,
The step of verifying the judgment is,
Characterized by using the multi-layer MLA structure to change the settings of the display panel on which the vision measurement game is displayed to the settings of the display panel corresponding to the best refractive power among the results of the game played multiple times.
A medium that can be read by a smart terminal that records a program.
delete According to claim 11,
A multi-layer MLA structure that corrects the screen of the display panel to the degree of the user's refractive power abnormality for users with eye refractive problems including presbyopia, myopia, and hyperopia,
a first MLA lens having a lower thickness of the first gap (gap_1) of the display panel, being spaced apart from the display panel by the first gap (gap_1), and having a height (lens_sag_height_1) of the first lens;
a first gap layer on the first MLA lens having a first thickness (gap_thinckness_1);
A second MLA lens formed on the first gap layer, has a lower thickness of the second gap (gap_2), is spaced apart from the first gap layer by the second gap (gap_2), and has a height (lens_sag_height_2) of the second lens. ;
a second gap layer on the second MLA lens having a second thickness (gap_thinckness_2);
It is formed on the (n-1)th gap layer, has a lower thickness of the nth gap (gap_n), is spaced apart from the (n-1)th gap layer by the nth gap (gap_n), and has a height of the nth lens (lens_sag_height_n) ) having the nth MLA lens (n is a natural number of 3 or more); and
Comprising: an n-th gap layer on the n-th MLA lens having an n-th thickness (gap_thinckness_n) (n is a natural number of 3 or more);
Media that can be read on smart devices.
According to claim 13,
When changing the settings in the game for visual acuity measurement, the image for correcting the user's refractive error in the display panel is processed,
Capturing an entire image of the underlying app;
Processing the entire image of the underlying app to generate an image prime corrected according to the degree of refractive error of the user; and
Processing the image, including displaying the image prime as the entire image of the front app,
Media that can be read on smart devices.
In an application that is stored in a storage medium in a smart terminal and operates in conjunction with the smart terminal, including a display panel with a multi-layer MLA structure that corrects the user's refractive power abnormality,
The smart terminal starting a game for visual acuity measurement according to the user's operation;
In the process of playing the game for measuring visual acuity for the user, obtaining information about the user's refractive power abnormality; and
Executing a step of verifying the judgment about the refractive power abnormality by changing settings in the vision measurement game based on information about the user's refractive power abnormality,
The starting step is,
Playing the visual acuity measurement game multiple times using videos, still images, animations, or screens corresponding to each of a plurality of refractive powers to the user,
The step of obtaining the information is,
Based on each of the plurality of results for the game played multiple times by the user, obtain information about the user's refractive power abnormality,
The step of verifying the judgment is,
Characterized by using the multi-layer MLA structure to change the settings of the display panel on which the vision measurement game is displayed to the settings of the display panel corresponding to the best refractive power among the results of the game played multiple times.
An application stored on a storage medium that can be read by the smart terminal.
delete According to claim 15,
A multi-layer MLA structure that corrects the screen of the display panel to the degree of the user's refractive power abnormality for users with eye refractive problems including presbyopia, myopia, and hyperopia,
a first MLA lens having a lower thickness of the first gap (gap_1) of the display panel, being spaced apart from the display panel by the first gap (gap_1), and having a height (lens_sag_height_1) of the first lens;
a first gap layer on the first MLA lens having a first thickness (gap_thinckness_1);
A second MLA lens formed on the first gap layer, has a lower thickness of the second gap (gap_2), is spaced apart from the first gap layer by the second gap (gap_2), and has a height (lens_sag_height_2) of the second lens. ;
a second gap layer on the second MLA lens having a second thickness (gap_thinckness_2);
It is formed on the (n-1)th gap layer, has a lower thickness of the nth gap (gap_n), is spaced apart from the (n-1)th gap layer by the nth gap (gap_n), and has a height of the nth lens (lens_sag_height_n) ) having the nth MLA lens (n is a natural number of 3 or more); and
Comprising: an n-th gap layer on the n-th MLA lens having an n-th thickness (gap_thinckness_n) (n is a natural number of 3 or more);
An application stored on a storage medium that can be read by the smart terminal.
According to claim 17,
When changing the settings in the game for visual acuity measurement, the image for correcting the user's refractive error in the display panel is processed,
Capturing an entire image of the underlying app;
Processing the entire image of the underlying app to generate an image prime corrected according to the degree of refractive error of the user; and
Processing the image, including displaying the image prime as the entire image of the front app,
An application stored on a storage medium that can be read by the smart terminal.

Images