WO2021004138A1

WO2021004138A1 - Screen display method, terminal device, and storage medium

Info

Publication number: WO2021004138A1
Application number: PCT/CN2020/087136
Authority: WO
Inventors: 梅锦振华
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2019-07-05
Filing date: 2020-04-27
Publication date: 2021-01-14
Also published as: CN110377385B; CN110377385A

Abstract

The present application provides a screen display method, a terminal device, and a storage medium, applicable to the technical field of computer vision in artificial intelligence. The method comprises: obtaining identity information of a user; searching for the degree of myopia of the user on the basis of the identity information; if the search for the degree of myopia fails, obtaining a face image of the user, and identifying the degree of myopia of the user on the basis of the face image; and obtaining a corresponding display parameter set by matching on the basis of the degree of myopia of the user, and performing display parameter adjustment on the screen on the basis of the obtained display parameter set. According to embodiments of the present application, a display parameter condition can be adjusted according to the actual situation of the user, so that the actual display situation can further satisfy the viewing comfort of the user, and the human-computer interaction efficiency between the user and the intelligent device is ensured.

Description

Screen display method, terminal equipment and storage medium

The present invention claims the priority of a Chinese patent application filed with the Chinese Patent Office on July 5, 2019, the application number is 201910603946.X, and the application name is "a screen display method, device and terminal equipment", the entire content of which is by reference Incorporated in this application.

Technical field

This application belongs to the field of computer vision technology in artificial intelligence, and in particular relates to screen display methods, terminal devices and storage media.

Background technique

With the development of technology, smart devices such as mobile phones, tablets, and computers have become an indispensable part of modern people’s lives. At the same time, due to living habits and work needs, people often face the screen of smart devices for a long time, which makes More and more people have become nearsighted.

In order to relieve the eyestrain of the user when viewing the screen and protect the eyesight of the user, the prior art has introduced screen display solutions such as "eye protection mode" and "warm color mode". The principle of these solutions is basically to directly saturate the color of the screen. The degree of brightness, etc. are adjusted to a fixed display parameter scheme, so that the user’s eye fatigue is reduced a bit when viewing the screen. In actual situations, although the display scheme using fixed display parameters can relieve eye fatigue to a certain extent, on the one hand, the actual situation of different users will lead to certain differences in the display schemes, making it difficult for fixed display parameters to meet the actual needs of different users. On the other hand, the greater the number and amplitude of the adjusted parameters, the greater the impact on the visual effect of the user viewing the screen, which makes the user extremely uncomfortable when viewing the screen, which in turn causes the efficiency of human-computer interaction between the smart device and the user to decrease. The inventor realizes that the existing screen display solutions are not very adaptable, and therefore cannot meet the actual eye fatigue relief needs of different users, and ensure the efficiency of human-computer interaction between users and smart devices.

technical problem

In view of this, the embodiments of the present application provide a screen display method, terminal device, and storage medium to solve the problem that the screen display solution in the prior art has poor adaptability and cannot meet the actual eye fatigue relief needs of different users to ensure that users and The efficiency of human-computer interaction of smart devices.

Technical solutions

The first aspect of the embodiments of the present application provides a screen display method, including:

Obtain user's identity information;

Searching for the myopia level of the user based on the identity information;

If the search for myopia level fails, acquiring a face image of the user, and identifying the myopia level of the user based on the face image;

Match the corresponding display parameter set based on the myopia level of the user, and adjust the display parameter of the screen based on the obtained display parameter set.

A second aspect of the embodiments of the present application provides a terminal device, the terminal device includes a memory and a processor, the memory stores a computer program that can run on the processor, and the processor executes the following steps : Obtain the user's identity information; search for the user's myopia level based on the identity information; if the myopia level search fails, obtain the user's face image, and identify the user's myopia level based on the face image; Match the corresponding display parameter set based on the myopia level of the user, and adjust the display parameter of the screen based on the obtained display parameter set.

The third aspect of the embodiments of the present application provides a computer-readable storage medium. When the computer-readable storage medium is executed by one or more processors, the one or more processors perform the following steps: obtain user identity information Find the myopia level of the user based on the identity information; if the myopia level search fails, obtain the face image of the user, and identify the myopia level of the user based on the face image; based on the myopia level of the user The level matches the corresponding display parameter set, and the display parameter adjustment of the screen is performed based on the obtained display parameter set.

Beneficial effect

Compared with the prior art, the embodiment of this application has the beneficial effect that: the embodiment of this application recognizes the user’s myopia level in real time according to the actual collected face images, and adaptively selects different display parameters according to the user’s different myopia conditions The solution adjusts the screen display, so that the screen display can specifically meet the actual eye fatigue relief needs of different users, and the display parameters are adjusted according to the user's actual situation, so that the actual display situation can further meet the user's viewing The comfort level ensures the efficiency of human-computer interaction between users and smart devices.

Description of the drawings

FIG. 1 is a schematic diagram of the implementation process of the screen display method provided by Embodiment 1 of the present application;

2 is a schematic diagram of the implementation process of the screen display method provided in the second embodiment of the present application;

3 is a schematic diagram of the implementation process of the screen display method provided by the third embodiment of the present application;

4 is a schematic diagram of the implementation process of the screen display method provided by the fourth embodiment of the present application;

FIG. 5 is a schematic diagram of the implementation process of the screen display method provided by Embodiment 5 of the present application;

6 is a schematic diagram of the implementation process of the screen display method provided by the sixth embodiment of the present application;

FIG. 7 is a schematic diagram of the implementation process of the screen display method provided in the seventh embodiment of the present application;

FIG. 8 is a schematic structural diagram of a screen display device provided in Embodiment 8 of the present application;

FIG. 9 is a schematic diagram of a terminal device provided in Embodiment 9 of the present application.

Embodiments of the invention

In order to facilitate the understanding of the application, the embodiments of the application are briefly described here. Since in the prior art, the methods for alleviating eye fatigue are directly displayed on the screen with a fixed display parameter scheme, which cannot meet the actual needs of different users. , It will also affect the user's screen viewing comfort and reduce the efficiency of human-computer interaction.

Taking into account the actual situation, on the one hand, for users with different degrees of myopia, due to the different wavelengths and refractive indexes of different colors of light, the same light source emits different colors of light in the final focus position of the eyeball, such as red light. If behind the retina, the green light is likely to fall in front of the retina. Therefore, even when facing the same screen, users with different degrees of myopia will see different display content, especially the clarity of the display content. Therefore, the user with the higher degree of myopia in the same display solution will have poorer and unclear visual effects, which makes the user look harder and prone to eye fatigue. On the other hand, users with different degrees of myopia have different tolerance to eye fatigue. Generally, those with severe myopia are more prone to fatigue. It is precisely based on the requirements of these actual conditions that in the embodiments of this application, the user’s myopia is divided into multiple different myopia levels according to the specific degree of myopia, and appropriate display parameter sets are set for users with different myopia levels to ensure Corresponding visual effects and fatigue relief effects, intelligently recognize the user’s myopia level when the user is watching the screen, and select a preset suitable display parameter set according to the user’s actual myopia level, and finally control according to the selected display parameter set The actual display color matching and other indicators of the screen enable the screen to display the most suitable effect, and realize the adjustment of the display parameters according to the actual situation of the user, so that the actual display scheme can further satisfy the user's viewing comfort and ensure that the user and The human-computer interaction efficiency of smart devices. The details are as follows:

Fig. 1 shows the implementation flowchart of the screen display method provided in the first embodiment of the present application, and the details are as follows:

S101: Obtain user identity information.

It should be noted that the screen display method in the embodiment of this application is applied to the terminal device, and in order to ensure the acquisition of the user's face image in the embodiment of this application, the terminal device in the embodiment of this application should have an image acquisition module/device , Such as a camera, or a communication connection with a third-party image acquisition device. Specifically, in the embodiments of the present application, the terminal device may be a personal computer, a tablet computer, a mobile phone, and other devices, which are specifically determined by actual application scenarios.

In the embodiments of this application, each user corresponds to an identity information. The identity information can be stored in the form of a unique identification, or in the form of images, etc., which are specifically set by the technicians according to actual needs. The embodiment will pre-store the identity information of some users and associate the identity information with the myopia level of the corresponding user, so as to realize the screen display adjustment in the first time according to the user's identity. Among them, the storage of user identity information and myopia level includes, but is not limited to: input by technicians based on acquired user information, input by users based on their actual myopia situation, and identification of the user's myopia level by the terminal device each time After that, the user's identity information is recorded at the same time and the two are stored in association, and any one or more of them are implemented. The classification rules for myopia levels can also be provided by the technicians themselves. For example, they can be simply divided into three levels: severe, minor, and normal, or they can be divided into more detailed categories, which are not limited here. At the same time, user identity information and The storage of myopia level can be stored locally in the terminal device, or in a third device such as a stored server, which is specifically set by the technician according to the actual situation.

After pre-associating and storing the identity information and myopia levels of some users, the embodiment of the present application will obtain the identity information of the current user to determine whether the current user has the corresponding myopia level. Among them, corresponding to the different storage methods of user identity information, this The application embodiment can also obtain user identity information in many ways, including but not limited to, directly identifying user identity information based on the personal information of the currently logged-in user, or performing biometric identification on the current user to determine identity information, etc. The technical personnel select the corresponding identity information acquisition method according to the actual user identity information storage method, which is not limited here.

S102: Search the user's myopia level based on the identity information.

After the identity information of the current user is determined, according to the storage method of the identity information and the myopia level, the embodiment of the present application will search for the corresponding myopia level locally or in the third device through a communication connection. The level is pre-stored, and the corresponding myopia level can be found directly at this time. If it has not been pre-stored, the search will fail at this time.

S103: If the search for myopia level fails, obtain a face image of the user, and identify the myopia level of the user based on the face image.

If the myopia level search fails, it means that the user's identity information cannot be directly obtained and screen adjustments are performed. In this case, the embodiment of the present application directly obtains the user's face image, and recognizes the user's myopia level based on the face image. Among them, the specific myopia level method is not limited here, and can be selected by the technicians according to the actual situation, including but not limited to: identifying the myopia level according to whether the user wears glasses, or according to the user's squinting degree (ie, human The degree of eye closure) is used to identify the myopia level, and the method of other related embodiments of this application can also be referred to for the myopia level identification. It should be noted that the embodiments of this application only need to roughly identify the degree of myopia of the user, and classify the myopia levels required by the embodiments of this application, that is, it is not necessary to identify the specific degree of myopia of the user during recognition. Therefore, the myopia level recognition method selected in the embodiments of this application can be a more accurate myopia data recognition algorithm, or a relatively low accuracy myopia level recognition algorithm to meet different hardware requirements. The actual requirements of the configured terminal device improve the adaptability of the embodiments of the present application to different scenarios.

S104: Match the corresponding display parameter set based on the user's myopia level, and adjust the display parameter of the screen based on the obtained display parameter set.

In the embodiment of the present application, the display parameter set includes at least one display parameter data of color matching, clarity, saturation, contrast, and brightness, etc. The specific data content contained therein can be supported by technicians according to actual user needs and terminal equipment The adjusted parameters are set.

As can be seen from the above description, this embodiment of the present application will pre-divide the user’s myopia degree into multiple myopia levels, and set a corresponding appropriate display parameter set for each myopia level. After obtaining the user’s myopia level, perform the procedure based on the myopia level. The matching determines a suitable display parameter set for the user, and finally adjusts the screen according to the specific display parameter data in the display parameter set, such as reducing the brightness.

As an optional embodiment of the present application, considering that in actual situations, because different colors of light have different effects on human eyes, there is an absorption peak on the retina of the human eye that is located in the blue region, which will excite and release free radicals. Ions, which easily cause the retinal pigment epithelium to shrink and cause the death of light-sensitive cells. Therefore, in general, blue light is very harmful to human eyes, while red light is also more harmful to human eyes. Therefore, when setting the display parameter set , The ratio of blue light and red light can be appropriately reduced to reduce eye damage and relieve eye fatigue. On the other hand, reducing these colors will directly affect the screen display effect. Therefore, this solution will target different myopia Level users carry out the matching design of the display scheme. For users with low myopia, the reduction of these lights in the corresponding display scheme is less to ensure better display effect. For users with high myopia, the reduction is larger. It is ensured that the eye fatigue relief effect is better, thereby achieving a balance between eye fatigue relief and display effect. Specifically, since blue and red are both harmful to the eyes, in the embodiment of the application, when setting the color matching for the display parameter set, a color matching with less blue and red content, or no blue and The red color is used as the background color, and multiple corresponding display schemes are set. The higher the myopia degree, the less red and blue the background color contains.

As a preferred embodiment of the present application, considering that in actual situations, the display effect and eye protection effect of the same screen brightness under different ambient light intensities will also be different. Therefore, the embodiment of the present application obtains the display parameter set while also It measures the light intensity of the environment where the terminal device is located, for example, uses a light sensor to detect the light intensity, then sets the screen display parameters according to the display parameter set, and adjusts the display brightness and contrast according to the ambient light intensity in real time.

In the embodiments of this application, the user’s myopia is divided into multiple different levels of myopia according to the specific degree of myopia, and appropriate display parameter sets are set for users of different levels of myopia to ensure the corresponding visual effect and the fatigue relief effect. The user intelligently recognizes the user’s myopia level when watching the screen, and selects a preset suitable display parameter set according to the user’s actual myopia level, and finally controls the actual display color and other indicators of the screen according to the selected display parameter set, so that the screen The most suitable effect can be displayed, and the display parameters can be adjusted according to the actual situation of the user, so that the actual display solution can further satisfy the user's viewing comfort and ensure the efficiency of human-computer interaction between the user and the smart device. Among them, the advantage of pre-storage is that there is more time to accurately measure the user’s myopia. The user can even directly input his own myopia degree and astigmatism into the terminal device, and then the terminal device converts it into the corresponding myopia level. , Thereby improving the accuracy of matching users.

As a specific implementation of myopia level recognition in the first embodiment of this application, considering that in actual situations, whether to wear glasses is an extremely effective method for identifying whether a user is nearsighted or not, therefore, the adoption of Identify whether the user wears glasses, and the distance between the eyes and the screen when wearing glasses, to quantify and identify the user's myopia level, as shown in Figure 2, including:

S201: If the face image contains glasses, obtain size information of the screen and detect the distance between the user's eyes and the screen.

In order to realize the recognition of whether the user wears glasses, in the embodiment of the present application, human eye recognition is first performed based on the acquired face image, and then the glasses are searched based on the identified human eye position. The details are as follows:

Step 1. Perform eye recognition and positioning on the face image, and determine the position of the eye in the face image.

Among them, there are many specific eye positioning methods, which can be selected by the technicians themselves. They are not limited here, including but not limited to pattern matching. Because the eye contour pattern is fixed, it can be matched to meet the requirements. Graphics, and then based on the symmetry of the two eyes, and the gray value of the surrounding skin will have a significant difference in gray value to determine the final eye position. Or first divide the acquired face image into 3 parts a, b, and c from top to bottom, the proportions of which are: a=30%, b=40%, c=30%, only analyze b Part, and use opencv to first detect the face of each frame, calculate the area of b after detection, and then perform gray-scale binarization on the b area to obtain the corresponding binarized image. Then the human eyes are positioned on the gray-scale binary image.

Step 2: Detect the closed curve contained in the face image based on the position of the human eye, and identify the closed curve containing the human eye in the curve based on the position of the human eye.

Step 3. If there are closed curves containing human eyes inside the curve, then these closed curves are recognized as glasses, and it is determined that the face image contains glasses. If there is no closed curve containing human eyes inside the curve, then it is determined that the face image contains glasses. Contains glasses.

After determining the position of the human eye, because the glasses are generally located around the glasses, and after the gray-scale binarization, the frame of the glasses will form a closed curve surrounding the eyes. Therefore, only need to search around the eyes in the face image It is enough to find a closed curve that meets the requirements. If it exists, it is determined that the user wears glasses. Since the shape of the glasses is generally known data, some possible glasses shapes can also be preset here, and then the closed curve is shape-matched before identifying whether the human eye is included, and only one of the shapes is matched. A high closed curve, such as a closed curve with a matching degree of more than 60%, is used to search and determine the position of the human eye.

At the same time, considering the actual situation for myopic users, when viewing the screen content, users with relatively normal eyesight will generally subconsciously close to the screen. For example, when looking at the screen of a mobile phone, they will generally be closer to the screen, and for screens of different sizes. In other words, the distance when users generally watch is also significantly different. For example, the distance when watching a large-size screen such as a computer TV is obviously higher than that of a small-size screen of a mobile phone tablet lamp. Therefore, in the embodiment of this application, the screen information will be obtained at the same time. The size information and the distance between the user’s eyes and the screen are used to improve the data for subsequent quantification of the user’s myopia level. Among them, the size information can be directly read the hardware parameters of the screen, and the human eye distance needs to be measured with the depth sensor based on the recognition and positioning of the human eye, so as to obtain the corresponding human eye distance. Therefore, when quantifying the distance between human eyes, the average distance between the two human eyes and the screen can be used as the final human eye distance.

S202: Identify the user's myopia level based on the size information and the human eye distance.

In order to realize the quantitative recognition of the user’s myopia level based on the screen size and the human eye distance, this embodiment of the present application will pre-calculate the habit data of some users with different myopia degrees in the case of different screen sizes, and obtain the corresponding data according to the data analysis. A threshold, that is, set different threshold arrays for screens of different sizes, and when judging the degree of myopia, first read the corresponding threshold array according to the device screen size, and then query and determine the user's myopia level based on the distance of the human eye.

As a specific implementation of myopia level recognition in the first embodiment of the present application, considering that in actual situations, myopic users are most likely to not wear glasses, so even on the basis of the second embodiment of the present application, it is difficult to be accurate Recognizing the myopia level of non-glass users, in order to realize the recognition of the myopia level of non-glass users, as shown in Figure 3, it includes:

S301: If the face image does not include glasses, detect the distance between the user's eyes and the screen, the degree of closure of the user's eyes, and the degree of frowning of the user based on the face image.

Similarly, in the embodiment of this application, it is also necessary to identify whether the user wears glasses. For the specific identification method, please refer to the description in the second embodiment of this application, which will not be repeated here, or it can be done by the technical personnel. Set up.

Considering that in actual situations, when users are viewing screen content, it is more difficult for nearsighted users who do not wear glasses to watch, so they usually habitually squint to limit the incidence of light so that they can see. It is more clear, and at the same time it is prone to fatigue and tension of the eye muscles, and some frowning occurs. In order to see more clearly, it will be relatively close to the screen. It is precisely based on the aforementioned symptoms that myopic users who do not wear glasses usually have, after recognizing that the user does not wear glasses, the embodiment of the present application will detect and evaluate the user from three aspects. in particular:

For the detection of the human eye distance, please refer to the related description of the second embodiment of the present application, which will not be repeated here.

For the detection of human eye closure, some human eye closure algorithms in the prior art can be directly selected for processing, such as detection based on the PERCLOS method, or it can be designed by the technicians themselves, or you can refer to the related description of Embodiment 4 of this application. There is no limitation here.

For the recognition of the degree of frowning, some existing facial expression recognition algorithms can be directly selected for processing, or they can be designed by the technicians themselves, or they can refer to the related description of Embodiment 5 of the present application, which is not limited here.

S302: Identify the myopia level of the user based on the distance of the human eye, the degree of eye closure, and the degree of frowning.

Among them, it should be noted that users who do not wear glasses may be non-myopia users to a large extent (in the embodiment of this application, such users can be classified as users with the lowest myopia level), so when performing myopia level recognition In the embodiments of this application, the screen size is not directly used as the quantification index of myopia level, but firstly determines whether the user is nearsighted based on the distance of the eyes, the degree of eye closure, and the degree of frowning. If the recognition result is that the user is not nearsighted, such as three indicators If all values are lower than the corresponding threshold, it indicates that the user is not myopic. In this case, the embodiment of the present application directly determines that the user has the lowest myopia level. Conversely, if the recognition result is that the user is nearsighted, this embodiment of the application will further read the size information of the screen, and further quantify the degree of the user's nearsightedness based on the size information, the distance of the human eye, the degree of eye closure, and the degree of frowning. Get the corresponding myopia level.

Among them, in order to realize the recognition of whether the user is nearsighted based on the distance of the human eye, the degree of closure of the human eye, and the degree of frowning, and the recognition and quantification of the degree of myopia of the user based on the size information, the distance of the human eye, the degree of closure of the human eye, and the degree of frowning, the embodiment of the present application will advance Statistics on the habit of viewing screens of users with different degrees of myopia (including non-myopia users) under different screen sizes. At the same time, on the one hand, for myopia users and non-myopia users, the eye distance, the degree of eye closure, and the degree of frown According to the analysis, the corresponding thresholds for distinguishing myopia and non-myopia are obtained. On the other hand, the eye distance, eye closure degree and frown degree are analyzed for different sizes of screens, and different threshold arrays corresponding to different myopia levels are obtained. And when judging the degree of myopia, first read the corresponding threshold array according to the device screen size, and then query and determine the user's myopia level according to the distance of the human eye.

The embodiment of this application realizes the effective identification and quantification of the degree of nearsightedness of users who do not wear glasses through the comprehensive analysis of multiple indicators of the user. At the same time, on the basis of the second embodiment of this application, adaptive scene recognition of different application scenarios can be realized. And the matching of adaptive identification methods, so that the embodiments of the present application can greatly improve the adaptive and accurate identification of different scenes and users.

As a specific implementation for detecting the degree of closure of human eyes in the third embodiment of the present application, as shown in FIG. 4, the fourth embodiment of the present application includes:

S401: Perform grayscale processing on an image of a human eye region in a face image.

S402: Calculate a grayscale threshold t based on formula (1), and perform binarization processing on the grayscale image based on the grayscale threshold t.

δ(t) ² -n ₁ (1-n ₁ )(r ₁ -r ₂ ) ² (1)

[Corrected according to Rule 26 13.05.2020]

Among them, N is the total number of pixels in the grayscale image, n _i represents the number of pixels with grayscale value i, k is the maximum grayscale value of the pixels in the grayscale image, t∈[0,k] , Δ(t)2 is the variance of t.

In the embodiment of the present application, the adaptive calculation of the gray-scale threshold t is performed based on the actual gray-scale value of the image pixels in the human eye area. When the variance takes the maximum value, t is the optimal threshold, which can ensure that the gray-scale The effect of binarization, so that the calculated gray threshold can better meet the needs of actual image binarization, and make the binarization effect of grayscale image better.

S403: Calculate the average value of the human eye height-to-width ratio in the gray-scale binarized image to obtain the degree of human eye closure.

Here, the traversal method can be used to traverse the upper and lower left and right edges of the human eye to calculate the eye aspect ratio r=Eh/Ew, or the method designed by the technicians can be used to calculate the average eye aspect ratio, which is not limited here. .

As a specific implementation of frowning degree detection in the third embodiment of the application, as shown in FIG. 5, the application

The fifth embodiment includes:

S501: Obtain an image of the eyebrow region in the face image, and detect the pupil distance between the pupil of the eye in the face image and the screen.

In order to recognize the degree of frowning, it is first necessary to locate the eyebrows in the face image. In the embodiment of this application, the position of the user’s eyes is determined based on the position of the eyes of the user, and then the position of the user’s eyes is further located based on the position of the eyes. The brow area between the eyes. Specifically, the forehead area is located according to the position of the human eye, and n% of the forehead area on the side of the human eye is positioned as the eyebrow area, where n can take 50 or other values.

At the same time, the embodiment of the present application will further locate the pupil of the human eye, and use a depth sensor to detect the distance between the pupil and the screen for subsequent comparison. The pupil positioning method can be set by the technician, including but not limited to positioning using a cluster model.

S502: Draw a depth space image corresponding to the image of the eyebrow region, and find the distance between the eyebrow center and the screen and the shortest distance between the eyebrow center and the screen in the depth space image based on the depth space image.

S503: Calculate the distance between the eyebrow center and the distance between the shortest distance and the pupil distance respectively, and calculate the ratio of the two differences. Considering that in actual situations, when the user frowns, the frowning of the eyebrows will cause the eyebrows and the eyebrows to have a more obvious height difference. Therefore, in theory, the degree of frowning can be quantified to a certain extent by comparing the distance between the eyebrow center and the screen and the distance between the eyebrow peak and the screen. However, in actual situations, considering the different situations of different people's frowns, it is difficult to directly calculate the height difference between the center of the eyebrows and the height of the eyebrows. It is difficult to set an effective threshold to ensure recognition accuracy, but the distance of the pupils will not change with the frown. Therefore, this application The embodiment uses the distance between the pupil and the screen as a reference value to quantify the distance between the center of the eyebrow and the peak of the eyebrow and the screen, and finally the ratio of the two calculated differences is used to achieve a certain degree of quantification of frowning behavior.

Specifically, the embodiment of the application first uses the depth sensor to depict the entire eyebrow region image corresponding to the depth space image of the screen, and at the same time locates the eyebrow center, and then finds the eyebrow center distance and the eyebrow peak distance (that is, the shortest distance from the screen). distance). Finally, calculate the difference between the eyebrow center distance and the pupil distance, the difference between the eyebrow peak distance and the pupil distance, and calculate the ratio of the two differences. Among them, the center of the eyebrows can be positioned directly as the center of the eyebrows. The greater the ratio, the higher the degree of frowning.

S504: Identify whether there are wrinkles in the image of the eyebrow region.

S505: Calculate the degree of frown based on the ratio and the result of wrinkle recognition.

Taking into account the actual situation, some people have a large difference between the height of the eyebrow center and the height of the eyebrow peak. Therefore, if the ratio is directly used to calculate the degree of frown, it may lead to misidentification of the frown. On the other hand, in the actual situation, the user is frowning The time is usually accompanied by the appearance of wrinkles. Therefore, the embodiment of the present application will simultaneously identify whether there are wrinkles on the user's eyebrows to assist in determining whether the user's frowning behavior exists and the degree of frowning. Among them, the specific wrinkle recognition method can be selected by the technicians, including but not limited to facial skin wrinkle recognition using Gabor filter and BP neural network, wrinkle recognition based on skin luster (ie, light reflection) can also be used.

If it is recognized that there is a wrinkle in the user's eyebrow area, the embodiment of the present application will determine that the user has a frowning behavior, and will quantify the degree of the user's wrinkle based on the ratio and one or more preset thresholds.

As the sixth embodiment of this application, as shown in Figure 6, it includes:

S601: If the myopia level search is successful, match the corresponding display parameter set according to the found myopia level, and adjust the display parameters of the screen based on the obtained display parameter set.

If the myopia level corresponding to the user identity information is found, this embodiment of the application will directly use the display parameter set corresponding to the myopia level to adjust the screen. For the specific adjustment principle and implementation principle, please refer to the related description of the first embodiment of this application. I will not repeat it here.

As the seventh embodiment of the present application, in order to realize the pre-associated storage of user identity information and the corresponding myopia level, the embodiment of the present application will perform a myopia level test on the user in advance, as shown in FIG. 7, including:

S701: The control screen simultaneously displays multiple preset patterns with the same attributes except for colors, where each preset pattern uniquely corresponds to one color, and the colors corresponding to the multiple preset patterns include at least red, green, and yellow.

S702: Output a pattern definition adjustment prompt to the user, so that the user inputs a corresponding definition adjustment instruction according to the definition of the preset pattern to perform definition adjustment, until the user has the same definition according to the plurality of patterns viewed.

S703: Receive a sharpness adjustment instruction for a preset pattern input by the user, and based on the instructions for the red, green, and yellow preset patterns in the sharpness adjustment instruction, identify the user's myopia level, and compare the identified myopia level with The user's identity information is stored in association.

White light is a composite light composed of seven colors of red, orange, yellow, green, cyan, blue, and purple. The wavelength and refractive index of each color light are different, so white light will occur after passing through a denser medium. Dispersion. In visible light, red light has the longest wavelength, the smallest refractive index, and the fastest speed, while the purple light has the shortest wavelength, the largest refractive index, and the slowest speed. Emmetropia can make the focus of yellow light fall on the retina, then the focus of red light falls behind the retina, and the focus of green light falls in front of the retina. The distance between red light and green light is basically equal to the retina, so red light and green light The diameter of the circle of confusion formed on the retina is basically the same, so the clarity of the optotype in the red and green background is basically the same.

Therefore, after correcting the refractive error, if the spherical degree is appropriate, the patient will feel that the red and green visual targets have the same clarity. If the subject is nearsighted (ie undercorrected nearsightedness or overcorrected for hyperopia), the focus of yellow light will fall in front of the retina, and the focus of red light will be behind the yellow light, so it is closer to the retina, while the focus of green light The yellow light is more forward and therefore farther away from the retina. Therefore, the diameter of the red light forming a circle of confusion on the retina is smaller than that of the green light, so the subject will feel that the visual target in the red background is clearer. Therefore, during the red-green test, if the user feels that the red vision mark is clearer, he needs to increase the nearsightedness or reduce the farsightedness to make the red and green visions equally clear.

If the subject is in a state of hyperopia (that is, when myopia is overcorrected or hyperopia is undercorrected), the focus of yellow light will fall behind the retina, and the focus of red light will be behind the yellow light, so it is farther away from the retina, while the focus of green light The yellow light is more forward, and therefore closer to the retina, so the diameter of the circle of dispersion formed by the green light on the retina is smaller than that of the red light, so the subject will feel that the visual target in the green background is clearer. Therefore, during the red and green test, if the user feels that the green vision mark is clearer, he needs to reduce the nearsightedness or increase the farsightedness to make the red and green sights equally clear.

Based on the above principles, in the embodiments of the present application, multiple color patterns with the same definition will be displayed at the same time, and the user will adjust the definition according to the actual definition he sees. The definition of all color patterns is the same, and the user's myopia level is quantified according to the actual adjustment degree of the red and green patterns and multiple preset thresholds.

In the embodiment of the present application, the test method for myopia level can be integrated into the system setting function of the terminal device. After the user completes the user registration or the user logs in, the user is prompted to perform the test to improve the effectiveness of the test.

As an embodiment of the present application, in order to ensure the viewing effect of the real-time user’s screen and effectively alleviate eye fatigue, the embodiment of the present application will update the user’s identity information in a preset period, and after discovering that the real-time user has changed, the screen will be updated again. Display parameter adjustment, including:

Update the identity information in a preset period and determine whether the identity information has changed before and after the update.

If the identity information changes, based on the updated identity information, return to perform the operation of finding the user's myopia level based on the identity information.

Among them, the specific value of the preset period can be set by the technician himself, preferably, it can be set to 5 minutes. For the method of reacquiring the identity information, please refer to the method of obtaining identity information in the first embodiment of this application, which will not be repeated here. If the updated identity information is different from the identity information before the update, it is determined that the identity information has changed.

After the identity information changes, the embodiment of this application will directly return to S102 of the first embodiment of this application to process the new identity information so as to adjust the screen display of the new user in time.

Corresponding to the method of the above embodiment, FIG. 8 shows a structural block diagram of a screen display device provided in an embodiment of the present application. For ease of description, only parts related to the embodiment of the present application are shown. The screen display device illustrated in FIG. 6 may be the execution subject of the screen display method provided in the first embodiment.

Referring to Figure 8, the screen display device includes:

The information obtaining module 81 is used to obtain the user's identity information.

The level search module 82 is configured to search for the myopia level of the user based on the identity information.

The level recognition module 83 is configured to obtain the face image of the user if the search for the myopia level fails, and identify the myopia level of the user based on the face image.

The display adjustment module 84 is configured to match a corresponding display parameter set based on the user's myopia level, and adjust the display parameters of the screen based on the obtained display parameter set.

Further, the level identification module 83 includes:

If the face image includes glasses, obtaining size information of the screen and detecting the distance between the user's eyes and the screen.

Based on the size information and the human eye distance, the myopia level of the user is identified.

Further, the level identification module 84 includes:

The data detection module is configured to detect, based on the face image, the distance between the user’s eyes and the screen, the degree of closure of the user’s eyes, and the user’s Degree of frown.

The myopia recognition module is configured to recognize the myopia level of the user based on the distance of the human eyes, the degree of closure of the human eyes, and the degree of frowning.

Further, the myopia recognition module includes:

Perform grayscale processing on the human eye region image in the face image.

Calculate the grayscale threshold t based on the following formula, and perform binarization processing on the grayscale image based on the grayscale threshold t:

δ(t) ² =n ₁ (1-n ₁ )(r ₁ -r ₂ ) ²

[Corrected according to Rule 26 13.05.2020]

Calculate the average value of the human eye aspect ratio in the gray-scale binarized image to obtain the human eye closure.

Further, the data detection module includes:

Obtain an image of the eyebrow region in the face image, and detect the pupil distance between the pupil of the eye in the face image and the screen.

Draw a depth space image corresponding to the image of the eyebrow region, and find the distance between the center of the eyebrow and the center of the eyebrow of the screen based on the depth space image, and the shortest distance between the center of the eyebrow and the screen in the depth space image.

Calculate the distance between the center of the eyebrow and the distance difference between the shortest distance and the pupil distance respectively, and calculate the ratio of the two differences.

Identify whether there are wrinkles in the eyebrow region image.

Based on the ratio and the result of wrinkle recognition, the degree of frown is calculated.

Further, the screen display device further includes:

If the myopia level search is successful, the corresponding display parameter set is matched according to the found myopia level, and the display parameter set of the screen is adjusted based on the obtained display parameter set.

Further, the screen display device further includes:

Control the screen to simultaneously display multiple preset patterns with the same attributes except for the color, where each preset pattern uniquely corresponds to one color, and the colors corresponding to the multiple preset patterns include at least Red, green and yellow.

The pattern definition adjustment prompt is output to the user, so that the user inputs the corresponding definition adjustment instruction according to the definition of the preset pattern to adjust the definition, until the user adjusts the definition according to the plurality of images viewed. The pattern definitions are the same.

Receive the sharpness adjustment instruction for the preset pattern input by the user, and identify the user's myopia based on the instruction for the red, green, and yellow preset patterns in the sharpness adjustment instruction And store the identified myopia level in association with the user’s identity information.

Further, the screen display device further includes:

Update the identity information in a preset period, and determine whether the identity information changes before and after the update.

If the identity information changes, based on the updated identity information, return to perform the operation of searching the user's myopia level based on the identity information.

For the process of implementing respective functions of each module in the screen display device provided in the embodiment of the present application, please refer to the description of the first embodiment shown in FIG. 1 for details, which will not be repeated here.

It should be understood that the size of the sequence number of each step in the foregoing embodiment does not mean the order of execution. The execution sequence of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiment of the present application.

It should also be understood that although the terms “first”, “second”, etc. are used in the text in some embodiments of the present application to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another. For example, the first table may be named the second table, and similarly, the second table may be named the first table without departing from the scope of the various described embodiments. The first form and the second form are both forms, but they are not the same form.

FIG. 9 is a schematic diagram of a terminal device provided by an embodiment of the present application. As shown in FIG. 9, the terminal device 9 of this embodiment includes: a processor 90 and a memory 91. The memory 91 stores a computer program 92 that can run on the processor 90. When the processor 90 executes the computer program 92, the steps in the above embodiments of the screen display method are implemented, for example, steps 101 to 108 shown in FIG. 1. Alternatively, when the processor 90 executes the computer program 92, the functions of the modules/units in the foregoing device embodiments, for example, the functions of the modules 81 to 84 shown in FIG. 8 are realized.

The terminal device 9 may be a computing device such as a desktop computer, a notebook, a palmtop computer, and a cloud server. The terminal device may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art can understand that FIG. 9 is only an example of the terminal device 9 and does not constitute a limitation on the terminal device 9. It may include more or less components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device may also include an input sending device, a network access device, a bus, and the like.

The so-called processor 90 may be a central processing unit (Central Processing Unit, CPU), other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Ready-made programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor or the like.

The memory 91 may be an internal storage unit of the terminal device 9, such as a hard disk or memory of the terminal device 9. The memory 91 may also be an external storage device of the terminal device 9, such as a plug-in hard disk equipped on the terminal device 9, a smart memory card (Smart Media Card, SMC), and a Secure Digital (SD) Card, Flash Card, etc. Further, the memory 91 may also include both an internal storage unit of the terminal device 9 and an external storage device. The memory 91 is used to store the computer program and other programs and data required by the terminal device. The memory 91 can also be used to temporarily store data that has been sent or will be sent.

In addition, the functional units in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated module/unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, this application implements all or part of the processes in the above-mentioned embodiments and methods, and can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the program is executed by the processor, the steps of the foregoing method embodiments can be implemented. Wherein, the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file, or some intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (Read-Only Memory, ROM) , Random Access Memory (RAM), electrical carrier signal, telecommunications signal, and software distribution media.

In one embodiment, this application also proposes a computer-readable storage medium that is stored with a volatile storage medium or a non-volatile storage medium, and the computer-readable storage medium is stored by one or more When the processor is executed, one or more processors are caused to perform the following steps: obtain the user's identity information; search for the user's myopia level based on the identity information; if the myopia level search fails, obtain the user's face image, And recognize the myopia level of the user based on the face image; match the corresponding display parameter set based on the myopia level of the user, and adjust the display parameters of the screen based on the obtained display parameter set.

Claims

A screen display method, which includes:

Obtain user's identity information;

Searching for the myopia level of the user based on the identity information;

If the search for myopia level fails, acquiring a face image of the user, and identifying the myopia level of the user based on the face image;

Match the corresponding display parameter set based on the myopia level of the user, and adjust the display parameter of the screen based on the obtained display parameter set.
8. The screen display method of claim 1, wherein the recognizing the myopia level of the user based on the face image comprises:

If the face image includes glasses, acquiring size information of the screen and detecting the distance between the user's eyes and the screen;

Based on the size information and the human eye distance, the myopia level of the user is identified.
8. The screen display method of claim 1, wherein the identifying the myopia level of the user based on the face image further comprises:

If the face image does not include glasses, detecting the distance between the user's eyes and the screen, the degree of closure of the user's eyes, and the degree of frowning of the user based on the face image;

Identify the myopia level of the user based on the distance of the human eye, the degree of closure of the human eye, and the degree of frowning.
[Corrected according to Rule 26 13.05.2020]
8. The screen display method of claim 3, wherein the calculating the user's eye closure based on the face image comprises:
Performing grayscale processing on the image of the human eye area in the face image;
Calculate the grayscale threshold t based on the following formula, and perform binarization processing on the grayscale image based on the grayscale threshold t:
δ(t) 2 =n 1 (1-n 1 ) (r 1 -r 2 ) 2

Among them, N is the total number of pixels in the grayscale image, n i represents the number of pixels with grayscale value i, k is the maximum grayscale value of the pixels in the grayscale image, t∈[0,k] , Δ(t)2 is the variance of t;
Calculate the average value of the human eye aspect ratio in the gray-scale binarized image to obtain the human eye closure.
8. The screen display method of claim 3, wherein the calculating the degree of frown of the user based on the face image comprises:

Acquiring an image of the eyebrow region in the face image, and detecting the pupil distance between the pupil of the eye in the face image and the screen;

Drawing a depth space image corresponding to the image of the eyebrow region, and finding the distance between the center of the eyebrow and the center of the eyebrow of the screen based on the depth space image, and the shortest distance between the center of the eyebrow and the screen in the depth space image;

Calculating the distance between the eyebrow center and the distance difference between the shortest distance and the pupil distance respectively, and calculating the ratio of the two differences;

Identifying whether there are wrinkles in the image of the brow region;

Based on the ratio and the result of wrinkle recognition, the degree of frown is calculated.
The screen display method of claim 1, further comprising:

If the myopia level search is successful, the corresponding display parameter set is matched according to the found myopia level, and the display parameter set of the screen is adjusted based on the obtained display parameter set.
The screen display method according to any one of claims 1 to 6, wherein before said obtaining the user's identity information, it further comprises:

Control the screen to simultaneously display multiple preset patterns with the same attributes except for the color, where each preset pattern uniquely corresponds to one color, and the colors corresponding to the multiple preset patterns include at least Red, green and yellow;

The pattern definition adjustment prompt is output to the user, so that the user inputs the corresponding definition adjustment instruction according to the definition of viewing the preset pattern to adjust the definition until the user sees multiple sheets of the The pattern definition is the same;

Receive the sharpness adjustment instruction for the preset pattern input by the user, and identify the user's myopia based on the instruction for the red, green, and yellow preset patterns in the sharpness adjustment instruction And store the identified myopia level in association with the user’s identity information.
7. The screen display method according to any one of claims 1 to 6, wherein after said adjusting the display parameters of the screen based on the obtained display parameter set, the method further comprises:

Update the identity information in a preset period, and determine whether the identity information changes before and after the update;

If the identity information changes, based on the updated identity information, return to perform the operation of finding the user's myopia level based on the identity information.
A terminal device, wherein the terminal device includes a memory and a processor, the memory stores a computer program that can run on the processor, and the processor implements a screen display when the computer program is executed Method, the screen display method includes the following steps:

Obtain the user's identity information; search for the user's myopia level based on the identity information; if the myopia level search fails, obtain the user's face image, and identify the user's myopia level based on the face image; The user's myopia level matches the corresponding display parameter set, and the screen is adjusted based on the obtained display parameter set.
The terminal device of claim 9, wherein the recognizing the myopia level of the user based on the face image comprises: if the face image contains glasses, obtaining size information of the screen and detecting all The human eye distance between the user’s eyes and the screen; and based on the size information and the human eye distance, the myopia level of the user is identified.
The terminal device according to claim 9, wherein the recognizing the myopia level of the user based on the face image further comprises: if the face image does not include glasses, detecting all based on the face image The distance between the user’s eyes and the screen, the degree of eye closure of the user, and the degree of frowning of the user; the user is identified based on the distance of the human eye, the degree of eye closure, and the degree of frowning Myopia level.
[Corrected according to Rule 26 13.05.2020]
The terminal device of claim 11, wherein the calculating the user's eye closure based on the face image comprises:
Performing grayscale processing on the image of the human eye area in the face image;
Calculate the grayscale threshold t based on the following formula, and perform binarization processing on the grayscale image based on the grayscale threshold t:
δ(t) 2 =n 1 (1-n 1 ) (r 1 -r 2 ) 2

Among them, N is the total number of pixels in the grayscale image, n i represents the number of pixels with grayscale value i, k is the maximum grayscale value of the pixels in the grayscale image, t∈[0,k] , Δ(t)2 is the variance of t; the average value of the human eye aspect ratio in the gray-scale binarized image is calculated to obtain the human eye closure.
The terminal device of claim 11, wherein the calculating the degree of frown of the user based on the face image comprises:

Acquire the eyebrow region image in the face image, and detect the pupil distance between the human eye pupil and the screen in the face image; draw the depth space image corresponding to the eyebrow region image, and based on the depth The space image searches for the distance between the eyebrow center and the screen and the shortest distance between the eyebrow center and the screen in the depth space image; respectively calculate the distance between the eyebrow center and the distance between the shortest distance and the pupil distance, and calculate The ratio of the two differences; identifying whether there are wrinkles in the image of the eyebrow region; and calculating the degree of frown based on the ratio and the result of wrinkle recognition.
The terminal device according to claim 9, further comprising:

If the myopia level search is successful, the corresponding display parameter set is matched according to the found myopia level, and the display parameter set of the screen is adjusted based on the obtained display parameter set.
The terminal device according to any one of claims 9 to 14, wherein before said obtaining the user’s identity information, it further comprises: controlling the screen to simultaneously display multiple preset patterns with the same attributes except for colors, Wherein, each of the preset patterns uniquely corresponds to one color, and the colors corresponding to the plurality of preset patterns include at least red, green, and yellow; the pattern definition adjustment prompt is output to the user, so that all The user adjusts the definition according to the definition input corresponding to the definition of the preset pattern until the definition of the plurality of patterns viewed by the user is the same; The sharpness adjustment instruction of the preset pattern, and based on the instructions for the red, green, and yellow preset patterns in the sharpness adjustment instruction, identify the myopia level of the user, and then identify the identified myopia The rank is stored in association with the user's identity information;

Wherein, after the adjustment of the display parameters of the screen based on the obtained display parameter set, the method further includes: updating the identity information in a preset period, and determining whether the identity information has changed before and after the update; if the identity information occurs Change, based on the updated identity information, returning to perform the operation of searching the user's myopia level based on the identity information.
A computer-readable storage medium, the computer-readable storage medium stores a computer program, wherein the computer program is executed by a processor to implement a screen display method, the screen display method includes the following steps: Identity information; find the myopia level of the user based on the identity information; if the myopia level search fails, obtain the face image of the user, and identify the myopia level of the user based on the face image; based on the user The myopia level matches the corresponding display parameter set, and adjusts the display parameters of the screen based on the obtained display parameter set.
The computer-readable storage medium according to claim 16, wherein the identifying the myopia level of the user based on the face image comprises: if the face image contains glasses, obtaining size information of the screen And detecting the human eye distance between the user's eyes and the screen; based on the size information and the human eye distance, identifying the myopia level of the user.
The computer-readable storage medium of claim 16, wherein the identifying the myopia level of the user based on the face image further comprises: if the face image does not contain glasses, based on the face image The image detects the distance between the user’s eyes and the screen, the degree of closure of the user’s eyes, and the degree of frowning of the user; recognition based on the distance of the human’s eyes, the degree of closure of the human eye, and the degree of frowning The myopia level of the user.
[Corrected according to Rule 26 13.05.2020]
18. The computer-readable storage medium of claim 18, wherein the calculating the user's eye closure based on the face image comprises:
Performing grayscale processing on the image of the human eye area in the face image;
Calculate the grayscale threshold t based on the following formula, and perform binarization processing on the grayscale image based on the grayscale threshold t:
δ(t) 2 =n 1 (1-n 1 ) (r 1 -r 2 ) 2

Among them, N is the total number of pixels in the grayscale image, n i represents the number of pixels with grayscale value i, k is the maximum grayscale value of the pixels in the grayscale image, t∈[0,k] , Δ(t)2 is the variance of t; calculate the average value of the human eye aspect ratio in the gray-scale binarized image to obtain the human eye closure.
18. The computer-readable storage medium of claim 18, wherein the calculating the degree of frown of the user based on the face image comprises:

Acquire the eyebrow region image in the face image, and detect the pupil distance between the human eye pupil and the screen in the face image; draw the depth space image corresponding to the eyebrow region image, and based on the depth The space image searches for the distance between the eyebrow center and the screen and the shortest distance between the eyebrow center and the screen in the depth space image; respectively calculate the distance between the eyebrow center and the distance between the shortest distance and the pupil distance, and calculate The ratio of the two differences; identifying whether there are wrinkles in the image of the eyebrow region; and calculating the degree of frown based on the ratio and the result of wrinkle recognition.
The computer-readable storage medium of claim 16, further comprising:

If the myopia level search is successful, the corresponding display parameter set is matched according to the found myopia level, and the display parameter set of the screen is adjusted based on the obtained display parameter set.
The computer-readable storage medium according to any one of claims 16 to 21, wherein before said obtaining the user’s identity information, it further comprises: controlling the screen to simultaneously display multiple presets with the same attributes except for colors. Set patterns, wherein each of the preset patterns uniquely corresponds to one color, and the colors corresponding to the plurality of preset patterns include at least red, green, and yellow; output a pattern definition adjustment prompt to the user, So that the user can input the corresponding definition adjustment instruction according to the definition of the preset pattern to adjust the definition until the definition of the plurality of patterns viewed by the user is the same; receive the user input The sharpness adjustment instruction for the preset pattern, and based on the instructions for the red, green, and yellow preset patterns in the sharpness adjustment instruction, identify the myopia level of the user, and identify The out-of-sight myopia level is stored in association with the user’s identity information;

Wherein, after the adjustment of the display parameters of the screen based on the obtained display parameter set, the method further includes: updating the identity information in a preset period, and determining whether the identity information has changed before and after the update; if the identity information occurs Change, based on the updated identity information, returning to perform the operation of searching the user's myopia level based on the identity information.