CN109756726B - Display device, display method thereof and virtual reality equipment - Google Patents
Display device, display method thereof and virtual reality equipment Download PDFInfo
- Publication number
- CN109756726B CN109756726B CN201910106916.8A CN201910106916A CN109756726B CN 109756726 B CN109756726 B CN 109756726B CN 201910106916 A CN201910106916 A CN 201910106916A CN 109756726 B CN109756726 B CN 109756726B
- Authority
- CN
- China
- Prior art keywords
- display
- focal planes
- focal
- determining
- saliency
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Abstract
The invention provides a display device, a display method thereof and virtual reality equipment, and belongs to the technical field of display. A display method applied to a display device, the display device comprising a display and a variable focal lens, the variable focal lens being capable of adjusting a position of a focal plane, the display method comprising: determining the number n of focal planes according to the display frequency of the display and the refreshing frequency of a display picture; detecting a current frame display picture, and determining m significant areas in the current frame display picture; and determining the positions of n focal planes according to the spatial depths of the m salient regions, distributing the m salient regions on the n focal planes for display, distributing each salient region on only one corresponding focal plane, and matching the spatial depth of the salient region with the position of the corresponding focal plane. The technical scheme of the invention can solve convergence conflict and solve the problems of discontinuous image depth and fuzzy image caused by fixing a plurality of focal planes.
Description
Technical Field
The present invention relates to the field of display technologies, and in particular, to a display device, a display method thereof, and a virtual reality device.
Background
In an existing VR (virtual reality) device, there is a problem of convergence conflict. Normally, both monocular focusing and binocular focusing are required for a human eye to clearly see a target object, but when a three-dimensional scene is presented by using an existing two-dimensional display, convergence conflict is caused due to incorrect monocular focusing, and multi-focal-plane display is used for solving the problem. In the existing multi-focal-plane display VR device, the configuration method of the multiple focal planes is: the focal plane is uniformly placed at a fixed position in diopters. This approach, while striving to take the entire scenario into account, may cause problems: a target object is rendered at a different focal plane, resulting in a perception of severe depth discontinuity; or that the key target object does not cause image blur on the focal plane.
Disclosure of Invention
The invention aims to provide a display device, a display method thereof and virtual reality equipment, which can solve convergence conflict and solve the problems of discontinuous image depth and fuzzy image caused by fixing a plurality of focal planes.
To solve the above technical problem, embodiments of the present invention provide the following technical solutions:
in one aspect, a display method is provided, which is applied to a display device, where the display device includes a display and a variable focus lens located on a light exit side of the display, and a focal plane of the variable focus lens is adjustable, and the display method includes:
determining the number n of focal planes according to the display frequency of the display and the refreshing frequency of a display picture;
detecting a current frame display picture, and determining m significant areas in the current frame display picture;
determining the positions of n focal planes according to the spatial depths of the m salient regions, and distributing the m salient regions on the n focal planes for display, wherein each salient region is only distributed on one corresponding focal plane, and the spatial depth of the salient region is matched with the position of the corresponding focal plane;
wherein m and n are positive integers.
Further, the determining the number n of focal planes according to the display frequency of the display and the refresh frequency of the display screen includes:
n is calculated using the following formula: and n is A/B, wherein A is the display frequency of the display, and B is the refresh frequency of the display screen.
Further, the detecting the current frame display image and determining m salient regions in the current frame display image includes:
determining three features of an image, the three features including a brightness, a color, and a direction of the image;
decomposing a current frame display picture into 9 layers of Gaussian pyramids to form a multi-scale image;
sequentially filtering the images of all scales by using a function filter, obtaining a feature map of each feature by applying a central-peripheral operator algorithm, and obtaining a saliency map of three features through cross-scale combination and normalization calculation;
and performing linear fusion on the three obtained saliency maps to obtain a total saliency map, and obtaining m saliency areas by a neural network winner totaling method.
Further, the determining the positions of the n focal planes according to the spatial depths of the m salient regions and allocating the m salient regions to the n focal planes for display includes:
dividing the display time of a frame of picture into n time periods;
and determining the positions of the n focal planes according to the spatial depths of the m saliency areas, controlling the variable focus lens to adjust the focal plane of the human eye through the variable focus lens to a k focal plane located at a k position in a k time period, and controlling the display to display the saliency areas located on the k focal plane, wherein k is a positive integer not greater than n.
Further, the determining the positions of the n focal planes according to the m spatial depths of the salient regions comprises:
when m is larger than or equal to n, determining the positions of n focal planes according to the spatial depths of n salient regions in the m salient regions, wherein the position of each focal plane is matched with the spatial depth of the corresponding salient region, and distributing the remaining m-n salient regions to the n focal planes according to the spatial depths of the remaining m-n salient regions;
when m is less than n, determining the positions of m focal planes according to the spatial depths of the m salient regions, and controlling the rest n-m focal planes to coincide with any one or more of the m focal planes.
The embodiment of the present invention further provides a display device, including a display and a variable focal length lens, where a focal plane of the variable focal length lens is adjustable, and the display device further includes:
the calculation module is used for determining the number n of focal planes according to the display frequency of the display and the refreshing frequency of the display picture;
the detection module is used for detecting the display picture of the current frame and determining m significant areas in the display picture of the current frame;
the processing module is used for determining the positions of n focal planes according to the spatial depths of the m salient regions, distributing the m salient regions on the n focal planes for display, distributing each salient region on only one corresponding focal plane, and matching the spatial depth of the salient region with the position of the corresponding focal plane;
wherein m and n are positive integers.
Further, the calculating module is specifically configured to calculate n by using the following formula: and n is A/B, wherein A is the display frequency of the display, and B is the refresh frequency of the display screen.
Further, the processing module is specifically configured to divide the display time of one frame of picture into n time periods;
and determining the positions of the n focal planes according to the spatial depths of the m saliency areas, controlling the variable focus lens to adjust the focal plane of the human eye through the variable focus lens to a k focal plane located at a k position in a k time period, and controlling the display to display the saliency areas located on the k focal plane, wherein k is a positive integer not greater than n.
The embodiment of the invention also provides virtual reality equipment which comprises the display device.
An embodiment of the present invention further provides a display device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps in the display method as described above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the display method are implemented as described above.
The embodiment of the invention has the following beneficial effects:
in the scheme, the number n of focal planes is determined according to the display frequency of a display and the refreshing frequency of a display picture; detecting a current frame display picture, and determining m significant areas in the current frame display picture; the positions of n focal planes are determined according to the spatial depths of m saliency areas, the m saliency areas are distributed on the n focal planes for display, each saliency area is only distributed on one corresponding focal plane, the spatial depth of each saliency area is matched with the position of the corresponding focal plane, therefore, several target objects which are most attractive to the attention of an observer in a scene of a display screen can be extracted, the focal planes are configured according to the spatial depths of the several target objects in the scene, and images of the focal planes are rendered. The display method can solve the problem of depth discontinuity caused by the fact that the same target object is rendered on different focal planes due to the configuration of the uniform focal planes, or the problem of blurring caused by the fact that the target object is not on the focal planes.
Drawings
FIG. 1 is a schematic view of a convergence conflict;
FIG. 2 is a schematic diagram of a target object being rendered at different focal planes;
FIG. 3 is a schematic diagram of a key target object not in the focal plane;
FIG. 4 is a flow chart illustrating a display method according to an embodiment of the present invention;
FIG. 5 is a model block diagram of the Itti algorithm;
FIG. 6 is a schematic diagram of a saliency map obtained from an image via an Itti model;
FIG. 7 is a schematic view of a display device according to an embodiment of the present invention;
FIG. 8 is a block diagram of a display device according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantages to be solved by the embodiments of the present invention clearer, the following detailed description will be given with reference to the accompanying drawings and specific embodiments.
In the existing VR device, there is a problem of convergence conflict. As shown in fig. 1, both monocular focusing and binocular focusing are required for a human eye to clearly see a target object under normal conditions, but when a three-dimensional scene is presented using an existing two-dimensional display, convergence conflict is caused due to incorrect monocular focusing, and multi-focal-plane display is used to solve this problem. In the existing multi-focal-plane display VR device, the configuration method of the multiple focal planes is: the focal plane is uniformly placed at a fixed position in diopters. This approach, while striving to take the entire scenario into account, may cause problems: a target object is rendered at a different focal plane (as shown in fig. 2), resulting in a severe sense of depth discontinuity; or the key target object does not cause image blur on the focal plane (as shown in fig. 3), where 1 is human eye, 2 is variable focus lens, 3 is display, 4 is first focal plane, 5 is second focal plane, and 6 is third focal plane.
The embodiment of the invention provides a display device, a display method thereof and virtual reality equipment, aiming at the problems of discontinuous image depth and fuzzy image generated by fixing a plurality of focal planes in the existing multi-focal plane display technology, and can solve convergence conflict and solve the problems of discontinuous image depth and fuzzy image generated by fixing a plurality of focal planes.
An embodiment of the present invention provides a display method applied to a display device, where the display device includes a display and a variable focus lens located on a light exit side of the display, and a focal plane of the variable focus lens is adjustable, as shown in fig. 4, the display method includes:
step 101: determining the number n of focal planes according to the display frequency of the display and the refreshing frequency of a display picture;
wherein, the display frequency of the display is an integral multiple of the refresh frequency (generally 60Hz) of the display picture;
step 102: detecting a current frame display picture, and determining m significant areas in the current frame display picture;
step 103: determining the positions of n focal planes according to the spatial depths of the m salient regions, and distributing the m salient regions on the n focal planes for display, wherein each salient region is only distributed on one corresponding focal plane, and the spatial depth of the salient region is matched with the position of the corresponding focal plane;
wherein m and n are positive integers.
In the embodiment, the number n of focal planes is determined according to the display frequency of the display and the refreshing frequency of the display picture; detecting a current frame display picture, and determining m significant areas in the current frame display picture; the positions of n focal planes are determined according to the spatial depths of m saliency areas, the m saliency areas are distributed on the n focal planes for display, each saliency area is only distributed on one corresponding focal plane, the spatial depth of each saliency area is matched with the position of the corresponding focal plane, therefore, several target objects which are most attractive to the attention of an observer in a scene of a display screen can be extracted, the focal planes are configured according to the spatial depths of the several target objects in the scene, and images of the focal planes are rendered. The display method can solve the problem of depth discontinuity caused by the fact that the same target object is rendered on different focal planes due to the configuration of the uniform focal planes, or the problem of blurring caused by the fact that the target object is not on the focal planes.
Wherein, the spatial depth of the saliency region is the spatial depth of the central point of the saliency region.
Wherein, the display device can be: the display device comprises a television, a display, a digital photo frame, a mobile phone, a tablet personal computer and any other product or component with a display function, wherein the display device further comprises a flexible circuit board, a printed circuit board and a back plate.
Further, the determining the number n of focal planes according to the display frequency of the display and the refresh frequency of the display screen includes:
n is calculated using the following formula: and n is A/B, wherein A is the display frequency of the display, and B is the refresh frequency of the display screen.
Further, the detecting the current frame display image and determining m salient regions in the current frame display image includes:
determining three features of an image, the three features including a brightness, a color, and a direction of the image;
decomposing a current frame display picture into 9 layers of Gaussian pyramids to form a multi-scale image;
sequentially filtering the images of all scales by using a function filter, obtaining a feature map of each feature by applying a central-peripheral operator algorithm, and obtaining a saliency map of three features through cross-scale combination and normalization calculation;
and performing linear fusion on the three obtained saliency maps to obtain a total saliency map, and obtaining m saliency areas by a neural network winner totaling method.
Further, the determining the positions of the n focal planes according to the spatial depths of the m salient regions and allocating the m salient regions to the n focal planes for display includes:
dividing the display time of a frame of picture into n time periods;
and determining the positions of the n focal planes according to the spatial depths of the m saliency areas, controlling the variable focus lens to adjust the focal plane of the human eye through the variable focus lens to a k focal plane located at a k position in a k time period, and controlling the display to display the saliency areas located on the k focal plane, wherein k is a positive integer not greater than n.
Further, the determining the positions of the n focal planes according to the m spatial depths of the salient regions comprises:
when m is larger than or equal to n, determining the positions of n focal planes according to the spatial depths of n salient regions in the m salient regions, wherein the position of each focal plane is matched with the spatial depth of the corresponding salient region, and distributing the remaining m-n salient regions to the n focal planes according to the spatial depths of the remaining m-n salient regions;
when m is less than n, determining the positions of m focal planes according to the spatial depths of the m salient regions, and controlling the rest n-m focal planes to coincide with any one or more of the m focal planes.
If the spatial depths of two or more salient regions are the same, the focal plane positions determined according to the spatial depths of the two or more salient regions are the same, namely the focal planes determined according to the spatial depths of the two or more salient regions are coincident, and the two or more salient regions are positioned on the same focal plane.
The saliency region is allocated to the focal plane or is located on the focal plane, that is, when the display displays the saliency region, the variable focus lens is controlled to adjust the focal length to the focal length corresponding to the focal plane.
If the spatial depths of s saliency regions in the m saliency regions are different from each other, and the value of s is greater than n, n focal planes can be determined according to the spatial depths of n saliency regions in the s saliency regions, and then the remaining saliency regions are allocated to the n focal planes for display according to the spatial depths of the remaining s-n saliency regions. Further, when n focal planes are determined according to the spatial depths of n salient regions of the s salient regions, the priority order of the s salient regions may be determined in advance, n salient regions are selected from the s salient regions according to the priorities from high to low, and the positions of the n focal planes are determined according to the n salient regions.
For example, if 3 salient regions are detected in total, and the first salient region, the second salient region and the third salient region are respectively a first salient region, a second salient region and a third salient region from high to low according to the priority, the spatial depths of the first salient region and the third salient region are the same, the spatial depths of the first salient region and the second salient region are different, and the number of focusing surfaces is 2, the position of the focusing surface is determined according to the spatial depths of the first salient region and the second salient region, and the third salient region is allocated to the focusing surface corresponding to the first salient region for display. For another example, a total of 3 saliency areas are detected, the first saliency area, the second saliency area and the third saliency area are respectively arranged from high to low according to the priority, the spatial depths of the 3 saliency areas are different, but the number of the focus-adjustable surfaces is 2, the position of the focus surface is determined according to the spatial depths of the first saliency area and the second saliency area, and the third saliency area is allocated to the focus surface with the highest spatial depth matching degree with the third saliency area for display.
The technical scheme of the invention is further described by combining the drawings and specific embodiments:
for one image, the user only has interest in a part of the region in the image, the part of the region of interest represents the query intention of the user, and most of the rest of the regions without interest are irrelevant to the query intention of the user. The saliency area is the area of the image that is most interesting to the user and most representative of the content of the image. A common approach is to calculate the saliency of the image based on the human attention mechanism. The research of cognitive psychology shows that some regions in images can attract attention remarkably, the regions contain larger information, cognitive scientists have proposed a plurality of mathematical models to simulate the attention mechanism of people, and the saliency regions extracted in the way are more consistent with the subjective evaluation of people due to the utilization of the general rules in the image cognition process.
In particular, regions of saliency in an image may be determined using an Itti model, which is divided into feature extraction and saliency map generation. FIG. 5 is a model block diagram of the Itti algorithm. The model selects the attributes of 3 lower layers as the characteristics of the image, namely the brightness, the color and the direction of the image. The whole process can be described as follows, firstly, an input image is decomposed into 9 layers of Gaussian pyramids to form a multi-scale image, then the image of each scale is sequentially filtered by a function filter, then a central-peripheral operator algorithm is applied to obtain a feature map of each feature, salient maps of three features are obtained through cross-scale combination and normalization calculation, the obtained three salient maps are subjected to linear fusion to obtain a total salient map, and finally, an attention area, namely a salient area, is obtained through a method of taking the whole by a neural network winner.
Taking the number of saliency areas as 2 and the number of focus-adjustable surfaces as 2 as an example, as shown in fig. 6, acquiring a saliency map from an image through an Itti model can obviously identify the saliency areas as two persons, determine the spatial positions of the two persons in a scene in the saliency map, and determine the depths of the central points in the space as D1 and D2, respectively, and configure the focus surfaces at the two depths. When displaying, the focal length of the lens needs to be changed by changing the voltage loaded on the variable focus lens (generally, an electric focus adjustable lens) so as to change the focal length of the lens between two focal lengths D1 and D2 at high frequency. The display synchronously displays the images rendered under the two focal planes with the zoom lens, and particularly, when the spatial depth of the first saliency region is D1, and the spatial depth of the second saliency region is D2, the display time of each frame of display screen is divided into a first time period and a second time period, the display displays the first saliency region in the first time period, and simultaneously controls the focal length of the zoom lens to be D1, and the first saliency region is rendered on the first focal plane; and the display displays the second saliency region in a second time period, simultaneously controls the focal length of the variable-focus lens to be D2, renders the second saliency region on a second focal plane, enables human eyes to perceive images on the two focal planes due to the persistence of vision effect, and forms a display image after superposition and fusion. Wherein the picture contents of other non-salient regions can be distributed to two focal planes for display.
The display frequency of the display determines the number of focal planes, and taking the refresh frequency of the display screen as 60Hz as an example, dividing the display frequency of the display by the refresh frequency of the display screen is the number of focus-adjustable planes, for example, if the number of focus-adjustable planes is 2, the display frequency of the display should be 60 × 2 — 120Hz in order to meet the requirement of human eyes.
As shown in fig. 7, in the display device of the present embodiment, the variable focus lens 8 is driven by the same computer 9 that controls the display 3 to display the image, allowing precise time synchronization between the virtual image distance and the screen content.
An embodiment of the present invention further provides a display device, including a display and a variable focal length lens, where a focal plane of the variable focal length lens is adjustable, as shown in fig. 8, the display device further includes:
the calculation module 21 is configured to determine the number n of focal planes according to the display frequency of the display and the refresh frequency of the display screen;
the detection module 22 is configured to detect a current frame display picture and determine m salient regions in the current frame display picture;
the processing module 23 is configured to determine positions of n focal planes according to the spatial depths of the m saliency areas, allocate the m saliency areas to the n focal planes, and display the m saliency areas, where each saliency area is allocated only to a corresponding one of the focal planes, and the spatial depth of the saliency area matches the position of the corresponding focal plane;
wherein m and n are positive integers.
In the embodiment, the number n of focal planes is determined according to the display frequency of the display and the refreshing frequency of the display picture; detecting a current frame display picture, and determining m significant areas in the current frame display picture; the positions of n focal planes are determined according to the spatial depths of m saliency areas, the m saliency areas are distributed on the n focal planes for display, each saliency area is only distributed on one corresponding focal plane, the spatial depth of each saliency area is matched with the position of the corresponding focal plane, therefore, several target objects which are most attractive to the attention of an observer in a scene of a display screen can be extracted, the focal planes are configured according to the spatial depths of the several target objects in the scene, and images of the focal planes are rendered. The display method can solve the problem of depth discontinuity caused by the fact that the same target object is rendered on different focal planes due to the configuration of the uniform focal planes, or the problem of blurring caused by the fact that the target object is not on the focal planes.
Wherein, the spatial depth of the saliency region is the spatial depth of the central point of the saliency region.
Wherein, the display device can be: the display device comprises a television, a display, a digital photo frame, a mobile phone, a tablet personal computer and any other product or component with a display function, wherein the display device further comprises a flexible circuit board, a printed circuit board and a back plate.
Further, the calculating module is specifically configured to calculate n by using the following formula: and n is A/B, wherein A is the display frequency of the display, and B is the refresh frequency of the display screen.
Further, the processing module is specifically configured to divide the display time of one frame of picture into n time periods;
and determining the positions of the n focal planes according to the spatial depths of the m saliency areas, controlling the variable focus lens to adjust the focal plane of the human eye through the variable focus lens to a k focal plane located at a k position in a k time period, and controlling the display to display the saliency areas located on the k focal plane, wherein k is a positive integer not greater than n.
If the spatial depths of two or more salient regions are the same, the focal plane positions determined according to the spatial depths of the two or more salient regions are the same, namely the focal planes determined according to the spatial depths of the two or more salient regions are coincident, and the two or more salient regions are positioned on the same focal plane.
The saliency region is allocated to the focal plane or is located on the focal plane, that is, when the display displays the saliency region, the variable focus lens is controlled to adjust the focal length to the focal length corresponding to the focal plane.
If the spatial depths of s saliency regions in the m saliency regions are different from each other, and the value of s is greater than n, n focal planes can be determined according to the spatial depths of n saliency regions in the s saliency regions, and then the remaining saliency regions are allocated to the n focal planes for display according to the spatial depths of the remaining s-n saliency regions. Further, when n focal planes are determined according to the spatial depths of n salient regions of the s salient regions, the priority order of the s salient regions may be determined in advance, n salient regions are selected from the s salient regions according to the priorities from high to low, and the positions of the n focal planes are determined according to the n salient regions.
For example, if 3 salient regions are detected in total, and the first salient region, the second salient region and the third salient region are respectively a first salient region, a second salient region and a third salient region from high to low according to the priority, the spatial depths of the first salient region and the third salient region are the same, the spatial depths of the first salient region and the second salient region are different, and the number of focusing surfaces is 2, the position of the focusing surface is determined according to the spatial depths of the first salient region and the second salient region, and the third salient region is allocated to the focusing surface corresponding to the first salient region for display. For another example, a total of 3 saliency areas are detected, the first saliency area, the second saliency area and the third saliency area are respectively arranged from high to low according to the priority, the spatial depths of the 3 saliency areas are different, but the number of the focus-adjustable surfaces is 2, the position of the focus surface is determined according to the spatial depths of the first saliency area and the second saliency area, and the third saliency area is allocated to the focus surface with the highest spatial depth matching degree with the third saliency area for display.
The embodiment of the invention also provides virtual reality equipment which comprises the display device.
An embodiment of the present invention further provides a display device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps in the display method as described above.
An embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the display method are implemented as described above.
It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For a hardware implementation, the Processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units configured to perform the functions described herein, or a combination thereof.
For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.
The embodiments in the present specification are described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments are referred to each other.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, apparatus, or computer program product. Accordingly, embodiments of the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
Embodiments of the present invention are described with reference to flowchart illustrations and/or block diagrams of methods, user equipment (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present invention have been described, additional variations and modifications of these embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the embodiments of the invention.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or user equipment that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or user equipment. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or user equipment that comprises the element.
While the preferred embodiments of the present invention have been described, it will be understood by those skilled in the art that various changes and modifications may be made without departing from the spirit and scope of the invention as defined in the following claims.
Claims (10)
1. A display method is applied to a display device, and is characterized in that the display device comprises a display and a variable focus lens positioned on the light-emitting side of the display, the focal plane of the variable focus lens is adjustable, and the display method comprises the following steps:
determining the number n of focal planes according to the display frequency of the display and the refreshing frequency of a display picture;
detecting a current frame display picture, and determining m significant areas in the current frame display picture;
determining the positions of n focal planes according to the spatial depths of the m salient regions, and distributing the m salient regions on the n focal planes for display, wherein each salient region is only distributed on one corresponding focal plane, and the spatial depth of the salient region is matched with the position of the corresponding focal plane;
wherein m and n are positive integers;
the determining the positions of the n focal planes according to the m significant region spatial depths comprises:
when m is larger than or equal to n, determining the positions of n focal planes according to the spatial depths of n salient regions in the m salient regions, wherein the position of each focal plane is matched with the spatial depth of the corresponding salient region, and distributing the remaining m-n salient regions to the n focal planes according to the spatial depths of the remaining m-n salient regions;
when m is less than n, determining the positions of m focal planes according to the spatial depths of the m salient regions, and controlling the rest n-m focal planes to coincide with any one or more of the m focal planes.
2. The method according to claim 1, wherein the determining the number n of focal planes according to the display frequency of the display and the refresh frequency of the display screen comprises:
n is calculated using the following formula: and n is A/B, wherein A is the display frequency of the display, B is the refresh frequency of the display screen, and the display frequency of the display is integral multiple of the refresh frequency of the display screen.
3. The display method according to claim 1, wherein the detecting the current frame display picture and the determining m salient regions in the current frame display picture comprise:
determining three features of an image, the three features including a brightness, a color, and a direction of the image;
decomposing a current frame display picture into 9 layers of Gaussian pyramids to form a multi-scale image;
sequentially filtering the images of all scales by using a function filter, obtaining a feature map of each feature by applying a central-peripheral operator algorithm, and obtaining a saliency map of three features through cross-scale combination and normalization calculation;
and performing linear fusion on the three obtained saliency maps to obtain a total saliency map, and obtaining m saliency areas by a neural network winner totaling method.
4. The display method according to claim 1, wherein the determining the positions of n focal planes according to the spatial depths of the m salient regions and the allocating the m salient regions to the n focal planes for display comprises:
dividing the display time of a frame of picture into n time periods;
and determining the positions of the n focal planes according to the spatial depths of the m saliency areas, controlling the variable focus lens to adjust the focal plane of the human eye through the variable focus lens to a k focal plane located at a k position in a k time period, and controlling the display to display the saliency areas located on the k focal plane, wherein k is a positive integer not greater than n.
5. A display device comprising a display and a variable focus lens, the focal plane of the variable focus lens being adjustable, the display device further comprising:
the calculation module is used for determining the number n of focal planes according to the display frequency of the display and the refreshing frequency of the display picture;
the detection module is used for detecting the display picture of the current frame and determining m significant areas in the display picture of the current frame;
the processing module is used for determining the positions of n focal planes according to the spatial depths of the m salient regions, distributing the m salient regions on the n focal planes for display, distributing each salient region on only one corresponding focal plane, and matching the spatial depth of the salient region with the position of the corresponding focal plane;
wherein m and n are positive integers;
the processing module is specifically configured to, when m is greater than or equal to n, determine positions of n focal planes according to spatial depths of n saliency areas of the m saliency areas, where the position of each focal plane matches the spatial depth of a corresponding saliency area, and allocate the remaining m-n saliency areas to the n focal planes according to the spatial depths of the remaining m-n saliency areas; when m is less than n, determining the positions of m focal planes according to the spatial depths of the m salient regions, and controlling the rest n-m focal planes to coincide with any one or more of the m focal planes.
6. The display device according to claim 5, wherein the calculation module is specifically configured to calculate n using the following formula: and n is A/B, wherein A is the display frequency of the display, B is the refresh frequency of the display screen, and the display frequency of the display is integral multiple of the refresh frequency of the display screen.
7. The display device according to claim 5, wherein the processing module is specifically configured to divide a display time of one frame into n time segments;
and determining the positions of the n focal planes according to the spatial depths of the m saliency areas, controlling the variable focus lens to adjust the focal plane of the human eye through the variable focus lens to a k focal plane located at a k position in a k time period, and controlling the display to display the saliency areas located on the k focal plane, wherein k is a positive integer not greater than n.
8. A virtual reality apparatus, comprising a display device as claimed in any one of claims 5 to 7.
9. A display device, comprising: memory, processor and computer program stored on the memory and executable on the processor, which computer program, when executed by the processor, carries out the steps in the display method according to any one of claims 1 to 4.
10. A computer-readable storage medium, having stored thereon a computer program which, when being executed by a processor, carries out the steps in the display method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910106916.8A CN109756726B (en) | 2019-02-02 | 2019-02-02 | Display device, display method thereof and virtual reality equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910106916.8A CN109756726B (en) | 2019-02-02 | 2019-02-02 | Display device, display method thereof and virtual reality equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109756726A CN109756726A (en) | 2019-05-14 |
| CN109756726B true CN109756726B (en) | 2021-01-08 |
Family
ID=66407332
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910106916.8A Active CN109756726B (en) | 2019-02-02 | 2019-02-02 | Display device, display method thereof and virtual reality equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109756726B (en) |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10386636B2 (en) * | 2014-01-31 | 2019-08-20 | Magic Leap, Inc. | Multi-focal display system and method |
| US10218968B2 (en) * | 2016-03-05 | 2019-02-26 | Maximilian Ralph Peter von und zu Liechtenstein | Gaze-contingent display technique |
| CN106210693A (en) * | 2016-06-29 | 2016-12-07 | 联想(北京)有限公司 | Method, electronic equipment and the electronic installation that a kind of image shows |
| US10330936B2 (en) * | 2017-01-19 | 2019-06-25 | Facebook Technologies, Llc | Focal surface display |
| CN107580207A (en) * | 2017-10-31 | 2018-01-12 | 武汉华星光电技术有限公司 | The generation method and generating means of light field 3D display cell picture |
| CN108919492B (en) * | 2018-07-25 | 2021-05-07 | 京东方科技集团股份有限公司 | Near-to-eye display device, system and display method |
-
2019
- 2019-02-02 CN CN201910106916.8A patent/CN109756726B/en active Active
Also Published As
| Publication number | Publication date |
|---|---|
| CN109756726A (en) | 2019-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20230388586A1 (en) | System and Method for Displaying a Stream of Images | |
| Vishwanath | Toward a new theory of stereopsis. | |
| Vishwanath et al. | Retinal blur and the perception of egocentric distance | |
| CN106959759B (en) | Data processing method and device | |
| Vishwanath et al. | Seeing in 3-D with just one eye: Stereopsis without binocular vision | |
| US10321820B1 (en) | Measuring optical properties of an eyewear device | |
| CN108230234B (en) | Image blurs processing method, device, storage medium and electronic equipment | |
| CN107515474B (en) | Automatic stereo display method and device and stereo display equipment | |
| CN105659200A (en) | Method, apparatus, and system for displaying graphical user interface | |
| CN109522866A (en) | Naked eye 3D rendering processing method, device and equipment | |
| CN111066026A (en) | Techniques for providing virtual light adjustments to image data | |
| CN106774929B (en) | Display processing method of virtual reality terminal and virtual reality terminal | |
| Toyama et al. | A natural interface for multi-focal plane head mounted displays using 3D gaze | |
| CN106200941B (en) | Control method of virtual scene and electronic equipment | |
| Guan et al. | Stereoscopic depth constancy | |
| Vienne et al. | The role of vertical disparity in distance and depth perception as revealed by different stereo-camera configurations | |
| JP6558365B2 (en) | Image processing apparatus, image processing method, and program | |
| CN107093395B (en) | Transparent display device and image display method thereof | |
| CN109756726B (en) | Display device, display method thereof and virtual reality equipment | |
| US11543655B1 (en) | Rendering for multi-focus display systems | |
| CN109816406B (en) | Article marking method, device, equipment and medium | |
| Orlosky et al. | The role of focus in advanced visual interfaces | |
| US11237413B1 (en) | Multi-focal display based on polarization switches and geometric phase lenses | |
| Hung et al. | Divisive display augmented reality (ddAR) for real-world warfighter performance | |
| US20200066004A1 (en) | Text focus for head mounted displays |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |