WO2022166712A1 - 图像的显示方法、装置、可读介质和电子设备 - Google Patents

图像的显示方法、装置、可读介质和电子设备 Download PDF

Info

Publication number
WO2022166712A1
WO2022166712A1 PCT/CN2022/074018 CN2022074018W WO2022166712A1 WO 2022166712 A1 WO2022166712 A1 WO 2022166712A1 CN 2022074018 W CN2022074018 W CN 2022074018W WO 2022166712 A1 WO2022166712 A1 WO 2022166712A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
row
pixels
sub
pixel points
Prior art date
Application number
PCT/CN2022/074018
Other languages
English (en)
French (fr)
Inventor
柯春桥
罗诚
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2022166712A1 publication Critical patent/WO2022166712A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/20—Processor architectures; Processor configuration, e.g. pipelining
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/00—3D [Three Dimensional] image rendering
    • G06T15/005—General purpose rendering architectures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
    • H04N21/4728—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content for selecting a Region Of Interest [ROI], e.g. for requesting a higher resolution version of a selected region
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/28—Indexing scheme for image data processing or generation, in general involving image processing hardware
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting

Definitions

  • the present application relates to the technical field of virtual reality, and in particular, to an image display method, apparatus, readable medium and electronic device.
  • VR Virtual Reality
  • VR is a computer simulation technology that can create a virtual world for users and give users an immersive experience. It can be widely used in medicine, entertainment, military aerospace, exhibition and other fields.
  • VR devices When people use VR devices (such as VR all-in-one machines), they have higher and higher requirements for the clarity, fidelity and immersion of virtual images. It means that the requirements for the computing and rendering capabilities of the Graphics Processing Unit (GPU) and the bandwidth of image data transmission are getting higher and higher.
  • the size, weight, and battery life of VR devices limit the computing power of the GPU and the transmission bandwidth of image data.
  • the foveated rendering solution can be used to reduce the calculation amount of the GPU.
  • a large buffer is required to cache the image data, which is bound to be the volume, weight, battery life, and screen time of the VR device. delays and product cost challenges.
  • Embodiments of the present application provide an image display method, apparatus, readable medium, and electronic device.
  • the technical solution of the present application is to render the image to be processed based on the gaze point rendering technology after acquiring the image to be processed at the system end, so as to obtain a sub-image corresponding to the gaze point area of the human eye and a sub-image corresponding to the periphery of the gaze point area of the human eye.
  • the system side interleaves and integrates the sub-images corresponding to the gaze point area of the human eye and the sub-images corresponding to the periphery of the gaze point area of the human eye according to the set line ratio to obtain the integrated image, and then sends the integrated image to the display on the display side. It can effectively reduce the display delay and reduce the size of the buffer.
  • an embodiment of the present application provides an image display method, including:
  • the gaze point position information is used to describe the position where the user's visual center falls on the screen of the electronic device; Rendering to obtain a first image and a second image, where the first image corresponds to the first area in the image to be processed, the second image corresponds to the second area in the image to be processed, and the first area is the user in the image to be processed
  • the area within the preset range of the position of the fixation point, and the second area is the area outside the first area in the image to be processed; each row of pixels in the first image and each row of pixels in the second image are set according to the set
  • the line ratio is staggered and integrated to obtain an integrated image; at least part of the pixel points in each row of pixels in the integrated image corresponding to the first image are cached, and the pixels in the integrated image corresponding to the second image are cached. Enlarging at least part of the pixel points in each row of pixels, combining the enlarged part of the pixel points and the buffer
  • the above-mentioned method further includes: the first image and the second image have the same resolution.
  • the above-mentioned method further includes: staggering and integrating each row of pixels in the first image and each row of pixels in the second image according to a set row ratio to obtain an integrated
  • the post image includes:
  • the above-mentioned method further includes: buffering at least part of the pixel points in each line of pixels in the integrated image corresponding to the first image, and storing the pixels in the integrated image Corresponding to at least part of the row pixels in each row of pixels in the second image, the enlarged part of the row pixels and the buffered part of the row pixels are combined to obtain the combined image including:
  • the above-mentioned method further includes: characterized in that the value of X is equal to the value of Y.
  • the above-mentioned method further includes: after the resolution of the second image is enlarged by Y times, the resolution is the same as that of the combined image.
  • the above-mentioned method further includes: buffering at least part of the pixel points in each line of pixels in the integrated image corresponding to the first image, and storing the pixels in the integrated image Corresponding to at least part of the row pixels in each row of pixels in the second image, the enlarged part of the row pixels and the buffered part of the row pixels are combined to obtain the combined image including:
  • the decompressed at least part of the row pixel points corresponding to the second image are enlarged by S times and combined with the decompressed at least part of the row pixel points corresponding to the first image to obtain a combined image.
  • the above-mentioned method further includes: after the resolution of the second image is enlarged by S times, the resolution is the same as that of the combined image.
  • the above-mentioned method further includes: buffering at least part of the pixel points in each line of pixels in the integrated image corresponding to the first image, and storing the pixels in the integrated image Corresponding to at least part of the row pixels in each row of pixels in the second image, the enlarged part of the row pixels and the buffered part of the row pixels are combined to obtain the combined image including:
  • the above-mentioned method further includes: after the resolution of the second image is enlarged by T times, the resolution is the same as that of the combined image.
  • an image display device including:
  • an acquisition module for acquiring the image to be processed and the user's gaze point position information, wherein the gaze point position information is used to describe the position where the user's visual center falls on the screen of the electronic device;
  • a first processing module configured to render the to-be-processed image based on the acquired to-be-processed image and the gaze point position information to obtain a first image and a second image, wherein the first image corresponds to the first area in the to-be-processed image,
  • the second image corresponds to a second area in the image to be processed, the first area is an area within a preset range of the user's gaze point in the image to be processed, and the second area is an area outside the first area in the image to be processed;
  • the second processing module is used for staggered integration of each row of pixels in the first image and each row of pixels in the second image according to a set row ratio to obtain an integrated image;
  • the third processing module is configured to cache at least part of the pixel points in each row of pixels in the integrated image corresponding to the first image, and store the pixels in each row of the integrated image corresponding to the second image. At least part of the row pixels of the device are enlarged, and the enlarged part of the row pixels and the buffered part of the row pixels are combined to obtain a combined image; and a display module is used to display the combined image.
  • an embodiment of the present application provides a readable medium, where an instruction is stored on the readable medium, and the instruction, when executed on an electronic device, causes the electronic device to perform the above-mentioned first aspect and various possible implementations of the first aspect. Any kind of image display method.
  • an electronic device including:
  • memory for storing instructions for execution by one or more processors of the electronic device
  • the processor which is one of the processors of the electronic device, is configured to execute the first aspect and any image display method in various possible implementations of the first aspect.
  • Fig. 1(a) is a schematic diagram showing an application scenario of an image display method according to some embodiments of the present application
  • Figure 1(b) shows a schematic diagram of the position of a human eye fixation point according to some embodiments of the present application
  • Fig. 2 shows the hardware structure block diagram of the system end shown in Fig. 1 (a) according to some embodiments of the present application;
  • FIG. 3 shows a block diagram of the hardware structure of the display terminal shown in FIG. 1(a) according to some embodiments of the present application;
  • FIG. 4 shows a display path of an image to be processed according to some embodiments of the present application
  • Fig. 5 shows the processing flow of the image to be processed corresponding to Fig. 4 according to some embodiments of the present application
  • FIG. 6 shows a sequence diagram of an image to be processed in the embodiment shown in FIG. 5 according to some embodiments of the present application
  • FIG. 7 shows a display path of an image to be processed according to some embodiments of the present application.
  • FIG. 8 shows the processing flow of the image to be processed corresponding to FIG. 7 according to some embodiments of the present application
  • Figure 9(a) shows sub-image A and sub-image B, both of which have M*N resolutions, according to some embodiments of the present application
  • Fig. 9(b) shows an integration manner of sub-image A and sub-image B according to some embodiments of the present application
  • Fig. 9(c) shows an integration manner of sub-image A and sub-image B according to some embodiments of the present application
  • Figure 9(d) shows an image with a horizontal resolution of M*X and a vertical resolution of N*X, according to some embodiments of the present application
  • Figure 9(e) shows another image with a horizontal resolution of M*X and a vertical resolution of N*X, according to some embodiments of the present application.
  • FIG. 10 shows a display timing diagram of the to-be-processed image shown in FIG. 8 according to some embodiments of the present application.
  • Fig. 11 shows the display path of the image to be processed in the display method provided by the present application according to some embodiments of the present application;
  • FIG. 12 shows the display process of the image to be processed shown in FIG. 11 according to some embodiments of the present application
  • Fig. 13 shows another display path of the image to be processed according to some embodiments of the present application.
  • FIG. 14 shows the display process of the image to be processed shown in FIG. 13 according to some embodiments of the present application
  • FIG. 15 shows a display timing diagram of the to-be-processed image shown in FIG. 14 according to some embodiments of the present application.
  • FIG. 16 shows a structural block diagram of an image display apparatus according to some embodiments of the present application.
  • Illustrative embodiments of the present application include, but are not limited to, image display methods, apparatuses, readable media, and electronic devices.
  • the image to be processed is rendered based on the gaze point rendering technology to obtain a sub-image corresponding to the gaze point area of the human eye (for convenience of description, hereinafter referred to as sub-image A) and a sub-image corresponding to the periphery of the gaze point region of the human eye (for convenience of description, hereinafter referred to as sub-image B).
  • sub-image A a sub-image corresponding to the gaze point area of the human eye
  • sub-image B sub-image corresponding to the periphery of the gaze point region of the human eye
  • the combined image sequence is : a0, a1, a2, b0, a3, a4, a5, b1, etc., and so on, to get the integrated image.
  • the system side outputs the integrated image to the display side, and the display side caches the received image after interlacing and integration.
  • the data currently buffered in the buffer of the display end includes some rows of the sub-image A, for example, the data currently buffered in the buffer are rows a0, a1, and a2 of the sub-image A.
  • the display end receives a row of the sub-image B corresponding to a part of the row of the sub-image A currently buffered in the buffer, it can enlarge the row of the sub-image B according to the resolution of the display screen of the display end, so as to enlarge the sub-image B.
  • This row of B is filled with pixels, so that the resolution of the sub-image B is the same as the resolution of the display screen.
  • This row of the enlarged image B is combined with some rows of the aforementioned sub-image A, and finally displayed on the display end.
  • the magnification of the sub-image B is the same as the value of the line ratio set when the sub-image A and the sub-image B are interlaced and integrated. Since the data written to the buffer of the display end each time is a part of the line of the sub-image A, it can be combined with the part of the line of the enlarged sub-image B, and the display screen is driven by the driving circuit to display the image, instead of the sub-image. A is written to the register of the display side all at once. Therefore, the display delay can be effectively reduced. In addition, the size of the buffer at the display end can be reduced, thereby helping to reduce the volume and weight of the product, reduce the cost, and improve the battery life of the product.
  • Fixation Point Rendering technology is a selective image rendering technology, which can select the gaze area of the human eye for full resolution rendering based on eye tracking technology, and perform blur rendering in the area outside the gaze area of the human eye. , so as to achieve a clear image of the human eye fixation point area and blurred non-fixation point area. In this way, the amount of data calculation and the bandwidth of data transmission can be greatly reduced, thereby saving computing resources and reducing power consumption.
  • FIG. 1( a ) shows a schematic diagram of an application scenario 10 of the image display method provided by the present application, according to some embodiments of the present application.
  • the scene 10 includes the system end 200 and the display end 100 .
  • the system terminal 200 can communicate with the display terminal 100 and update the firmware of the display terminal 100 through various wired (eg, USB interface) or wireless means (eg, Bluetooth communication link, radio frequency identification technology, short-range wireless communication technology, etc.).
  • the user can perform three-dimensional stereoscopic visual experience of watching movies, shopping, games, distance teaching, etc. through the display terminal 100 .
  • the system terminal 200 can calculate and obtain the position information of the user's visual center on the display screen when the user is looking at the display screen of the display terminal 100 based on the eye tracking technology, that is, the user's visual center Gaze position information.
  • the user's visual center is located at the center of the display screen, that is, the position of the gaze point is at the center of the display screen, and the area near the position of the gaze point can be regarded as the gaze point area , the area outside the fixation area can be regarded as the periphery of the fixation area.
  • the image to be processed (that is, the image to be displayed by the display terminal 100 ) is rendered by the foveated rendering technology, so as to obtain the sub-image A and the sub-image B.
  • the system end 200 may perform interleaving and integration of the sub-image A and the sub-image B according to the set line ratio.
  • the resolutions of sub-image A and sub-image B are both 1280*1280, that is, sub-image A and sub-image B have 1280 rows of pixels and 1280 columns of pixels. Points are marked as lines a1 to a1280; pixel points of 1280 rows of sub-image B are marked as lines b1 to b1280.
  • sub-image A and sub-image B are staggered and integrated in a ratio of 3:1, the integrated sequence is: a1, a2, a3, b1, a4, a5, a6, b2, a7, a8, a9, b3, etc., And so on.
  • the blank pixels are used to interleave and integrate with the remaining rows in sub-image B in the same manner as described above.
  • the system end 200 outputs the integrated image to the display end 100 , and the display end buffers the received interlaced and integrated image.
  • the data currently buffered in the buffer of the display end includes some rows of the sub-image A, for example, the data currently buffered in the buffer is the a1, a2, and a3 rows of the sub-image A.
  • the display end receives a row in the sub-image B corresponding to a part of the row of the sub-image A currently buffered in the buffer, for example, the b1 row of the sub-image B
  • the sub-image B can be adjusted according to the resolution of the display screen of the display end.
  • the row is enlarged, that is, the row of the sub-image B is filled with pixels, so that the resolution of the sub-image B is consistent with the resolution of the display screen.
  • the magnification of the sub-image B is the same as the value of the line ratio set when the sub-image A and the sub-image B are interlaced and integrated.
  • the data written to the buffer of the display side each time is a part of the line of the sub-image A, it can be combined with the part of the line of the magnified sub-image B, and the display screen is driven by the driving circuit to display the image instead of the sub-image.
  • A is written to the register of the display side all at once.
  • the data currently written into the buffer of the display end is lines a1, a2, and a3 of sub-image A.
  • the display end receives line b1 of sub-image B, the pixels of line b1 of sub-image B can be tripled. Pixel padding to get b1', b2', b3'.
  • b1' and a1 are combined, for example, covered by a rectangle, and then the image is displayed on the display screen.
  • sub-image B does not need to be cached. Therefore, the display time delay can be effectively reduced, and the size of the buffer at the display end can be reduced, thereby helping to reduce the volume and weight of the product, reduce the cost, and improve the battery life of the product.
  • the display end 100 may further compress the received sub-image A and sub-image B with different magnifications before caching, and the compression magnification of sub-image B is greater than the compression magnification of sub-image A .
  • the compressed sub-image A and sub-image B buffered in the buffer are decompressed, and then the decompressed sub-image B is enlarged and combined with the decompressed sub-image A for image display. Since the display end 100 first compresses the sub-image A and the sub-image B with different magnifications and then caches them, the cache pressure of the buffer is greatly reduced, and the capacity requirement of the buffer is reduced.
  • the display terminal 100 may also directly cache the received sub-image A. After the display terminal 100 receives one of the rows of the sub-image B, the display terminal 100 can store the row of the sub-image B. to zoom in. The row of the enlarged sub-image B is combined with the sub-image A for image display. Since the display end needs to cache all of the sub-image A and one row of the sub-image B, and does not need to cache all of the sub-image B, the cache pressure of the buffer can be greatly reduced, and the capacity requirement of the buffer can be reduced, thereby It is beneficial to reduce the volume and weight of the product, reduce the cost, and improve the battery life.
  • the display terminal 100 can be any kind of virtual world capable of generating a three-dimensional space, providing simulation of the user's senses such as vision, hearing, touch, etc., and forming a virtual reality field of view in the user's brain visual system, An electronic device or part of an electronic device that produces stereoscopic vision. This application does not limit this.
  • the display end 100 and the system end 200 may be located in the same virtual reality or augmented reality device, such as a VR all-in-one machine.
  • the display end 100 and the system end 200 may also be two independent electronic devices that can communicate with each other.
  • the display end 100 may be VR/AR glasses, VR/AR helmet, etc.
  • the system end 200 may be a server, a distributed server cluster composed of multiple servers, a mobile phone, a tablet computer, a laptop computer, a desktop computer, and the like.
  • FIG. 2 shows a block diagram of the hardware structure of the system end 200 shown in FIG. 1( a ) according to some embodiments of the present application.
  • the system end 200 includes a central processing unit (Central Processing Unit, CPU) 201, a graphics processing unit (Graphics Processing Unit, GPU) 202, a memory 203, a communication module 204, an interface module 205, and the like.
  • CPU Central Processing Unit
  • GPU Graphics Processing Unit
  • the CPU 201 can be used to analyze the acquired picture including the human eye, and calculate the gaze point position information of the human eye.
  • the GPU 202 is used to render the to-be-processed image according to the gaze point position information of the human eye calculated by the CPU 201 to obtain a sub-image of the gaze point area of the human eye and a sub-image of the periphery of the gaze point area of the human eye.
  • the GPU 202 may also be used to interlace and integrate the sub-images of the gaze point area of the human eye and the sub-images of the periphery of the gaze point area of the human eye according to a set line ratio.
  • the memory 203 is used to store software programs and data, and the CPU 201 executes various functional applications and data processing of the system side 20 by running the software programs and data stored in the memory 203.
  • the memory 203 may store the image including the human eye obtained by the CPU 201, the position information of the gaze point of the human eye calculated according to the image including the human eye, and the human eye rendered by the graphics processor 202.
  • the communication module 204 can be used to enable the system end 200 and the display end 100 to communicate through various wired communication methods or wireless communication methods.
  • the system end 200 uses wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS) ), frequency modulation (FM), near field communication (NFC), infrared technology (infrared, IR) and other wireless communication methods to communicate with the display terminal 100.
  • WLAN wireless local area networks
  • Wi-Fi wireless fidelity
  • BT wireless fidelity
  • GNSS global navigation satellite system
  • FM frequency modulation
  • NFC near field communication
  • infrared technology infrared, IR
  • the interface module 205 includes an external memory interface, a universal serial bus (universal serial bus, USB) interface, and the like.
  • the external memory interface can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the system end 200 .
  • the external memory card communicates with the central processing unit 201 through the external memory interface to realize the data storage function.
  • the universal serial bus interface can be used for communication between the system end 200 and the display end 100 or other electronic devices.
  • the interface module 205 may further include a subscriber identification module (subscriber identification module, SIM) card interface.
  • SIM subscriber identification module
  • the system side 200 also includes a power supply, a power management system, and the like.
  • the power management system is used to connect the power supply, the CPU 201, the GPU 202, the memory 203, the communication module 204, and the interface module 205.
  • the power management system 106 receives the input of the power supply, and supplies power to the CPU 201, the GPU 202, the memory 203, the communication module 204, the interface module 205, and the like.
  • the power management system can also be provided in the CPU 201.
  • FIG. 2 is only an exemplary structure for realizing the functions of the system end 200 in the technical solution of the present application, and the system end 200 having other structures and capable of realizing similar functions is also applicable to the technology of the present application scheme, which is not limited here.
  • FIG. 3 shows a block diagram of the hardware structure of the display terminal 100 shown in FIG. 1( a ) according to some embodiments of the present application.
  • the display terminal 100 includes an interface module 101, a communication module 102, a processor 103, a camera 105, an audio module 106, a display screen 107, a sensor module 108, a lens 109, and the like.
  • the processor 103 may include one or more processing units, for example, may include a central processing unit (Central Processing Unit, CPU), an image signal processor (Image Signal Processor, ISP), a digital signal processor (Data Signal Processor, DSP) , Micro-programmed Control Unit (MCU), artificial intelligence (Artificial Intelligence, AI) processor or programmable logic device (Field Programmable Gate Array, FPGA) and other processing modules or processing circuits.
  • CPU Central Processing Unit
  • ISP image Signal Processor
  • DSP Digital Signal Processor
  • MCU Micro-programmed Control Unit
  • AI Artificial Intelligence
  • FPGA Field Programmable Gate Array
  • the processor 103 can be configured to perform calculations according to the data transmitted by the sensor module 108 to generate spatial position and orientation information of the user's head, and the like.
  • the power module 111 may include power supplies, power management components, and the like.
  • the power source can be a battery.
  • the power management part is used to manage the charging of the power supply and the power supply of the power supply to other modules.
  • the power management component is used to connect the power supply, the charging management module and the processor 103 .
  • the lens 109 includes two lenses. Since there is only one screen, the images viewed by the left eye and the right eye must be separated independently to have stereoscopic vision. The left and right eye images are continuously and alternately displayed on the screen, that is, using the parallax of the left and right eyes, and the physiological characteristics of human visual persistence, the user can experience a three-dimensional visual image.
  • the display screen 107 is used to display images.
  • the display screen 107 includes a display driver integrated circuit (Display Driver Integrated Circuit, DDIC) 104, a display panel 110, and the like.
  • DDIC Display Driver Integrated Circuit
  • the DDIC 104 may include row driver circuits, column driver circuits, timing control circuits, buffers, and the like. It is used to send driving signals and data to the display screen 107 in the form of electrical signals. By controlling the brightness and color of the display panel 110 , image information such as letters and pictures can be presented on the display panel 110 .
  • the DDIC 104 is configured to compress the sub-images of the gaze point area of the human eye and the sub-images of the periphery of the gaze point area of the human eye output by the GPU 202 of the system end 200 shown in FIG. 2 , After decompression, enlargement, and combination, the display screen 107 is driven to display images.
  • the DDIC 104 can also be used for the sub-image and the human eye gaze point region output by the GPU 202 of the system end 200 shown in FIG.
  • the sub-images at the periphery of the eye fixation area are enlarged and combined to drive the display screen 107 for image display.
  • the timing control circuit selects the pixel points of each row of the pixel array of the display screen 107 by controlling the row driving circuit, and then outputs the sub-image A and sub-image B buffered in the buffer to display on the display screen through the column driving circuit .
  • the buffer is used for buffering the sub-image of the human eye gaze point area and the sub-images of the periphery of the human eye gaze point area output by the GPU 202 of the system side 200 shown in FIG. 2, or to the system side 200 shown in FIG.
  • the sub-images of the human eye fixation area and the sub-images of the periphery of the human eye fixation area that are output by the GPU 202 and are interleaved and integrated according to the set line ratio are cached. It should be noted that, in some embodiments of the present application, only the sub-images of the gaze point region of the human eye may also be cached, or the sub-images of the gaze point region of the human eye after interleaving and integration.
  • the display panel 110 can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic).
  • LCD liquid crystal display
  • OLED organic light-emitting diode
  • active-matrix organic light-emitting diode active-matrix organic light-emitting diode
  • active-matrix organic active-matrix organic light-emitting diode
  • light emitting diode AMOLED
  • flexible light emitting diode flexible light emitting diode
  • FLED flexible light emitting diode
  • quantum dot light emitting diode quantum dot light emitting diode
  • the camera 105 is used for collecting human eye images or videos, so that the CPU 201 of the system end 200 shown in FIG. 2 can use the eye tracking technology to calculate the gaze point position information of the human eye. Then, the GPU 202 uses the foveated rendering technology to render the sub-images of the foveated region of the human eye and the sub-images of the periphery of the foveated region of the human eye.
  • the camera 105 may be an infrared camera, and the position information of the user's eye gazing on the display screen 107 is determined by the position of the infrared light emitted by the infrared light emitting module (not shown) in the cornea of the user's eye.
  • the sensor module 108 may include an inertial measurement unit (Inertial Measurement Unit, IMU) nine-axis sensor, a spatial position sensor, etc., for generating spatial position and orientation information of the user's head.
  • IMU Inertial Measurement Unit
  • Audio module 106 may convert digital audio information to analog audio signal output, or convert analog audio input to digital audio signal. Audio module 106 may also be used to encode and decode audio signals. In some embodiments, the audio module 106 may include a speaker, earpiece, microphone, and headphone jack.
  • the communication module 102 can be used to enable the system end 200 and the display end 100 to communicate through various wired communication methods or wireless communication methods.
  • the interface module 101 includes an external memory interface, a universal serial bus (universal serial bus, USB) interface, and the like.
  • the communication module 102 is similar to the communication module 204 shown in FIG. 2
  • the interface module 101 is similar to the interface module 205 shown in FIG. 2 , and details are not repeated here.
  • the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the display terminal 100 .
  • the display terminal 100 may include more or less components than shown, or some components may be combined, or some components may be split, or different component arrangements.
  • the illustrated components may be implemented in hardware, software, or a combination of software and hardware.
  • the display path of the image to be processed is shown.
  • the data source can include the position information of the human eye gaze point and the image to be processed.
  • the GPU 202 of the system end system 200 performs sub-regional rendering of the image to be processed based on the gaze point rendering technology and according to the position information of the human eye gaze point, and obtains the sub-image A and the sub-image B. Then the GPU 202 outputs the sub-image A and the sub-image B to the display terminal 100.
  • the buffer in the display terminal 100 buffers the sub-image A and the sub-image B, and then the sub-image B is enlarged and combined with the sub-image A, and then the column driving circuit and the row driving circuit are controlled by the timing control circuit to make the pixel array.
  • the combined sub-image A and sub-image B are displayed.
  • the display scheme using the foveated rendering technology in the related art generally includes:
  • Step 500 The CPU 201 of the system end 200 calculates and obtains the gaze point information of the human eye based on the eye tracking technology.
  • Step 502 The GPU 202 of the system end 200 obtains the sub-image A and the sub-image B based on the gaze-point rendering technology according to the obtained human eye gaze point information, and outputs the sub-image A and the sub-image B.
  • Step 504 The display end 100 receives and buffers the sub-image A and the sub-image B, and then enlarges the sub-image B and combines it with the sub-image A.
  • the purpose of enlarging is to make the resolution of sub-image B and the display screen consistent.
  • Step 506 The timing control circuit controls the column driving circuit and the row driving circuit, so that the pixel array displays the combined image.
  • FIG. 6 shows a timing diagram of the image to be processed in the embodiment shown in FIG. 5 .
  • the vertical synchronization (V-Sync) signal is generated by the display terminal 100 when a frame of pictures on the display screen of the display terminal 100 is completely refreshed from left to right and from top to bottom.
  • V-Sync vertical synchronization
  • the GPU 202 of the system end 200 enables V-Sync
  • the GPU 202 will wait for the V-Sync signal of the display end 100 to be sent out before rendering a new frame, so as to solve the tearing phenomenon of the picture and increase the smoothness of the picture.
  • the display terminal 100 since the display terminal 100 needs to cache the sub-image A in its buffer, it will enlarge the received sub-image B, and display it after combining with the sub-image A.
  • the display time of a frame of image on the display screen starts from the sub-image B of the current frame of image written in the buffer until the sub-image of the next frame of image is written in the buffer. B ends.
  • the capacity of the buffer is required to be high, which makes the buffer of the DDIC 104 larger, resulting in an increase in the size of the DDIC 104, which is not conducive to cost control and product size reduction.
  • the embodiment of the present application proposes that after the system end 200 renders the sub-image A and the sub-image B corresponding to the image to be processed, The sub-image A and the sub-image B are interleaved and integrated at the system end 200 according to the set line ratio, and then the integrated image is output to the display end 100 .
  • the DDIC 104 in the display end 100 processes the interlaced and integrated images before displaying them.
  • the display method provided according to some embodiments of the present application will be explained below with reference to FIGS. 7 to 10 .
  • the data source includes the image to be processed, and the image including the human eye captured by the camera of the VR glasses 100 and received by the computer 200 .
  • the image to be processed may be a picture frame of a movie downloaded by a movie playing application on the computer 200, an image stored in an image database of the computer 200, an image interface displayed on a shopping website of the computer 200, and the like. It can be understood that the movie picture that the user sees is composed of continuous still images frame by frame.
  • any frame of the above-mentioned consecutive images is collectively referred to as an image to be processed.
  • the CPU 201 calculates and obtains the position information of the gaze point of the human eye according to the image including the human eye.
  • the GPU 202 performs sub-regional rendering of the image to be processed based on the gaze point rendering technology and according to the position information of the human eye gaze point to obtain sub-image A and sub-image B. The resolutions of sub-image A and sub-image B are the same.
  • the GPU 202 interleaves and integrates the sub-image A and the sub-image B according to the set line ratio to obtain the interleaved and integrated image, and then outputs the interleaved and integrated image to the VR glasses 100.
  • the VR glasses 100 cache the received interlaced and integrated images.
  • the data currently buffered in the buffer of the VR glasses 100 includes some rows of the sub-image A, for example, the data currently buffered in the buffer is rows a1, a2, and a3 of the sub-image A.
  • the VR glasses 100 receive a row of the sub-image B corresponding to the partial row of the sub-image A currently buffered in the buffer, for example, the b1 row of the sub-image B. That is, the row of the sub-image B can be enlarged according to the resolution of the display screen of the VR glasses 100, that is, the row of the sub-image B is filled with pixels, so that the resolution of the sub-image B is consistent with the resolution of the display screen. .
  • the magnification of the sub-image B is the same as the value of the line ratio set when the sub-image A and the sub-image B are interlaced and integrated.
  • the computer 200 interleaves and integrates the sub-image A and the sub-image B according to the set line ratio, and then performs image transmission. That is, when the computer 200 transmits the integrated image to the VR glasses 100, the arrangement order of each row of the sub-image A and each row of the sub-image B is determined. Therefore, each time the data written into the buffer of the DDIC 104 in the VR glasses 100 is a partial row of the sub-image A, it can be combined with a partial row of the enlarged sub-image B, and then the pixel array is driven by the DDIC 104 to perform image processing. show. Instead of writing the sub-image A into the buffer of the DDIC 104 in the VR glasses 100 all at once. And sub-image B does not need to be cached.
  • the VR glasses 100 do not need to wait until the sub-image A is completely cached by the buffer in the DDIC 104 before processing and displaying the image, which can effectively reduce the display delay and improve the user experience.
  • the capacity requirement for the buffer of the DDIC 104 can be reduced, the size of the DDIC 104 can be reduced, and further, when designing a product, it is beneficial to reduce the volume, weight, power consumption, etc. of the product.
  • the display process of the to-be-processed image shown in FIG. 7 is described in detail below. Specifically, as shown in FIG. 8 , the display process of the to-be-processed image includes:
  • Step 800 The VR glasses 100 and the computer 200 establish a connection.
  • the VR glasses 100 and the computer 200 can be connected through a data cable, wireless local area network, Bluetooth, etc., so that the VR glasses 100 and the computer 200 can communicate.
  • the VR glasses 100 and the computer 200 may also establish a connection in other manners, which are not limited to the above method.
  • Step 802 The VR glasses 100 send the image including the human eye captured by the camera to the computer 200 .
  • the VR glasses 100 can emit infrared light to the human eye through an infrared light emitter.
  • the human eye forms infrared light in the cornea of the human eye after receiving the infrared light emitted by the infrared light emitter. reflection point.
  • the image including the human eye is acquired in real time by the infrared camera, and then the image including the human eye captured by the infrared camera is sent to the computer 200 .
  • Step 804 The computer 200 performs calculation according to the received image including the human eye to obtain the position information of the gaze point of the human eye.
  • the gaze point position information of the human eye can be calculated in the following manner: the CPU 201 of the computer 200 processes the received image including the human eye to obtain the gradient of the gray level in the set direction
  • the maximum value of the gradient of the gray level is determined as the pupil edge feature of the human eye image, and then the pupil edge feature can be fitted to determine the center position of the fitted image, and the center position is determined as the human eye image.
  • the center position of the pupil in the eye image, and the gray image obtained after processing the human eye image is based on the spot recognition algorithm to obtain the area with high gray value as the Purkins spot, and the location of the Purkins spot is the light spot. , or the "reflection point" location.
  • the corneal curvature center is obtained, and the gaze point position information of the human eye is determined according to the pupil center position, the corneal curvature center and the preset angle between the optical axis of the eye and the visual axis. It can be understood that, in other embodiments, the gaze point position information of the human eye may also be determined in other manners. This application does not limit this.
  • Step 806 the computer 200 obtains the sub-image A and the sub-image B based on the gaze point rendering technology according to the obtained gaze point position information of the human eye. Specifically, after the GPU 202 of the computer 200 obtains the image to be processed and the position information of the human eye gaze point, it performs image rendering through vertex transformation calculation, pixel filling calculation, etc., to obtain the sub-image A corresponding to the gaze point position area and the corresponding Sub-image B in areas other than the fixation point.
  • Step 808 The computer 200 interleaves and integrates the sub-image A and the sub-image B according to the set line ratio. To determine the arrangement order of each row of pixel points in the sub-image A and the sub-image B when the GPU 202 is performing data transmission, so that when the GPU 202 sends the image data, it is not necessary to completely send the sub-image A and then send the sub-image B, and then The buffer in the DDIC 104 of the VR glasses 100 can only buffer part of the lines of the sub-image A, which reduces the display delay and reduces the size of the buffer.
  • the set line ratio (denoted as X) may be any numerical value set as required.
  • the GPU 202 of the computer 200 renders the sub-image A and the sub-image B with a resolution of M*N as shown in FIG. 9(a).
  • the resolutions of sub-image A and sub-image B are both 1280*1280, that is, sub-image A and sub-image B have 1280 rows of pixels and 1280 columns of pixels.
  • the 1280-line pixel points of sub-image A are denoted as a1 to a1280 lines; the 1280-line pixel points of sub-image B are denoted as b1-b1280 lines. If sub-image A and sub-image B are staggered and integrated according to the ratio of 3, the integrated sequence is: a1, a2, a3, b1, a4, a5, a6, b2, a7, a8, a9, b3, etc. .
  • the position of the gaze point of the human eye when the position of the gaze point of the human eye is above the screen, according to the integration method shown in FIG. integration. In some embodiments, when the position of the gaze point of the human eye is at the center of the screen, according to the integration method shown in FIG. integration.
  • Step 810 the computer 200 sends the integrated image to the VR glasses 100 .
  • the computer 200 sends the sub-image A and the sub-image B integrated according to the set line ratio to the VR glasses 100 by means of wireless local area network, Bluetooth or the like.
  • the VR glasses 100 For the VR glasses 100 to display images.
  • the sequence of the sent images can be, for example, a1, a2, a3, b1, a4, a5 as shown in FIG. 9(b). , a6, b2, a7, a8, a9, b3, a10, a11, a12, b4, and so on.
  • sub-image A After sub-image A is sent, it will be 3 blanks, b5, 3 blanks, b6, 3 blanks, b7, 3 blanks, b8, 3 blanks, b9, 3 blanks, b10, 3 blanks, b11, 3 lines of blank, b12, that is, until all the content of the sub-image B is sent.
  • Step 812 The VR glasses 100 receive the integrated image, and buffer some of the lines corresponding to the sub-image A therein. For subsequent processing and display by the VR glasses 100 .
  • the sequence of receiving images by the VR glasses 100 is a1, a2, a3, b1, a4, a5, a6, b2, a7, a8, a9, b3, a10, a11, a12, b4, after all the lines of the sub-image A are received, then there are 3 blank lines, b5, 3 blank lines, b6, 3 blank lines, b7, 3 blank lines, b8, 3 blank lines, b9, 3 Line blank, b10, 3 lines blank, b11, 3 lines blank, b12, until all lines of sub-image B are received.
  • the data written to the buffer of the DDIC 104 in the VR glasses 100 each time is a part of the row of the sub-image A, not the whole of A.
  • the current cached content in the buffer of DDIC 104 is a1, a2, a3, after a1, a2, a3 are displayed, the next cached content is a4, a5, a6, after a4, a5, a6 are displayed, the next The content of a cache is a7, a8, a9, and so on.
  • Step 814 The VR glasses 100 enlarge a row of pixels in the sub-image B corresponding to a part of the row of the sub-image A currently buffered.
  • the contents currently buffered by the buffer of the DDIC 104 are a1, a2, and a3.
  • the VR glasses 100 can use an interpolation algorithm, such as the nearest neighbor Interpolation, bilinear interpolation, high-order interpolation algorithm, etc., perform pixel filling (ie, enlargement) on the pixel points of the b1 line of the sub-image B.
  • an interpolation algorithm such as the nearest neighbor Interpolation, bilinear interpolation, high-order interpolation algorithm, etc.
  • the resolutions of the sub-image A and the sub-image B are both M*N, the resolution of the display screen 107 of the VR glasses 100 is H*V, and the sub-image B is enlarged
  • the magnification is Y, where Y can be 1.33, 2, 2.5, 2.85, 3, 4, etc.
  • the line ratio X (ie a:b) of the sub-image A in each a line and the sub-image B in each b line is equal to the magnification Y at which the sub-image B is magnified in the VR glasses 100 .
  • both a and b are integers. Generally, in order to minimize the size of the buffer, a and b take the smallest integer.
  • Step 816 The VR glasses 100 combine the enlarged row pixels in the sub-image B with the corresponding partial rows in the sub-image A, so that the sub-image B and the sub-image A are finally combined into a complete image.
  • the DDIC 104 of the VR glasses can overlay the sub-image A to a position in the enlarged sub-image B corresponding to the position information of the gaze point of the human eye according to the position information of the gaze point of the human eye (for example, a rectangular overlay, or Processed into a circle and then covered) to obtain a combined image.
  • smooth fusion processing is performed on the splicing boundary of the combined image, for example, using low-pass filtering and other methods to perform smooth fusion processing on the boundary area of the combined image, so as to obtain the final image to be displayed.
  • smooth fusion processing is performed on the splicing boundary of the combined image, for example, using low-pass filtering and other methods to perform smooth fusion processing on the boundary area of the combined image, so as to obtain the final image to be displayed.
  • the position of the sub-image A varies with the position of the gaze point of the human eye.
  • the final image presented is the image at the center of human vision (ie, sub-image A) is clear, while the image outside the visual center (ie, sub-image B) is clear. ) is rather vague. Since human eyes are not sensitive to the freshness of images outside the visual center, the amount of data calculation can be reduced while ensuring the user's visual experience.
  • Step 818 The timing control circuit of the VR glasses 100 controls the column driving circuit and the row driving circuit, so that the pixel array displays the combined image.
  • the row driving circuit of the DDIC 104 can be controlled by the timing control circuit in the DDIC 104 of the VR glasses to select the pixels of each row of the pixel array of the display screen 107, and then the column driving circuit of the DDIC 104 The pixel point information of each row after the combination is displayed on the display screen 107 .
  • the user looks at the display screen 107 of the VR glasses 100 through the lenses of the VR glasses 100, since the images seen by the user's left and right eyes are independent and separated, and the physiological characteristics of human vision persistence are added, three-dimensional vision can be seen. Stereoscopic image.
  • FIG. 10 shows a display timing diagram of the image to be processed in the embodiment shown in FIG. 8 .
  • the GPU 202 of the computer 100 enables vertical synchronization, the GPU 202 will wait for the V-Sync signal of the VR glasses 100 to be sent out before rendering a new frame, so as to solve the phenomenon of screen tearing and increase the smoothness of the screen.
  • the buffer of the DDIC 104 completes the buffering of part of the lines of the sub-image A, as long as a line of the sub-image B is received, the line of the sub-image B can be enlarged and compared with that in the sub-image A.
  • the corresponding lines are combined and displayed.
  • the display time delay is greatly reduced, the requirement for the capacity of the buffer of the DDIC 104 is reduced, and the size of the DDIC 104 is reduced.
  • the VR glasses 100 can respectively compress the received sub-image A and sub-image B with different magnifications before buffering . In this way, the buffer pressure on the buffer can be greatly reduced, and the capacity requirement of the buffer can be reduced.
  • the data source includes the image to be processed, and the image including the human eye captured by the camera of the VR glasses 100 and received by the computer 200 .
  • the image to be processed is an image stored in an image database on the computer 200 .
  • the CPU 201 calculates and obtains the position information of the gaze point of the human eye according to the image including the human eye.
  • the GPU 202 performs sub-regional rendering of the image to be processed based on the gaze point rendering technology and according to the position information of the human eye gaze point to obtain sub-image A and sub-image B.
  • the rendering process is similar to the foregoing rendering process, and will not be repeated here.
  • the computer 200 sends the sub-image A and the sub-image B to the VR glasses 100, and the VR glasses 100 respectively compress the received sub-image A and the sub-image B and then cache it, wherein the compression ratio of the sub-image B is greater than the compression of the sub-image A. magnification.
  • the VR glasses 100 then read the buffered compressed sub-image A and sub-image B from the buffer, decompress the compressed sub-image B, enlarge the decompressed sub-image B, and combine it with the sub-image A for image display. Because the VR glasses 100 respectively compress the received sub-image A and sub-image B with different magnifications before buffering, and the compression magnification of the sub-image B is greater than the compression magnification of the sub-image A. In this way, the buffer pressure of the buffer can be greatly reduced, the capacity requirement of the buffer can be reduced, and the size of the DDIC 104 can be reduced.
  • the display process of the to-be-processed image shown in FIG. 12 will be described in detail below. Specifically, as shown in Figure 12, the display process of the image to be processed includes:
  • Step 1200 The VR glasses 100 and the computer 200 are connected.
  • the VR glasses 100 and the computer 200 can be connected through a data cable, wireless local area network, Bluetooth, etc., so that the VR glasses 100 and the computer 200 can communicate.
  • the VR glasses 100 and the computer 200 may also establish a connection in other manners, which are not limited to the above method.
  • Step 1202 The VR glasses 100 send the image including the human eye captured by the camera to the computer 200 .
  • the computer 200 perform calculation analysis according to the human eye image. For example, an image including the human eye is acquired in real time by an infrared camera, and then the image including the human eye captured by the infrared camera is sent to the computer 200 .
  • Step 1204 The computer 200 performs calculation according to the received image including the human eye, and obtains the gaze point position information of the human eye.
  • the calculation method is similar to step 804, and details are not repeated here.
  • Step 1206 The computer 200 obtains the sub-image A and the sub-image B based on the gaze point rendering technology according to the obtained gaze point position information of the human eye.
  • the rendering process is similar to step 806, and details are not repeated here.
  • Step 1208 the computer 200 sends the sub-image A and the sub-image B to the VR glasses 100 .
  • the computer 200 sends the sub-image A and the sub-image B to the VR glasses 100 by means of wireless local area network, Bluetooth, or the like.
  • the VR glasses 100 For the VR glasses 100 to display images.
  • the computer 200 can also stagger and integrate the sub-image A and the sub-image B in a manner similar to the above-mentioned step 808 according to the set line ratio, so as to obtain the integrated sub-image A and the sub-image B. Then, the integrated image is sent to the VR glasses 100 .
  • Step 1210 The VR glasses 100 receive the sub-image A and the sub-image B, respectively compress the sub-image A and the sub-image B and then cache them. Since the human eye is not sensitive to the image quality outside the visual center, in order to reduce the requirement on the capacity of the buffer, the compression ratio of the sub-image B can be made larger than the compression ratio of the sub-image A. For example, the compression ratio of sub-image B is 100, and the compression ratio of sub-image A is 10.
  • sub-picture A and sub-picture B pictures may be compressed by block division, prediction, transformation, quantization, arithmetic coding, and the like.
  • Step 1212 The VR glasses 100 decompress the compressed sub-image A and sub-image B read from the buffer.
  • the compressed sub-image A and sub-image B may be decompressed by an image decompression algorithm.
  • Step 1214 The VR glasses 100 enlarge the decompressed sub-image B.
  • an interpolation algorithm such as nearest neighbor interpolation, bilinear interpolation, high-order interpolation algorithm, etc., can be used to make the resolution of sub-image B and display screen 107 smaller. Consistent.
  • Step 1216 The VR glasses 100 combine the enlarged sub-image B with the sub-image A. So that sub-image B and sub-image A are combined into a complete image.
  • the combination method is similar to that of step 816, and details are not repeated here.
  • Step 1218 The timing control circuit of the VR glasses 100 controls the column driving circuit and the row driving circuit, so that the pixel array displays the combined image.
  • the VR glasses 100 may buffer all the received sub-image A, and the first line of the sub-image B arrives After that, it is directly enlarged and displayed after being combined with the sub-image A. Since the buffer in the DDIC 104 of the VR glasses 100 needs to buffer all the data of the sub-image A, only one row of the sub-image B needs to be buffered in real time, which can greatly reduce the buffering pressure of the buffer and reduce the need for buffering. capacity requirements of the device.
  • the horizontal scanning frequency of the display screen 107 of the VR glasses 100 needs to be doubled.
  • the frame rate of the display screen 107 of the VR glasses 100 is 90 Hz.
  • the display screen 107 of the VR glasses 100 can light up the pixels of each row within 11 milliseconds.
  • the display The screen 107 needs to light up each row of pixels within 5.5 milliseconds.
  • the data source includes the image to be processed, and the image including the human eye captured by the camera of the VR glasses 100 and received by the computer 200 .
  • the image to be processed may be a game screen of a game application installed on the computer 200 .
  • the CPU 201 calculates and obtains the position information of the gaze point of the human eye according to the image including the human eye.
  • the GPU 202 performs sub-regional rendering of the image to be processed based on the gaze point rendering technology and according to the position information of the human eye gaze point to obtain sub-image A and sub-image B.
  • the rendering process is similar to the foregoing rendering process, and will not be repeated here.
  • the computer 200 sends the sub-image A and the sub-image B to the VR glasses 100, and the VR glasses 100 buffer the received sub-image A. As long as one row of the sub-image B is received, the image of the row can be directly enlarged, and the sub-image B can be directly enlarged. Image A is combined and displayed as an image.
  • the buffer in the DDIC 104 of the VR glasses 100 needs to buffer all the data of the sub-image A, only one row of the sub-image B needs to be buffered in real time, which can greatly reduce the buffering pressure of the buffer and reduce the need for buffering. capacity requirements of the device.
  • the display process of the to-be-processed image shown in FIG. 14 is described in detail below. Specifically, as shown in FIG. 14 , the display process of the to-be-processed image includes:
  • Step 1400 The VR glasses 100 and the computer 200 are connected.
  • the VR glasses 100 and the computer 200 can be connected through a data cable, wireless local area network, Bluetooth, etc., so that the VR glasses 100 and the computer 200 can communicate.
  • the VR glasses 100 and the computer 200 may also establish a connection in other manners, which are not limited to the above method.
  • Step 1402 The VR glasses 100 send the image including the human eye captured by the camera to the computer 200 .
  • the computer 200 perform calculation analysis according to the human eye image. For example, an image including the human eye is acquired in real time by an infrared camera, and then the image including the human eye captured by the infrared camera is sent to the computer 200 .
  • Step 1404 The computer 200 performs calculation according to the received image including the human eye to obtain the position information of the gaze point of the human eye.
  • the calculation method is similar to step 804, and details are not repeated here.
  • Step 1406 The computer 200 obtains the sub-image A and the sub-image B based on the gaze point rendering technology according to the obtained gaze point position information of the human eye.
  • the rendering process is similar to step 806, and details are not repeated here.
  • Step 1408 the computer 200 sends the sub-image A and the sub-image B to the VR glasses 100 .
  • the computer 200 sends the sub-image A and the sub-image B to the VR glasses 100 via wireless local area network, Bluetooth, or the like.
  • the VR glasses 100 For the VR glasses 100 to display images.
  • the computer 200 can also stagger and integrate the sub-image A and the sub-image B in a manner similar to the above-mentioned step 808 according to the set line ratio, so as to obtain the integrated sub-image A and the sub-image B. Then, the integrated image is sent to the VR glasses 100 .
  • Step 1410 The VR glasses 100 receive the sub-image A and the sub-image B, and after the sub-image A is cached, one row of the sub-image B is cached. For example, in some embodiments, after sub-image A is cached, the first row of sub-image B is cached. Since the buffer in the DDIC 104 of the VR glasses 100 needs to buffer all the data of the sub-image A, only one row of the sub-image B needs to be buffered in real time, which can greatly reduce the buffering pressure of the buffer and reduce the need for buffering. capacity requirements of the device.
  • Step 1412 After the VR glasses 100 have cached a row of the sub-image B, the VR glasses 100 zoom in on the row. For example, use an interpolation algorithm to enlarge a row of sub-image B written by a factor of 3.
  • Step 1414 The VR glasses 100 combine the enlarged sub-image B with the sub-image A.
  • the combination method is similar to that of step 816, and details are not repeated here.
  • Step 1416 The timing control circuit of the VR glasses 100 controls the column driving circuit and the row driving circuit, so that the pixel array displays the combined sub-image A and sub-image B.
  • the row driving circuit of the DDIC 104 can be controlled by the timing control circuit in the DDIC 104 of the VR glasses to select the pixels of each row of the pixel array of the display screen 107, and then the column driving circuit of the DDIC 104 The sub-image A and the sub-image B buffered in the buffer of the DDIC 104 are output and displayed on the display screen 107.
  • FIG. 15 shows the display timing diagram of the image to be processed in the embodiment shown in FIG. 14 .
  • the GPU 202 of the computer 100 enables vertical synchronization, the GPU 202 will wait for the V-Sync signal of the VR glasses 100 to be sent before rendering a new frame.
  • the buffer of DDIC 104 completes the buffering of sub-image A, as long as a row of sub-image B is received, the row of sub-image B can be enlarged and corresponding to it in sub-image A. displayed after combining the lines of the .
  • the requirement on the capacity of the buffer of the DDIC 104 is reduced, and the size of the DDIC 104 is reduced.
  • FIG. 16 provides an image display apparatus 1600 according to some embodiments of the present application, specifically, including:
  • an acquisition module 1602 configured to acquire the image to be processed and the user's gaze point position information, wherein the gaze point position information is used to describe the position where the user's visual center falls on the screen of the electronic device;
  • the first processing module 1604 is configured to render the to-be-processed image based on the acquired to-be-processed image and the gaze point position information to obtain a first image and a second image, wherein the first image corresponds to the first area in the to-be-processed image , the second image corresponds to the second area in the image to be processed, the first area is an area within a preset range of the user's gaze point position in the image to be processed, and the second area is an area outside the first area in the image to be processed;
  • the second processing module 1606 is used for staggered integration of each row of pixels in the first image and each row of pixels in the second image according to a set row ratio to obtain an integrated image;
  • the third processing module 1608 is configured to buffer at least part of the pixel points in each row of pixels in the integrated image corresponding to the first image, and store the pixels in each row of the integrated image corresponding to the second image. at least part of the row pixels in the enlarged, and the enlarged part of the row pixels and the buffered part of the row pixels are combined to obtain a combined image;
  • the display module 1610 is used for displaying the combined image.
  • the image display device 1600 shown in FIG. 16 corresponds to the image display method provided by the present application, and the technical details in the specific description of the image display method provided by the present application are still applicable to the image display method shown in FIG. 16 .
  • the display device 1600 of the image please refer to the above for the specific description, which will not be repeated here.
  • Embodiments of the mechanisms disclosed herein may be implemented in hardware, software, firmware, or a combination of these implementation methods.
  • Embodiments of the present application may be implemented as a computer program or program code executing on a programmable system including at least one processor, a storage system (including volatile and nonvolatile memory and/or storage elements) , at least one input device, and at least one output device.
  • Program code may be applied to input instructions to perform the functions described herein and to generate output information.
  • the output information can be applied to one or more output devices in a known manner.
  • a processing system includes any processor having a processor such as, for example, a Digital Signal Processor (DSP), a microcontroller, an Application Specific Integrated Circuit (ASIC), or a microprocessor system.
  • DSP Digital Signal Processor
  • ASIC Application Specific Integrated Circuit
  • the program code may be implemented in a high-level procedural language or an object-oriented programming language to communicate with the processing system.
  • the program code may also be implemented in assembly or machine language, if desired.
  • the mechanisms described in this application are not limited in scope to any particular programming language. In either case, the language may be a compiled language or an interpreted language.
  • the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof.
  • the disclosed embodiments can also be implemented as instructions carried by or stored on one or more transitory or non-transitory machine-readable (eg, computer-readable) storage media, which can be executed by one or more processors read and execute.
  • the instructions may be distributed over a network or over other computer-readable media.
  • a machine-readable medium can include any mechanism for storing or transmitting information in a form readable by a machine (eg, a computer), including, but not limited to, floppy disks, optical disks, optical disks, read only memories (CD-ROMs), magnetic CD-ROM, Read Only Memory (ROM), Random Access Memory (RAM), Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Only Memory (EPROM) Read-Only Memory (Electrically Erasable Programmable Read-Only Memory, EEPROM), magnetic or optical cards, flash memory, or for the use of the Internet to transmit information in electrical, optical, acoustic or other forms of propagation signals (for example, carrier waves, infrared signals, digital signals etc.) tangible machine-readable storage.
  • machine-readable media includes any type of machine-readable media suitable for storing or transmitting electronic instructions or information in a form readable by a machine (eg, a computer).
  • each unit/module mentioned in each device embodiment of this application is a logical unit/module.
  • a logical unit/module may be a physical unit/module or a physical unit/module.
  • a part of a module can also be implemented by a combination of multiple physical units/modules.
  • the physical implementation of these logical units/modules is not the most important, and the combination of functions implemented by these logical units/modules is the solution to the problem of this application. The crux of the technical question raised.
  • the above-mentioned device embodiments of the present application do not introduce units/modules that are not closely related to solving the technical problems raised in the present application, which does not mean that the above-mentioned device embodiments do not exist. other units/modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请涉及一种图像的显示方法、装置、可读介质和电子设备。该方法包括：获取待处理图像以及用户的注视点位置信息；基于获取的待处理图像以及注视点位置信息，对待处理图像进行渲染，得到第一图像和第二图像；将第一图像和第二图像按照设定的行比例进行交错整合，得到整合后的图像；将对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像。本申请通过将注视点区域图像和外围区域图像按照设定的行比例进行交错整合后再进行缓存、放大、组合及显示，可以有效降低显示时延并减少缓存器的大小。

Description

图像的显示方法、装置、可读介质和电子设备
本申请要求于2021年02月03日提交中国专利局、申请号为202110151925.6、申请名称为“图像的显示方法、装置、可读介质和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及虚拟现实技术领域，特别涉及一种图像的显示方法、装置、可读介质和电子设备。
背景技术
虚拟现实(Virtual Reality，VR)技术是可以为用户创建虚拟世界，并使用户有身临其境的体验感的计算机仿真技术。被广泛可以应用于医学、娱乐、军事航天、展览展示等领域。
人们在使用VR设备(例如VR一体机)时，对于虚拟画面的清晰度、画面逼真程度，以及沉浸感的要求越来越高。意味着对图形处理器(Graphics Processing Unit，GPU)的计算渲染能力以及图像数据传输带宽的要求也越来越高。然而VR设备的体积、重量、续航能力限制了GPU的计算能力以及图像数据的传输带宽。通常可以利用注视点渲染的方案减少GPU的计算量，但是由于GPU渲染出的图像数据较大，需要较大的缓存器来缓存图像数据，势必为VR设备的体积、重量、续航能力、画面时延以及产品成本带来挑战。
发明内容
本申请实施例提供了一种图像的显示方法、装置、可读介质和电子设备。本申请的技术方案通过在系统端获取到待处理图像之后，基于注视点渲染技术对待处理的图像进行渲染，得到对应于人眼注视点区域的子图像和对应于人眼注视点区域外围的子图像。系统端将对应于人眼注视点区域的子图像和对应于人眼注视点区域外围的子图像按照设定的行比例进行交错整合后，得到整合后的图像，再将整合后的图像发送至显示端进行显示。可以有效降低显示时延并减少缓存器的大小。
第一方面，本申请实施例提供了一种图像的显示方法，包括：
获取待处理图像以及用户的注视点位置信息，其中注视点位置信息用于描述用户的视觉中心落在电子设备的屏幕上的位置；基于获取的待处理图像以及注视点位置信息，对待处理图像进行渲染，得到第一图像和第二图像，其中，第一图像对应于待处理图像中的第一区域，第二图像对应于待处理图像中的第二区域，第一区域为待处理图像中用户的注视点位置预设范围内的区域，第二区域为待处理图像中第一区域以外的区域；将第一图像中的每一行像素点以及第二图像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像；将整合后的图像中对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将整合后的图像中对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像；以及显示组合后的图像。
在上述第一方面的一种可能的实现中，上述方法还包括：第一图像和第二图像的分辨率相同。
在上述第一方面的一种可能的实现中，上述方法还包括：将第一图像中的每一行像素点以及第二图 像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像包括：
将第一图像中的每a行像素点为一组和第二图像中的每b行像素点为一组进行交错整合，得到整合后的图像，其中，a:b＝X，a>b，a和b为正整数。
在上述第一方面的一种可能的实现中，上述方法还包括：将整合后的图像中对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将整合后的图像中对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像包括：
将整合后的图像中对应于第一图像的每a行像素点进行缓存，并且将整合后的图像中对应于第二图像的每b行像素点放大Y倍；
读取当前被缓存的第一图像的a行像素点，并且将第二图像中被放大的b行像素点与读取的第一图像中当前被缓存的a行像素点进行组合，得到组合后的图像。
在上述第一方面的一种可能的实现中，上述方法还包括：其特征在于，X的数值等于Y的数值。
在上述第一方面的一种可能的实现中，上述方法还包括：第二图像的分辨率被放大Y倍后与组合后的图像的分辨率相同。
在上述第一方面的一种可能的实现中，上述方法还包括：将整合后的图像中对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将整合后的图像中对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像包括：
将整合后的图像中对应于第一图像的每一行像素点以及整合后的图像中对应于第二图像的每一行像素点进行压缩，其中对应于第一图像的每一行像素点的压缩倍率大于对应于第二图像的每一行像素点的压缩倍率；
将被压缩后的对应于第一图像的至少部分行像素点以及被压缩后的对应于第二图像的至少部分行像素点进行缓存；
读取当前被缓存的被压缩后的对应于第一图像的至少部分行像素点以及被压缩后的对应于第二图像的至少部分行像素点，并且将读取的当前被缓存的被压缩后的对应于第一图像的至少部分行像素点以及被压缩后的对应于第二图像的至少部分行像素点进行解压；
将被解压后的对应于第二图像的至少部分行像素点放大S倍后与被解压后的对应于第一图像的至少部分行像素点进行组合，得到组合后的图像。
在上述第一方面的一种可能的实现中，上述方法还包括：第二图像的分辨率被放大S倍后与组合后的图像的分辨率相同。
在上述第一方面的一种可能的实现中，上述方法还包括：将整合后的图像中对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将整合后的图像中对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像包括：
将整合后的图像中对应于第一图像的每一行像素点以及将整合后的图像中对应于第二图像中的其中一行像素点进行缓存；
读取当前被缓存的对应于第一图像中的部分行像素点，以及对应于第二图像中的其中一行像素点，并且将读取的对应于第二图像中的其中一行像素点放大T倍；
将被放大的对应于第二图像中的一行像素点与读取的当前被缓存的对应于第一图像中的部分行像 素点进行组合，得到组合后的图像。
在上述第一方面的一种可能的实现中，上述方法还包括：第二图像的分辨率被放大T倍后与组合后的图像的分辨率相同。
第二方面，本申请实施例提供了一种图像的显示装置，包括：
获取模块，用于获取待处理图像以及用户的注视点位置信息，其中注视点位置信息用于描述用户的视觉中心落在电子设备的屏幕上的位置；
第一处理模块，用于基于获取的待处理图像以及注视点位置信息，对待处理图像进行渲染，得到第一图像和第二图像，其中，第一图像对应于待处理图像中的第一区域，第二图像对应于待处理图像中的第二区域，第一区域为待处理图像中用户的注视点位置预设范围内的区域，第二区域为待处理图像中第一区域以外的区域；
第二处理模块，用于将第一图像中的每一行像素点以及第二图像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像；
第三处理模块，用于将整合后的图像中对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将整合后的图像中对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像；以及显示模块，用于显示组合后的图像。
第三方面，本申请实施例提供了一种可读介质，可读介质上存储有指令，该指令在电子设备上执行时使电子设备执行上述第一方面以及第一方面的各种可能实现中的任意一种图像的显示方法。
第四方面，本申请实施例提供了一种电子设备，包括：
存储器，用于存储由电子设备的一个或多个处理器执行的指令，以及
处理器，是电子设备的处理器之一，用于执行上述第一方面以及第一方面的各种可能实现中的任意一种图像的显示方法。
附图说明
图1(a)根据本申请的一些实施例，示出了一种图像的显示方法的应用场景的示意图；
图1(b)根据本申请的一些实施例，示出了一种人眼注视点位置的示意图；
图2根据本申请的一些实施例，示出了图1(a)所示的系统端的硬件结构框图；
图3根据本申请的一些实施例，示出了图1(a)所示的显示端的硬件结构框图；
图4根据本申请的一些实施例，示出了一种待处理图像的显示路径；
图5根据本申请的一些实施例，示出了与图4对应的待处理图像的处理流程；
图6根据本申请的一些实施例，示出了图5所示的实施例中待处理图像的时序图；
图7根据本申请的一些实施例，示出了一种待处理图像的显示路径；
图8根据本申请的一些实施例，示出了与图7对应的待处理图像的处理流程；
图9(a)根据本申请的一些实施例，示出了分辨率均为M*N子图像A和子图像B；
图9(b)根据本申请的一些实施例，示出了子图像A和子图像B的一种整合方式；
图9(c)根据本申请的一些实施例，示出了子图像A和子图像B的一种整合方式；
图9(d)根据本申请的一些实施例，示出了横向分辨率为M*X，纵向分辨率为N*X的图像；
图9(e)根据本申请的一些实施例，示出了另一种横向分辨率为M*X，纵向分辨率为N*X的图像；
图10根据本申请的一些实施例，示出了图8所示的待处理图像的显示时序图；
图11根据本申请的一些实施例，示出了本申请提供的显示方法中待处理图像的显示路径；
图12根据本申请的一些实施例，示出了图11所示的待处理图像的显示过程；
图13根据本申请的一些实施例，示出了待处理图像的另一种显示路径；
图14根据本申请的一些实施例，示出了图13所示的待处理图像的显示过程；
图15根据本申请的一些实施例，示出了图14所示的待处理图像的显示时序图；
图16根据本申请的一些实施例，示出了一种图像显示装置的结构框图。
具体实施方式
本申请的说明性实施例包括但不限于图像的显示方法、装置、可读介质和电子设备。
本申请的技术方案通过在系统端获取到待处理图像之后，基于注视点渲染技术对待处理的图像进行渲染，得到对应于人眼注视点区域的子图像(为了叙述方便，以下简称子图像A)和对应于人眼注视点区域外围的子图像(为了叙述方便，以下简称子图像B)。系统端将子图像A和子图像B按照设定的行比例进行交错整合。举例来说，将子图像A的每3行像素点为一组(例如a0、a1、a2)和子图像B的每1行像素点为一组(例如b0)进行组合，组合后的图像顺序为:a0、a1、a2、b0、a3、a4、a5、b1等等，以此类推，得到整合后的图像。然后系统端将整合后的图像输出给显示端，显示端将接收到的经过交错整合后的图像进行缓存。其中，显示端的缓存器中当前缓存的数据包括子图像A的部分行，例如缓存器中当前缓存的数据为子图像A的a0、a1、a2行。一旦显示端接收到子图像B中与缓存器中当前缓存的子图像A的部分行对应的一行，即可根据显示端的显示屏的分辨率对子图像B的该行进行放大，以将子图像B的该行进行像素填充，使子图像B的分辨率和显示屏的分辨率一致。放大后的图像B的该行与前述子图像A的部分行进行组合，最终由显示端进行显示。其中，子图像B被放大的倍数与前述子图像A和子图像B进行交错整合时的设定的行比例的数值相同。由于每次写入显示端的缓存器的数据为子图像A的部分行，即可与放大后的子图像B的部分行进行组合后，通过驱动电路驱动显示屏进行图像显示，而非将子图像A一次全部写入显示端的缓存器。因此，可以有效降低显示时延。并且可以减小显示端缓存器的大小，从而有利于减小产品的体积、重量，降低成本，提升产品续航能力。
注视点渲染(Fixation Point Rendering)技术是一种选择性图像渲染技术，其可以基于眼动追踪技术选择人眼的注视区域进行全分辨率渲染，而在人眼的注视区域以外的区域进行模糊渲染，从而实现人眼注视点区域清晰、非注视点区域模糊的图像。这样可以大大降低数据计算量，降低数据传输带宽，从而节省运算资源、降低功耗。
下面将结合附图对本申请的实施例作进一步地详细描述。
图1(a)根据本申请的一些实施例，示出了本申请提供的图像的显示方法的应用场景10的示意图。其中，在场景10中包括系统端200和显示端100。系统端200可以通过各种有线(例如通过USB接口)或无线方式(例如蓝牙通信链路、射频识别技术、近距离无线通信技术等)和显示端100进行通信以及对显示端100进行固件更新。用户可以通过显示端100进行观影、购物、游戏、远程教学等的三维立体视觉体验。
在图1(a)所示的实施例中，系统端200可以基于眼动追踪技术计算得到用户在注视显示端100的显示屏时，用户在显示屏上的视觉中心的位置信息，即用户的注视点位置信息。例如，在图1(b)所示的实施例中，用户的视觉中心落在显示屏的中心处，即注视点位置处于显示屏的中心处，注视点位置附近的区域可以视为注视点区域，注视点区域以外的区域可以视为注视点区域外围。然后通过注视点渲染技术对待处理的图像(即要通过显示端100进行显示的图像)进行渲染，得到子图像A和子图像B。 系统端200可以将子图像A和子图像B按照设定的行比例进行交错整合。例如子图像A和子图像B的分辨率均为1280*1280，即子图像A和子图像B均有1280行像素点和1280列像素点，按照从上到下的顺序将子图像A的1280行像素点记为a1至a1280行；将子图像B的1280行像素点记为b1至b1280行。若将子图像A和子图像B按照3：1的比例进行交错整合，则整合后的顺序为：a1、a2、a3、b1、a4、a5、a6、b2、a7、a8、a9、b3等，以此类推。当子图像A的1280行像素点被整合完毕后，采用空白像素点与子图像B中剩余的行按照与上述相同的方式进行交错整合。然后系统端200将整合后的图像输出给显示端100，显示端将接收到的经过交错整合后的图像进行缓存。其中，显示端的缓存器中当前缓存的数据包括子图像A的部分行，例如，缓存器中当前缓存的数据为子图像A的a1、a2、a3行。一旦显示端接收到子图像B中与缓存器中当前缓存的子图像A的部分行对应的一行，例如子图像B的b1行，即可根据显示端的显示屏的分辨率对子图像B的该行进行放大，即：将子图像B的该行进行像素填充，以使子图像B的分辨率和显示屏的分辨率一致。并且与前述子图像A的部分行进行组合，最终由显示端进行显示。其中，子图像B被放大的倍数与前述子图像A和子图像B进行交错整合时的设定的行比例的数值相同。
由于每次写入显示端的缓存器的数据为子图像A的部分行，即可与被放大的子图像B的部分行进行组合后，通过驱动电路驱动显示屏进行图像显示，而非将子图像A一次全部写入显示端的缓存器。例如当前写入显示端的缓存器的数据为子图像A的a1、a2、a3行，一旦显示端接收到子图像B的b1行之后，即可将子图像B的b1行的像素进行3倍的像素填充，得到b1’、b2’、b3’。然后将b1’和a1例如通过矩形覆盖的方式进行组合后通过显示屏进行图像显示。以此类推，将b2’和a2进行组合；将b3’和a3进行组合后通过显示屏进行图像显示。并且不需要对子图像B进行缓存。因此，可以有效降低显示时延，减小显示端缓存器的大小，从而有利于减小产品的体积、重量，降低成本，提升产品续航能力。
在本申请的一些实施例中，还可以在显示端100对接收到的子图像A和子图像B分别进行不同倍率的压缩后再进行缓存，并且子图像B的压缩倍率大于子图像A的压缩倍率。在图像显示时，将缓存器中缓存的被压缩的子图像A和子图像B进行解压，再将解压后的子图像B进行放大，与解压后的子图像A进行组合后进行图像显示。由于显示端100是先将子图像A和子图像B分别进行不同倍率的压缩后再进行缓存，如此大大减小了缓存器的缓存压力，降低对缓存器的容量要求。
此外，在本申请的一些实施例中，显示端100还可以对接收到的子图像A直接进行缓存，当显示端100接收到子图像B的其中一行之后，即可将子图像B的该行进行放大。放大后的子图像B的该行与子图像A进行组合后进行图像显示。由于显示端需要将子图像A的全部和子图像B的其中一行进行缓存，不需要将子图像B全部进行缓存，因此，可以大大减小缓存器的缓存压力，降低对缓存器的容量要求，从而有利于减少产品的体积、重量，降低成本，提升续航能力。
此外，可以理解的是，显示端100可以是任意一种能够产生一个三维空间的虚拟世界，提供用户关于视觉、听觉、触觉等感官的模拟，在用户大脑视觉系统中形成一个虚拟现实视场，产生立体视觉的电子设备或电子设备的部分模块。本申请对此不作限定。
显示端100和系统端200可以位于同一个虚拟现实或者增强现实设备中，例如VR一体机。显示端100和系统端200也可以为两个独立的可以相互通信的电子设备。例如显示端100可以为VR/AR眼镜、VR/AR头盔等，系统端200可以为服务器、多个服务器组成的分布式服务器集群、手机、平板电脑、膝上型计算机、台式计算机等。
图2根据本申请的一些实施例，示出了图1(a)所示的系统端200的硬件结构框图。系统端200 包括中央处理器(Central Processing Unit，CPU)201、图形处理器(Graphics Processing Unit，GPU)202、存储器203、通信模块204、接口模块205等。
CPU 201可以用于对获取的包括人眼的图片进行分析，计算出人眼的注视点位置信息。GPU202用于根据CPU 201计算出的人眼的注视点位置信息对待处理图像进行渲染，得到人眼注视点区域的子图像和人眼注视点区域外围的子图像。GPU 202还可以用于对人眼注视点区域的子图像和人眼注视点区域外围的子图像按照设定的行比例进行交错整合。
存储器203用于存储软件程序以及数据，CPU 201通过运行存储在存储器203的软件程序以及数据，执行系统端20的各种功能应用以及数据处理。例如，在本申请的一些实施例中，存储器203可以存储CPU 201获取的包括人眼的图像、根据包括人眼的图像计算得到的人眼注视点位置信息、图形处理器202渲染得到的人眼注视点区域的子图像和人眼注视点区域外围的子图像，以及对人眼注视点区域的子图像和人眼注视点区域外围的子图像按照设定的行比例进行交错整合后得到的图像等。
通信模块204可以用来使系统端200和显示端100通过各种有线通信方式或者无线通信方式进行通信。例如系统端200通过无线局域网(wireless local area networks，WLAN)(如无线保真(wireless fidelity，Wi-Fi)网络)，蓝牙(blue tooth，BT)，全球导航卫星系统(global navigation satellite system，GNSS)，调频(frequency modulation，FM)，近距离无线通信技术(near field communication，NFC)，红外技术(infrared，IR)等无线通信方式和显示端100进行通信。
接口模块205包括外部存储器接口、通用串行总线(universal serial bus，USB)接口等。其中外部存储器接口可以用于连接外部存储卡，例如Micro SD卡，实现扩展系统端200的存储能力。外部存储卡通过外部存储器接口与中央处理器201通信，实现数据存储功能。通用串行总线接口可以用于系统端200和显示端100或者其他电子设备进行通信。在一些实施例中，当系统端200为智能手机时，接口模块205还可以包括用户标识模块(subscriber identification module，SIM)卡接口。
系统端200还包括电源、电源管理系统等。电源管理系统用于连接电源、CPU 201、GPU 202、存储器203、通信模块204、接口模块205。电源管理系统106接收电源的输入，为CPU 201、GPU 202、存储器203、通信模块204、接口模块205等供电。在其他一些实施例中，电源管理系统也可以设置于CPU 201中。
可以理解，图2所示的硬件结构仅仅是实现本申请技术方案中系统端200的功能的一种示例性结构，具有其他结构并能实现类似的功能的系统端200也适用于本申请的技术方案，在此不作限制。
图3根据本申请的一些实施例，示出了图1(a)所示的显示端100的硬件结构框图。显示端100包括接口模块101、通信模块102、处理器103、摄像头105、音频模块106、显示屏107、传感器模块108以及镜片109等。
处理器103可以包括一个或多个处理单元，例如，可以包括中央处理器(Central Processing Unit，CPU)、图像信号处理器(Image Signal Processor，ISP)、数字信号处理器(Data Signal Processor，DSP)、微处理器(Micro-programmed Control Unit，MCU)、人工智能(Artificial Intelligence，AI)处理器或可编程逻辑器件(Field Programmable Gate Array，FPGA)等的处理模块或处理电路。例如，处理器103可以用来根据传感器模块108传送的数据进行计算，生成用户头部的空间位置与方向信息等。
电源模块111可以包括电源、电源管理部件等。电源可以为电池。电源管理部件用于管理电源的充电和电源向其他模块的供电。电源管理部件用于连接电源，充电管理模块与处理器103。
镜片109包括两个透镜，由于屏幕只有一个，因此必须要让左、右眼所看的图像各自独立分 开，才能有立体视觉。使左、右眼画面连续互相交替显示在屏幕上，即利用左、右眼的视差，加上人眼视觉暂留的生理特性，就可以使用户体验到立体的视觉图像。
显示屏107用于显示图像。显示屏107包括显示器驱动集成电路(Display Driver Integrated Circuit，DDIC)104、显示面板110等。
DDIC 104可以包括行驱动电路、列驱动电路、时序控制电路以及缓存器等。用于以电信号的形式向显示屏107发送驱动信号和数据，通过对显示面板110亮度和色彩的控制，使得诸如字母、图片等图像信息得以在显示面板110上呈现。例如，在本申请的一些实施例中，DDIC 104用于对图2所示的系统端200的GPU 202输出的人眼注视点区域的子图像和人眼注视点区域外围的子图像进行压缩、解压、放大、组合后驱动显示屏107进行图像显示。在本申请的一些实施例中，DDIC 104还可以用于对图2所示的系统端200的GPU 202输出的按照设定的行比例进行交错整合后的人眼注视点区域的子图像和人眼注视点区域外围的子图像进行放大、组合后驱动显示屏107进行图像显示。其中，时序控制电路通过控制行驱动电路选通显示屏107的像素阵列的每一行的像素点，然后通过列驱动电路将缓存器中缓存的子图像A与子图像B输出在显示屏上进行显示。缓存器用于对图2所示的系统端200的GPU 202输出的人眼注视点区域的子图像和人眼注视点区域外围的子图像进行缓存，或者是对图2所示的系统端200的GPU 202输出的按照设定的行比例进行交错整合后的人眼注视点区域的子图像和人眼注视点区域外围的子图像进行缓存。需要说明的是，在本申请的一些实施例中，还可以只缓存人眼注视点区域的子图像，或者经过交错整合后的人眼注视点区域的子图像。
显示面板110可以采用液晶显示屏(liquid crystal display，LCD)，有机发光二极管(organic light-emitting diode，OLED)，有源矩阵有机发光二极体或主动矩阵有机发光二极体(active-matrix organic light emitting diode的，AMOLED)，柔性发光二极管(flex light-emitting diode，FLED)，量子点发光二极管(quantum dot light emitting diodes，QLED)等。
摄像头105用于采集人眼图像或视频，以供图2所示的系统端200的CPU 201采用眼动追踪技术，计算出人眼的注视点位置信息。进而由GPU 202采用注视点渲染技术，渲染得到人眼注视点区域的子图像和人眼注视点区域外围的子图像。在一些实施例中，摄像头105可以为红外摄像头，通过红外光发射模块(未示出)发出的红外光在用户眼睛的角膜中的位置，进而确定用户眼睛注视显示屏107的位置信息。
传感器模块108可以包括惯性测量单元(Inertial Measurement Unit，IMU)九轴传感器、空间位置传感器等，用于生成用户头部的空间位置与方向信息。
音频模块106可以将数字音频信息转换成模拟音频信号输出，或者将模拟音频输入转换为数字音频信号。音频模块106还可以用于对音频信号编码和解码。在一些实施例中，音频模块106可以包括扬声器、听筒、麦克风以及耳机接口。
通信模块102可以用来使系统端200和显示端100通过各种有线通信方式或者无线通信方式进行通信。接口模块101包括外部存储器接口、通用串行总线(universal serial bus，USB)接口等。通信模块102与图2所示的通信模块204类似，接口模块101与图2所示的接口模块205类似，在此不再赘述。
可以理解的是，本申请实施例示意的结构并不构成对显示端100的具体限定。在本申请另一些实施例中，显示端100可以包括比图示更多或更少的部件，或者组合某些部件，或者拆分某些部件，或者不同的部件布置。图示的部件可以以硬件，软件或软件和硬件的组合实现。
首先参考图4至图6说明相关技术中采用注视点渲染技术的显示方案。
在图4所示的实施例中，示出了待处理图像的显示路径。其中，数据源可以包括人眼注视点 位置信息和待处理图像。系端统200的GPU 202基于注视点渲染技术，根据人眼注视点位置信息，对待处理图像进行分区域渲染，得到子图像A和子图像B。然后GPU 202将子图像A和子图像B输出给显示端100。显示端100中的缓存器对子图像A和子图像B进行缓存，然后通过将子图像B进行放大后与子图像A进行组合，再经由时序控制电路控制列驱动电路和行驱动电路，使像素阵列显示组合后的子图像A与子图像B。
参考图5，上述相关技术中采用注视点渲染技术的显示方案大致包括：
步骤500：系统端200的CPU 201基于眼动追踪技术计算得到人眼注视点信息。
步骤502：系统端200的GPU 202根据得到的人眼注视点信息，基于注视点渲染技术得到子图像A和子图像B，并输出子图像A和子图像B。
步骤504：显示端100接收并缓存子图像A和子图像B，然后将子图像B进行放大后与子图像A进行组合。放大的目的是为了使子图像B和显示屏的分辨率一致。
步骤506：时序控制电路控制列驱动电路和行驱动电路，使像素阵列显示组合后的图像。
图6示出了图5所示的实施例中待处理图像的时序图。其中，垂直同步(V-Sync)信号是当显示端100的显示屏上的一帧画面从左至右，从上到下完全刷新完毕后，由显示端100产生的。当系统端200的GPU 202开启垂直同步后，GPU 202会等待显示端100的V-Sync信号发出后，才进行新的一帧渲染，以解决画面撕裂现象，增加画面流畅度。参考图6可知，由于显示端100需要将子图像A在其缓存器中缓存完毕后，再将接收到的子图像B进行放大，并且和子图像A进行组合后再进行显示。因此，从GPU 202开始渲染到最终在显示屏上显示图像，有约半帧的显示时延。例如，GPU 202的帧率为90赫兹，则有约5.5毫秒的显示时延，影响用户体验。此外，从图6中可以看出，显示屏上一帧图像的显示时间是从缓存器中写入当前一帧图像的子图像B开始，一直到缓存器中写入下一帧图像的子图像B结束。由于缓存器需要对子图像A和子图像B都进行缓存，因此，对缓存器的容量要求较高，使得DDIC104的缓存器较大，导致DDIC 104尺寸增大，不利于控制成本和降低产品尺寸。
因此，为了解决显示时延较长，对DDIC 104中的缓存器容量要求较高的问题，本申请实施例提出了在系统端200渲染出与待处理图像对应的子图像A和子图像B后，在系统端200按照设定的行比例对子图像A和子图像B进行交错整合，然后将整合后的图像输出给显示端100。显示端100中的DDIC104再对交错整合后的图像进行处理后再进行显示。
下面以图1所示的显示端100为VR眼镜，系统端200为计算机为例，对本申请的一些实施例提供的显示方法进行详细介绍。
以下将参考图7至图10，解释根据本申请的一些实施例提供的显示方法。在图7所示的实施例中，示出了本申请提供的图像的显示方法中待处理图像的显示路径。其中，数据源包括待处理图像，以及计算机200接收到的VR眼镜100的摄像头拍摄的包括人眼的图像。在一些实施例中，待处理图像可以为计算机200上的影片播放应用程序下载的电影的图片帧、计算机200的图像数据库中存储的图像、计算机200的购物网站上显示的图像界面等等。可以理解的是，用户看到的电影画面是由一帧帧连续的静止的图像组成的。计算机200和VR眼镜100在进行图像处理时，每一帧图像都执行相同的处理方式。为了方便叙述本申请方案，以下将上述一帧帧连续的图像中的任意一帧统称为待处理图像。CPU 201根据包括人眼的图像进行计算得到人眼注视点位置信息。GPU 202基于注视点渲染技术，根据人眼注视点位置信息，对待处理图像进行分区域渲染，得到子图像A和子图像B。其中，子图像A和子图像B的分辨率相同。然后GPU 202将子图像A和子图像B按照设定的行比例进行交错整合，得到交错整合后的图像，再将交错整合后的图像输出给VR眼镜100。
VR眼镜100将接收到的经过交错整合后的图像进行缓存。其中，VR眼镜100的缓存器中当前缓存的数据包括子图像A的部分行，例如缓存器中当前缓存的数据为子图像A的a1、a2、a3行。一旦VR眼镜100接收到子图像B中与缓存器中当前缓存的子图像A的部分行对应的一行，例如子图像B的b1行。即可根据VR眼镜100的显示屏的分辨率对子图像B的该行进行放大，即：将子图像B的该行进行像素填充，以使子图像B的分辨率和显示屏的分辨率一致。并且与前述子图像A的部分行进行组合，最终在VR眼镜100上进行显示。其中，子图像B被放大的倍数与前述子图像A和子图像B进行交错整合时的设定的行比例的数值相同。
由于计算机200将子图像A和子图像B按照设定的行比例进行交错整合后，再进行图像传送。即：在计算机200将整合后的图像传送给VR眼镜100时，子图像A的每一行和子图像B的每一行的排列顺序是确定的。因此，每次写入VR眼镜100中DDIC 104的缓存器的数据为子图像A的部分行，即可与被放大的子图像B的部分行进行组合后，再经由DDIC 104驱动像素阵列进行图像显示。而非将子图像A一次全部写入VR眼镜100中DDIC 104的缓存器。并且不需要对子图像B进行缓存。即：VR眼镜100不需要等到DDIC 104中的缓存器将子图像A完全缓存完毕后才进行图像的处理和显示，可以有效降低显示时延，提升用户体验。并且可以降低对DDIC 104的缓存器的容量要求，减小DDIC 104的尺寸，进而对产品设计时，有利于减小产品的体积、重量、功耗等。
下面对图7所示的待处理图像的显示过程进行详细描述，具体地，如图8所示，待处理图像的显示过程包括：
步骤800：VR眼镜100和计算机200建立连接。例如，用户想要通过VR眼镜100体验三维购物，可以将VR眼镜100和计算机200通过数据线、无线局域网、蓝牙等方式进行连接，以使VR眼镜100和计算机200进行通信。可以理解，在其他实施例中，VR眼镜100和计算机200也可以采用其他方式建立连接，不限于上述方法。
步骤802：VR眼镜100将摄像头拍摄到的包括人眼的图像发送给计算机200。以使计算机200根据人眼图像进行计算分析。在一些实施例中，VR眼镜100可以通过红外光发射器向人眼发射红外光，对应地，人眼在接收到这红外光发射器发出的红外光后，在人眼的角膜中形成红外光反射点。通过红外摄像头实时获取包括人眼的图像，然后将红外摄像头拍摄包括人眼的图像发送给计算机200。
步骤804：计算机200根据接收到的包括人眼的图像进行计算，得到人眼的注视点位置信息。
具体地，在一些实施例中，可以通过以下方式计算出人眼的注视点位置信息：计算机200的CPU 201对接收到的包括人眼的图像进行处理得到在设定方向上的灰度的梯度值，将该灰度的梯度最大值所处位置确定为该人眼图像的瞳孔边缘特征，然后可以对瞳孔边缘特征进行拟合，确定出拟合图像的中心位置，并将该中心位置确定为人眼图像中瞳孔中心位置，并且可以对人眼图像进行处理之后得到的灰度图像基于斑点识别算法获取灰度值较高的区域为普尔钦斑，该普尔钦斑的所处的位置即为光斑，或者称为“反射点”位置。根据反射点位置和角膜曲率半径，得到角膜曲率中心，根据瞳孔中心位置、角膜曲率中心和预设的眼部光轴与视轴的夹角，确定出人眼的注视点位置信息。可以理解，在其他实施例中，也可以采用其他方式确定出人眼的注视点位置信息。本申请对此不作限定。
步骤806：计算机200根据得到的人眼的注视点位置信息，基于注视点渲染技术得到子图像A和子图像B。具体地，计算机200的GPU 202获取到待处理图像和人眼注视点位置信息后，通过顶点变换计算，像素点的填充计算等进行图像渲染，得到对应于注视点位置区域的子图像A和对 应于注视点以外的区域的子图像B。
步骤808：计算机200将子图像A和子图像B按照设定的行比例进行交错整合。以确定GPU202在进行数据传送时，其中子图像A和子图像B中每一行像素点的排列顺序，使得GPU 202在发送图像数据的时候，无需将子图像A完全发送完再发送子图像B，进而使得VR眼镜100的DDIC104中的缓存器可以只缓存子图像A的部分行，降低显示时延，并且可以减小缓存器的大小。
其中，设定的行比例(记为X)可以是根据需要设定的任意的数值。例如，在一些实施例中，计算机200的GPU 202渲染得到如图9(a)所示的分辨率均为M*N子图像A和子图像B。GPU 202将子图像A和子图像B按照X＝3的行比例，对子图像A和子图像B进行交错整合。例如，在一些实施例中，子图像A和子图像B的分辨率均为1280*1280，即子图像A和子图像B均有1280行像素点和1280列像素点，按照从上到下的顺序将子图像A的1280行像素点记为a1至a1280行；将子图像B的1280行像素点记为b1至b1280行。若将子图像A和子图像B按照3的比例进行交错整合，则整合后的顺序为：a1、a2、a3、b1、a4、a5、a6、b2、a7、a8、a9、b3等以此类推。
在一些实施例中，当人眼注视点位置在屏幕上方时，按照如图9(b)所示的整合方式，将子图像A中的每3行像素点和子图像B的每一行像素点进行整合。在一些实施例中，当人眼注视点位置在屏幕中心时，按照如图9(c)所示的整合方式，将子图像A中的每3行像素点和子图像B的每一行像素点进行整合。可以理解的是，由于在图9(a)所示的实施例中，子图像A和子图像B的分辨率均为M*N，即分辨率相同。因此在按照X＝3的行比例对子图像A和子图像B进行整合时，子图像A的像素点先被排列完毕，可以使用空白与子图像B中未被排列完毕的部分进行整合。
步骤810：计算机200将整合后的图像发送给VR眼镜100。
例如，在一些实施例中，计算机200通过无线局域网、蓝牙等方式将按照设定行比例进行整合后的子图像A和子图像B发送给VR眼镜100。以供VR眼镜100进行图像显示。可以理解的是，计算机200将整合后的子图像A和子图像B发送给VR眼镜100时，发送的图像顺序可以是例如图9(b)所示的a1、a2、a3、b1、a4、a5、a6、b2、a7、a8、a9、b3、a10、a11、a12、b4，以此类推。子图像A被发送完后，则是3行空白、b5、3行空白、b6、3行空白、b7、3行空白、b8、3行空白、b9、3行空白、b10、3行空白、b11、3行空白、b12，即直至子图像B的所有内容被发送完毕。
步骤812：VR眼镜100接收整合后的图像，并将其中对应于子图像A的部分行进行缓存。以供VR眼镜100进行后续处理并显示。例如，在如图9(b)所示的组合方式中，VR眼镜100接收图像的顺序是a1、a2、a3、b1、a4、a5、a6、b2、a7、a8、a9、b3、a10、a11、a12、b4，子图像A的所有行被接收完后，则是3行空白、b5、3行空白、b6、3行空白、b7、3行空白、b8、3行空白、b9、3行空白、b10、3行空白、b11、3行空白、b12，直至子图像B的所有行被接收完毕。每次写入VR眼镜100中DDIC 104的缓存器数据为子图像A的部分行，而非A的全部。例如，DDIC 104的缓存器中当前缓存的内容为a1、a2、a3，a1、a2、a3显示完毕后，下一次缓存的内容为a4、a5、a6，a4、a5、a6显示完毕后，下一次缓存的内容为a7、a8、a9，以此类推。
步骤814：VR眼镜100将子图像B中与当前缓存的子图像A的部分行对应的一行像素点进行放大。
例如，在一些实施例中，DDIC 104的缓存器当前缓存的内容为a1、a2、a3，则VR眼镜100 接收到子图像B中与之对应的b1后，即可采用插值算法，例如最近邻插值、双线性插值、高阶插值算法等，对子图像B的b1行的像素点进行像素填充(即放大)。需要说明的是，子图像B中每一行的放大倍数和前述设定的行比例X的数值相同。例如，在图9(a)所示的实施例中，子图像A和子图像B的分辨率均为M*N，VR眼镜100的显示屏107的分辨率为H*V，子图像B被放大的倍率为Y，其中Y可以为1.33、2、2.5、2.85、3、4等。则每a行子图像A和每b行子图像B的行比例X(即a：b)与子图像B在VR眼镜100被放大的倍率为Y相等。且a和b都为整数，一般为了最小化缓存器的大小，a和b取最小整数。例如，在一些实施例中，若显示屏107的分辨率H*V＝3840*3840，子图像B被放大的倍率Y等于3；则M＝1280，N＝1280，a＝3，b＝1或a＝6，b＝2。以此类推，若H*V＝3840*3840，X等于2.5；则M＝1536，N＝1536，a＝5，b＝2或a＝10，b＝4。
步骤816：VR眼镜100将子图像B中被放大的行像素点与子图像A中与之对应的部分行进行组合，以使子图像B与子图像A最终组合成一帧完整的图像。在一些实施例中，VR眼镜的DDIC104可以根据人眼注视点位置信息，将子图像A覆盖到放大后的子图像B中的与人眼注视点位置信息相对应的位置(例如矩形覆盖，或者处理成圆形再覆盖)，得到组合后的图像。再对组合后的图像的拼接边界进行平滑融合处理，例如采用低通滤波等方法对组合后的图像的边界区域进行平滑融合处理，以得到最终要显示的图像。例如，当人眼注视点位置位于显示屏107的上方时，最终组合成如图9(d)所示的横向分辨率为M*X，纵向分辨率为N*X的图像。又例如，当人眼注视点位置位于显示屏107的中心时，最终组合成如图9(e)所示的横向分辨率为M*X，纵向分辨率为N*X的图像。
此外，可以理解的是，子图像A的位置随着人眼注视点的位置的变化而变化。并且，由于子图像A未被放大，只放大了子图像B，因此，最终呈现的图像是人眼视觉中心处的图像(即子图像A)清晰，而视觉中心以外的图像(即子图像B)较为模糊。由于人眼对视觉中心以外的图像的清新度不敏感，因此可以保证用户的视觉体验的同时，减少数据计算量。
步骤818：VR眼镜100的时序控制电路控制列驱动电路和行驱动电路，使像素阵列显示组合后的图像。例如，在一些实施例中，可以通过VR眼镜的DDIC 104中的时序控制电路控制DDIC 104的行驱动电路选通显示屏107的像素阵列的每一行的像素点，然后通过DDIC 104的列驱动电路将组合后每一行像素点信息在显示屏107上进行显示。当用户通过VR眼镜100的镜片看VR眼镜100的显示屏107时，由于用户的左、右眼看到的图像是各自独立分开的，加上人眼视觉暂留的生理特性，就可以看到三维立体图像。
图10示出了图8所示的实施例中待处理图像的显示时序图。当计算机100的GPU 202开启垂直同步后，GPU 202会等待VR眼镜100的V-Sync信号发出后，才进行新的一帧渲染，以解决画面撕裂现象，增加画面流畅度。参考图10可见，当DDIC 104的缓存器完成对子图像A的部分行的缓存后，只要一接收到子图像B的一行，即可将子图像B的该行进行放大后与子图像A中与之对应的一行进行组合后进行显示。相比较于图4和图5所示的方案，大大降低了显示时延，且降低了对DDIC 104的缓存器的容量的要求，减小DDIC 104的尺寸。
为了解决对DDIC 104中的缓存器容量要求较高的问题，在本申请的一些实施例中，可以在VR眼镜100对接收到的子图像A和子图像B分别进行不同倍率的压缩后再进行缓存。如此可以大大减小缓存器的缓存压力，降低对缓存器的容量要求。
以下，将参考图11和图12，解释根据本申请的另一些实施例提供的图像的显示方法。
具体地，参考图11和图12，在图11所示的实施例中，示出了本申请提供的图像的显示方法中待处理图像的显示路径。其中，数据源包括待处理图像，以及计算机200接收到的VR眼镜100的摄像头拍摄的包括人眼的图像。例如，在一些实施例中，针对用户图片预览场景，待处理图像为计算机200上的图像数据库中存储的图像。CPU 201根据包括人眼的图像进行计算得到人眼注视点位置信息。GPU 202基于注视点渲染技术，根据人眼注视点位置信息，对待处理图像进行分区域渲染，得到子图像A和子图像B。渲染的过程与前述渲染的过程类似，在此不再赘述。计算机200将子图像A和子图像B发送给VR眼镜100，VR眼镜100对接收到的子图像A和子图像B分别进行压缩后再进行缓存，其中，子图像B的压缩倍率大于子图像A的压缩倍率。然后VR眼镜100将从缓存器中读取缓存的被压缩的子图像A和子图像B进行解压，再将解压后的子图像B进行放大，与子图像A进行组合后进行图像显示。由于VR眼镜100对接收到的子图像A和子图像B分别进行不同倍率的压缩后再进行缓存，并且子图像B的压缩倍率大于子图像A的压缩倍率。如此可以大大减小缓存器的缓存压力，降低对缓存器的容量要求，减小DDIC 104的尺寸。
下面对图12所示的待处理图像的显示过程进行详细描述。具体地，如图12所示，待处理图像的显示过程包括：
步骤1200：VR眼镜100和计算机200建立连接。例如，用户想要通过VR眼镜100阅览图片，可以将VR眼镜100和计算机200通过数据线、无线局域网、蓝牙等方式进行连接，以使VR眼镜100和计算机200进行通信。可以理解，在其他实施例中，VR眼镜100和计算机200也可以采用其他方式建立连接，不限于上述方法。
步骤1202：VR眼镜100将摄像头拍摄到的包括人眼的图像发送给计算机200。以使计算机200根据人眼图像进行计算分析。例如通过红外摄像头实时获取包括人眼的图像，然后将红外摄像头拍摄包括人眼的图像发送给计算机200。
步骤1204：计算机200根据接收到的包括人眼的图像进行计算，得到人眼的注视点位置信息。计算方法与步骤804类似，在此不再赘述。
步骤1206：计算机200根据得到的人眼的注视点位置信息，基于注视点渲染技术得到子图像A和子图像B。渲染过程与步骤806类似，在此不再赘述。
步骤1208：计算机200将子图像A和子图像B发送给VR眼镜100。例如，在一些实施例中，计算机200通过无线局域网、蓝牙等方式将子图像A和子图像B发送给VR眼镜100。以供VR眼镜100进行图像显示。
可以理解的是，在一些实施例中，计算机200还可以将子图像A和子图像B按照类似于上述步骤808中的方式将子图像A和子图像B按照设定的行比例进行交错整合，得到整合后的图像，再将整合后的图像发送给VR眼镜100。
步骤1210：VR眼镜100接收子图像A和子图像B，并分别将子图像A和子图像B压缩后进行缓存。由于人眼对视觉中心以外的图像质量不敏感，因此，为了降低对缓存器的容量的要求，可以使子图像B的压缩倍率大于子图像A的压缩倍率。例如，子图像B的压缩倍率为100，子图像A的压缩倍率为10。在一些实施例中，可以对子图像A和子图像B图像通过块划分，预测，变换，量化，算数编码等进行压缩。
步骤1212：VR眼镜100将从缓存器中读取的压缩后的子图像A和子图像B进行解压。例如，在一些实施例中，可以通过图像解压算法对压缩后的子图像A和子图像B进行解压。
步骤1214：VR眼镜100将解压后的子图像B进行放大。例如，在一些实施例中，为了实现子图 像B的满屏显示，可以使用插值算法，例如最近邻插值、双线性插值、高阶插值算法等，使子图像B和显示屏107的分辨率一致。
步骤1216：VR眼镜100将放大后的子图像B与子图像A进行组合。以使子图像B与子图像A组合成一帧完整的图像。组合方式与步骤816类似，在此不再赘述。
步骤1218：VR眼镜100的时序控制电路控制列驱动电路和行驱动电路，使像素阵列显示组合后的图像。
另外，为了解决对DDIC 104中的缓存器容量要求较高的问题，在本申请的一些实施例中，VR眼镜100可以对接收到的子图像A全部进行缓存，子图像B的第一行到达后，直接将其进行放大，再与子图像A进行组合后进行显示。由于VR眼镜100的DDIC 104中的缓存器要对子图像A的全部数据进行缓存后，只需要实时缓存子图像B的其中一行即可，如此可以大大减小缓存器的缓存压力，降低对缓存器的容量要求。
需要说明的是，在一些实施例中，VR眼镜100的显示屏107的水平扫描频率需要加倍。例如，VR眼镜100的显示屏107的帧率为90赫兹，理论上VR眼镜100的显示屏107在11毫秒内将每一行的像素点亮即可，但在本申请的一些实施例中，显示屏107需要在5.5毫秒内将每一行的像素点亮。
以下，将参考图13至图15，解释根据本申请的另一些实施例提供的显示方法。
具体地，首先参考图13和图14，在图13所示的实施例中，示出了本申请提供的图像的显示方法中待处理图像的显示路径。其中，数据源包括待处理图像，以及计算机200接收到的VR眼镜100的摄像头拍摄的包括人眼的图像。例如，在一些实施例中，针对用户玩游戏的场景，待处理图像可以为计算机200上安装的游戏应用程序的游戏画面。CPU 201根据包括人眼的图像进行计算得到人眼注视点位置信息。GPU 202基于注视点渲染技术，根据人眼注视点位置信息，对待处理图像进行分区域渲染，得到子图像A和子图像B。渲染的过程与前述渲染的过程类似，在此不再赘述。计算机200将子图像A和子图像B发送给VR眼镜100，VR眼镜100对接收到的子图像A进行缓存，只要接收到子图像B的其中一行，即可将该行图像直接进行放大，与子图像A进行组合后进行图像显示。由于VR眼镜100的DDIC 104中的缓存器要对子图像A的全部数据进行缓存后，只需要实时缓存子图像B的其中一行即可，如此可以大大减小缓存器的缓存压力，降低对缓存器的容量要求。
下面对图14所示的待处理图像的显示过程进行详细描述，具体地，如图14所示，待处理图像的显示过程包括：
步骤1400：VR眼镜100和计算机200建立连接。例如，用户想要通过VR眼镜100阅览图片，可以将VR眼镜100和计算机200通过数据线、无线局域网、蓝牙等方式进行连接，以使VR眼镜100和计算机200进行通信。可以理解，在其他实施例中，VR眼镜100和计算机200也可以采用其他方式建立连接，不限于上述方法。
步骤1402：VR眼镜100将摄像头拍摄到的包括人眼的图像发送给计算机200。以使计算机200根据人眼图像进行计算分析。例如通过红外摄像头实时获取包括人眼的图像，然后将红外摄像头拍摄包括人眼的图像发送给计算机200。
步骤1404：计算机200根据接收到的包括人眼的图像进行计算，得到人眼的注视点位置信息。计算方法与步骤804类似，在此不再赘述。
步骤1406：计算机200根据得到的人眼的注视点位置信息，基于注视点渲染技术得到子图像A和子图像B。渲染过程与步骤806类似，在此不再赘述。
步骤1408：计算机200将子图像A和子图像B发送给VR眼镜100。例如，在一些实施例中，计 算机200通过无线局域网、蓝牙等方式将子图像A和子图像B发送给VR眼镜100。以供VR眼镜100进行图像显示。
可以理解的是，在一些实施例中，计算机200还可以将子图像A和子图像B按照类似于上述步骤808中的方式将子图像A和子图像B按照设定的行比例进行交错整合，得到整合后的图像，再将整合后的图像发送给VR眼镜100。
步骤1410：VR眼镜100接收子图像A和子图像B，并将子图像A进行缓存完毕后，缓存子图像B的其中一行。例如，在一些实施例中，在子图像A缓存完毕后，缓存子图像B的第一行。由于VR眼镜100的DDIC 104中的缓存器要对子图像A的全部数据进行缓存后，只需要实时缓存子图像B的其中一行即可，如此可以大大减小缓存器的缓存压力，降低对缓存器的容量要求。
步骤1412：当VR眼镜100缓存完子图像B的一行后，VR眼镜100将该行进行放大。例如，使用插值算法将写入的子图像B的一行放大3倍。
步骤1414：VR眼镜100将放大后的子图像B与子图像A进行组合。组合方式与步骤816类似，在此不再赘述。
步骤1416：VR眼镜100的时序控制电路控制列驱动电路和行驱动电路，使像素阵列显示组合后的子图像A与子图像B。例如，在一些实施例中，可以通过VR眼镜的DDIC 104中的时序控制电路控制DDIC 104的行驱动电路选通显示屏107的像素阵列的每一行的像素点，然后通过DDIC 104的列驱动电路将DDIC 104的缓存中缓存的子图像A与子图像B输出在显示屏107上进行显示。
图15示出了图14所示的实施例中待处理图像的显示时序图。当计算机100的GPU 202开启垂直同步后，GPU 202会等待VR眼镜100的V-Sync信号发出后，才进行新的一帧渲染。参考图15可见，当DDIC 104的缓存器完成对子图像A的缓存后，只要一接收到子图像B的一行，即可将子图像B的该行进行放大后与子图像A中与之对应的一行进行组合后进行显示。相比较于图4和图5所示的方案，降低了对DDIC 104的缓存器的容量的要求，减小DDIC 104的尺寸。
图16根据本申请的一些实施例，提供了一种图像的显示装置1600，具体地，包括：
获取模块1602，用于获取待处理图像以及用户的注视点位置信息，其中注视点位置信息用于描述用户的视觉中心落在电子设备的屏幕上的位置；
第一处理模块1604，用于基于获取的待处理图像以及注视点位置信息，对待处理图像进行渲染，得到第一图像和第二图像，其中，第一图像对应于待处理图像中的第一区域，第二图像对应于待处理图像中的第二区域，第一区域为待处理图像中用户的注视点位置预设范围内的区域，第二区域为待处理图像中第一区域以外的区域；
第二处理模块1606，用于将第一图像中的每一行像素点以及第二图像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像；
第三处理模块1608，用于将整合后的图像中对应于第一图像的每一行像素点中的至少部分行像素点进行缓存，将整合后的图像中对应于第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的部分行像素点和被缓存的部分行像素点进行组合，得到组合后的图像；以及
显示模块1610，用于显示组合后的图像。
可以理解，图16所示的图像的显示装置1600与本申请提供的图像的显示方法相对应，以上关于本申请的提供的图像的显示方法的具体描述中的技术细节依然适用于图16所示的图像的显示装置1600，具体描述请参见上文，在此不再赘述。
本申请公开的机制的各实施例可以被实现在硬件、软件、固件或这些实现方法的组合中。本申请的 实施例可实现为在可编程系统上执行的计算机程序或程序代码，该可编程系统包括至少一个处理器、存储系统(包括易失性和非易失性存储器和/或存储元件)、至少一个输入设备以及至少一个输出设备。
可将程序代码应用于输入指令，以执行本申请描述的各功能并生成输出信息。可以按已知方式将输出信息应用于一个或多个输出设备。为了本申请的目的，处理系统包括具有诸如例如数字信号处理器(Digital Signal Processor，DSP)、微控制器、专用集成电路(Application Specific Integrated Circuit，ASIC)或微处理器之类的处理器的任何系统。
程序代码可以用高级程序化语言或面向对象的编程语言来实现，以便与处理系统通信。在需要时，也可用汇编语言或机器语言来实现程序代码。事实上，本申请中描述的机制不限于任何特定编程语言的范围。在任一情形下，该语言可以是编译语言或解释语言。
在一些情况下，所公开的实施例可以以硬件、固件、软件或其任何组合来实现。所公开的实施例还可以被实现为由一个或多个暂时或非暂时性机器可读(例如，计算机可读)存储介质承载或存储在其上的指令，其可以由一个或多个处理器读取和执行。例如，指令可以通过网络或通过其他计算机可读介质分发。因此，机器可读介质可以包括用于以机器(例如，计算机)可读的形式存储或传输信息的任何机制，包括但不限于，软盘、光盘、光碟、只读存储器(CD-ROMs)、磁光盘、只读存储器(Read Only Memory，ROM)、随机存取存储器(Random access memory，RAM)、可擦除可编程只读存储器(Erasable Programmable Read Only Memory，EPROM)、电可擦除可编程只读存储器(Electrically Erasable Programmable Read-Only Memory，EEPROM)、磁卡或光卡、闪存、或用于利用因特网以电、光、声或其他形式的传播信号来传输信息(例如，载波、红外信号数字信号等)的有形的机器可读存储器。因此，机器可读介质包括适合于以机器(例如计算机)可读的形式存储或传输电子指令或信息的任何类型的机器可读介质。
在附图中，可以以特定布置和/或顺序示出一些结构或方法特征。然而，应该理解，可能不需要这样的特定布置和/或排序。而是，在一些实施例中，这些特征可以以不同于说明性附图中所示的方式和/或顺序来布置。另外，在特定图中包括结构或方法特征并不意味着暗示在所有实施例中都需要这样的特征，并且在一些实施例中，可以不包括这些特征或者可以与其他特征组合。
需要说明的是，本申请各设备实施例中提到的各单元/模块都是逻辑单元/模块，在物理上，一个逻辑单元/模块可以是一个物理单元/模块，也可以是一个物理单元/模块的一部分，还可以以多个物理单元/模块的组合实现，这些逻辑单元/模块本身的物理实现方式并不是最重要的，这些逻辑单元/模块所实现的功能的组合才是解决本申请所提出的技术问题的关键。此外，为了突出本申请的创新部分，本申请上述各设备实施例并没有将与解决本申请所提出的技术问题关系不太密切的单元/模块引入，这并不表明上述设备实施例并不存在其它的单元/模块。
需要说明的是，在本专利的示例和说明书中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者设备所固有的要素。在没有更多限制的情况下，由语句“包括一个”限定的要素，并不排除在包括所述要素的过程、方法、物品或者设备中还存在另外的相同要素。
虽然通过参照本申请的某些优选实施例，已经对本申请进行了图示和描述，但本领域的普通技术人员应该明白，可以在形式上和细节上对其作各种改变，而不偏离本申请的精神和范围。

Claims (13)

  1. 一种图像的显示方法，用于作为显示端的电子设备，其特征在于，包括：
    获取待处理图像以及用户的注视点位置信息，其中所述注视点位置信息用于描述用户的视觉中心落在所述电子设备的屏幕上的位置；
    基于获取的所述待处理图像以及所述注视点位置信息，对所述待处理图像进行渲染，得到第一图像和第二图像，其中，所述第一图像对应于所述待处理图像中的第一区域，所述第二图像对应于所述待处理图像中的第二区域，所述第一区域为所述待处理图像中用户的注视点位置预设范围内的区域，所述第二区域为所述待处理图像中所述第一区域以外的区域；
    将所述第一图像中的每一行像素点以及所述第二图像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像；
    将所述整合后的图像中对应于所述第一图像的每一行像素点中的至少部分行像素点进行缓存，将所述整合后的图像中对应于所述第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的所述部分行像素点和被缓存的所述部分行像素点进行组合，得到组合后的图像；以及
    显示所述组合后的图像。
  2. 根据权利要求1所述的方法，其特征在于，所述第一图像和所述第二图像的分辨率相同。
  3. 根据权利要求1或2所述的方法，其特征在于，所述将所述第一图像中的每一行像素点以及所述第二图像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像包括：
    将所述第一图像中的每a行像素点为一组和所述第二图像中的每b行像素点为一组进行交错整合，得到整合后的图像，其中，a:b＝X，a>b，a和b为正整数。
  4. 根据权利要求3所述的方法，其特征在于，所述将所述整合后的图像中对应于所述第一图像的每一行像素点中的至少部分行像素点进行缓存，将所述整合后的图像中对应于所述第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的所述部分行像素点和被缓存的所述部分行像素点进行组合，得到组合后的图像包括：
    将所述整合后的图像中对应于所述第一图像的每a行像素点进行缓存，并且将所述整合后的图像中对应于所述第二图像的每b行像素点放大Y倍；
    读取当前被缓存的所述第一图像的a行像素点，并且将所述第二图像中被放大的b行像素点与读取的所述第一图像中当前被缓存的a行像素点进行组合，得到组合后的图像。
  5. 根据权利要求4所述的方法，其特征在于，X的数值等于Y的数值。
  6. 根据权利要求4所述的方法，其特征在于，所述第二图像的分辨率被放大Y倍后与所述组合后的图像的分辨率相同。
  7. 根据权利要求1或2所述的方法，其特征在于，所述将所述整合后的图像中对应于所述第一图像的每一行像素点中的至少部分行像素点进行缓存，将所述整合后的图像中对应于所述第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的所述部分行像素点和被缓存的所述部分行像素点进行组合，得到组合后的图像包括：
    将所述整合后的图像中对应于所述第一图像的每一行像素点以及所述整合后的图像中对应于所述第二图像的每一行像素点进行压缩，其中对应于所述第一图像的每一行像素点的压缩倍率大于对应于所述第二图像的每一行像素点的压缩倍率；
    将被压缩后的对应于所述第一图像的至少部分行像素点以及被压缩后的对应于所述第二图像的至少部分行像素点进行缓存；
    读取当前被缓存的被压缩后的对应于所述第一图像的至少部分行像素点以及被压缩后的对应于所述第二图像的至少部分行像素点，并且将读取的当前被缓存的被压缩后的对应于所述第一图像的至少部分行像素点以及被压缩后的对应于所述第二图像的至少部分行像素点进行解压；
    将被解压后的对应于所述第二图像的至少部分行像素点放大S倍后与被解压后的对应于所述 第一图像的至少部分行像素点进行组合，得到组合后的图像。
  8. 根据权利要求7所述的方法，其特征在于，所述第二图像的分辨率被放大S倍后与所述组合后的图像的分辨率相同。
  9. 根据权利要求1或2所述的方法，其特征在于，所述将所述整合后的图像中对应于所述第一图像的每一行像素点中的至少部分行像素点进行缓存，将所述整合后的图像中对应于所述第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的所述部分行像素点和被缓存的所述部分行像素点进行组合，得到组合后的图像包括：
    将所述整合后的图像中对应于所述第一图像的每一行像素点以及将所述整合后的图像中对应于所述第二图像中的其中一行像素点进行缓存；
    读取当前被缓存的对应于所述第一图像中的部分行像素点，以及对应于所述第二图像中的其中一行像素点，并且将读取的对应于所述第二图像中的其中一行像素点放大T倍；
    将被放大的对应于所述第二图像中的一行像素点与读取的当前被缓存的对应于所述第一图像中的部分行像素点进行组合，得到组合后的图像。
  10. 根据权利要求9所述的方法，其特征在于，所述第二图像的分辨率被放大T倍后与所述组合后的图像的分辨率相同。
  11. 一种图像的显示装置，其特征在于，包括：
    获取模块，用于获取待处理图像以及用户的注视点位置信息，其中所述注视点位置信息用于描述用户的视觉中心落在电子设备的屏幕上的位置；
    第一处理模块，用于基于获取的所述待处理图像以及所述注视点位置信息，对所述待处理图像进行渲染，得到第一图像和第二图像，其中，所述第一图像对应于所述待处理图像中的第一区域，所述第二图像对应于所述待处理图像中的第二区域，所述第一区域为所述待处理图像中用户的注视点位置预设范围内的区域，所述第二区域为所述待处理图像中所述第一区域以外的区域；
    第二处理模块，用于将所述第一图像中的每一行像素点以及所述第二图像中的每一行像素点按照设定的行比例进行交错整合，得到整合后的图像；
    第三处理模块，用于将所述整合后的图像中对应于所述第一图像的每一行像素点中的至少部分行像素点进行缓存，将所述整合后的图像中对应于所述第二图像的每一行像素点中的至少部分行像素点进行放大，将被放大的所述部分行像素点和被缓存的所述部分行像素点进行组合，得到组合后的图像；以及
    显示模块，用于显示所述组合后的图像。
  12. 一种可读介质，其特征在于，所述可读介质上存储有指令，该指令在电子设备上执行时使电子设备执行权利要求1-10中任一项所述的图像的显示方法。
  13. 一种电子设备，其特征在于，包括：
    存储器，用于存储由电子设备的一个或多个处理器执行的指令，以及
    处理器，是电子设备的处理器之一，用于执行权利要求1-10中任一项所述的图像的显示方法。
PCT/CN2022/074018 2021-02-03 2022-01-26 图像的显示方法、装置、可读介质和电子设备 WO2022166712A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110151925.6A CN114860062A (zh) 2021-02-03 2021-02-03 图像的显示方法、装置、可读介质和电子设备
CN202110151925.6 2021-02-03

Publications (1)

Publication Number Publication Date
WO2022166712A1 true WO2022166712A1 (zh) 2022-08-11

Family

ID=82623551

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/074018 WO2022166712A1 (zh) 2021-02-03 2022-01-26 图像的显示方法、装置、可读介质和电子设备

Country Status (2)

Country Link
CN (1) CN114860062A (zh)
WO (1) WO2022166712A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024093835A1 (zh) * 2022-11-01 2024-05-10 华为技术有限公司 一种图像数据的处理方法及相关设备

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116382549B (zh) * 2023-05-22 2023-09-01 昆山嘉提信息科技有限公司 基于视觉反馈的图像处理方法及装置

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110809889A (zh) * 2018-06-04 2020-02-18 京东方科技集团股份有限公司 处理具有减小的传输带宽的图像数据的方法和显示设备
US20200412983A1 (en) * 2018-03-08 2020-12-31 Sony Interactive Entertainment Inc. Electronic device, head-mounted display, gaze point detector, and pixel data readout method

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200412983A1 (en) * 2018-03-08 2020-12-31 Sony Interactive Entertainment Inc. Electronic device, head-mounted display, gaze point detector, and pixel data readout method
CN110809889A (zh) * 2018-06-04 2020-02-18 京东方科技集团股份有限公司 处理具有减小的传输带宽的图像数据的方法和显示设备

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2024093835A1 (zh) * 2022-11-01 2024-05-10 华为技术有限公司 一种图像数据的处理方法及相关设备

Also Published As

Publication number Publication date
CN114860062A (zh) 2022-08-05

Similar Documents

Publication Publication Date Title
US11961431B2 (en) Display processing circuitry
US20200342577A1 (en) Virtual, augmented, and mixed reality systems and methods
CN110322818B (zh) 显示装置及操作方法
US11303875B2 (en) Split rendering between a head-mounted display (HMD) and a host computer
WO2022166712A1 (zh) 图像的显示方法、装置、可读介质和电子设备
US10572764B1 (en) Adaptive stereo rendering to reduce motion sickness
AU2016304612B2 (en) Naked-eye 3D display method and system thereof
WO2020140758A1 (zh) 图像显示方法、图像处理方法和相关设备
JP2019197224A (ja) 映像表示システム
US20200029057A1 (en) Systems and methods for correcting color separation in field-sequential displays
JP2016116162A (ja) 映像表示装置、映像表示システム、及び映像表示方法
US11892637B2 (en) Image generation apparatus, head-mounted display, content processing system, and image display method
KR20200079162A (ko) 실감형 콘텐츠 제공 장치 및 방법
US20220172440A1 (en) Extended field of view generation for split-rendering for virtual reality streaming
US20240185380A1 (en) Methods to improve the perceptual quality of foveated rendered images
CN115185083B (zh) 一种ar头戴显示器自适应刷新率和分辨率渲染方法
KR20220157323A (ko) 포비에이티드 렌더링을 이용한 원근 보정 벡터 그래픽
CN115174883A (zh) 基于平面oled屏幕的主动式3d效果显示方法、系统及设备
KR20210125403A (ko) 손실 없는 데이터 감소를 통해 통신 부하를 감소하기 위한 방법, 장치 및 저장 매체
US9137522B2 (en) Device and method for 3-D display control
WO2024093835A1 (zh) 一种图像数据的处理方法及相关设备
US12034786B2 (en) Image processing device, image data transfer device, and image generation method
JP7365183B2 (ja) 画像生成装置、ヘッドマウントディスプレイ、コンテンツ処理システム、および画像表示方法
US20240212090A1 (en) Dual detail encoding in a distributed system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22748991

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22748991

Country of ref document: EP

Kind code of ref document: A1