CN114040184A

CN114040184A - Image display method, system, storage medium and computer program product

Info

Publication number: CN114040184A
Application number: CN202111423175.XA
Authority: CN
Inventors: 鄢名扬; 张�浩; 陈丽莉; 苗京花; 李文宇; 彭金豹; 范清文; 郑超; 李治富; 王龙辉; 郝帅; 李茜; 王雪丰
Original assignee: BOE Technology Group Co Ltd; Beijing BOE Display Technology Co Ltd
Current assignee: BOE Technology Group Co Ltd; Beijing BOE Display Technology Co Ltd
Priority date: 2021-11-26
Filing date: 2021-11-26
Publication date: 2022-02-11

Abstract

The present disclosure provides an image display method, system, storage medium, and computer program product. The image display system includes: a server configured to: acquiring a first image set, wherein the first image set comprises multi-viewpoint image data matched with a plurality of viewpoints; acquiring eyeball position coordinates, and acquiring a first prediction viewpoint based on the eyeball position coordinates and a first prediction duration; selecting image data matched with the first prediction viewpoint from the first image set to generate a second image set, and sending the second image set to a display terminal; a display terminal configured to: collecting eyeball position coordinates and sending the eyeball position coordinates to the server; and receiving the second image set and displaying based on the second image set. The image display method, system, storage medium and computer program product of the present disclosure can reduce the transmission amount and processing amount of image data.

Description

Image display method, system, storage medium and computer program product

Technical Field

The present disclosure relates to the field of display technologies, and in particular, to an image display method, system, storage medium, and computer program product.

Background

With the continuous development of display technology, three-dimensional (3D) display technology is receiving more and more attention. The three-dimensional display technology can make the display picture become three-dimensional vivid. The principle is as follows: the left eye image and the right eye image with certain parallax are received by the left eye and the right eye of a person respectively, and after the two parallax images are received by the left eye and the right eye of the person respectively, image information is overlapped and fused through the brain, so that a 3D visual display effect can be constructed.

Currently, a 3D display device can realize multi-viewpoint image display. In the content acquisition and transmission links, the implementation method needs to process the data volume of multiple paths of videos at the same time, which causes great burden on network bandwidth.

Disclosure of Invention

In view of the above, the present disclosure is directed to an image display method, system, storage medium and computer program product.

Based on the above object, the present disclosure provides an image display system for naked eye 3D display, including:

a server configured to: acquiring a first image set, wherein the first image set comprises multi-viewpoint image data matched with a plurality of viewpoints; acquiring eyeball position coordinates, and acquiring a first prediction viewpoint based on the eyeball position coordinates and a first prediction duration; selecting image data matched with the first prediction viewpoint from the first image set to generate a second image set, and sending the second image set to a display terminal;

a display terminal configured to: collecting eyeball position coordinates and sending the eyeball position coordinates to the server; and receiving the second image set and displaying based on the second image set.

Optionally, the display terminal is further configured to: acquiring the eyeball position coordinate and a second predicted time length; determining a second predicted viewpoint based on the eyeball position coordinates and a second predicted duration; selecting image data matched with the second prediction viewpoint from the second image set for display;

wherein the first predicted duration is greater than the second predicted duration, the first predicted view comprising the second predicted view.

Optionally, the method further includes:

an image processing platform configured to: acquiring a third image set, the eyeball position coordinates and a third prediction duration; determining a third predicted viewpoint based on the eyeball position coordinates and the third predicted duration; selecting image data matched with the third prediction viewpoint from the third image set to be coded so as to generate the first image set, and sending the first image set to the server;

wherein the third predicted duration is greater than the first predicted duration, and the third predicted view comprises the first predicted view.

Optionally, the image processing platform is further configured to: acquiring a fourth image set and a fourth predicted duration; determining a fourth predicted viewpoint based on the eyeball position coordinates and a fourth predicted duration; selecting image data from the fourth set of images that match the fourth predicted viewpoint for image processing to generate the third set of images;

wherein the fourth predicted duration is greater than the third predicted duration, the fourth predicted view comprising the third predicted view.

Optionally, the method further includes:

an image acquisition device including an image acquisition unit for acquiring multi-viewpoint image data; is configured to: acquiring the eyeball position coordinate and a fifth prediction duration, and determining a fifth prediction viewpoint based on the eyeball position coordinate and the fifth prediction duration; controlling an image acquisition unit matched with the fifth prediction viewpoint to acquire images so as to generate a fourth image set, and sending the fourth image set to the image processing platform;

wherein the fifth predicted duration is greater than the fourth predicted duration, the fifth predicted view comprising the fourth predicted view.

Optionally, the display terminal is further configured to: acquiring the eyeball position coordinates of a user in real time; determining an eyeball motion vector based on the eyeball position coordinate of the current collection time and the eyeball position coordinate of the previous collection time in the eyeball position coordinates;

the server further configured to: receiving eyeball position coordinates and eyeball motion vectors at the current acquisition time, which are sent by the display terminal; determining a first eyeball position prediction coordinate according to the eyeball position coordinate at the current acquisition time, the first prediction time length, the eyeball tracking acquisition interval time length and the eyeball motion vector, and determining the first prediction viewpoint according to the first eyeball position prediction coordinate.

Optionally, the method further includes:

the server further configured to: determining monocular eyeball position prediction coordinates according to the eyeball position coordinates, and acquiring monocular prediction viewpoints according to the monocular eyeball position prediction coordinates; and acquiring the pupil distance of the two eyes, determining a predicted viewpoint of the other eye according to the pupil distance of the two eyes and the single-eye predicted viewpoint, and determining the first predicted viewpoint according to the single-eye predicted viewpoint and the predicted viewpoint of the other eye.

Optionally, the server is further configured to: determining an eyeball motion track according to the eyeball position coordinates and the eyeball position prediction coordinates at the current acquisition moment, and acquiring a viewpoint which is coincident with the eyeball motion track as the first prediction viewpoint.

Optionally, the display terminal is further configured to: selecting image data matched with the first prediction viewpoint from the second image set to establish a frame buffer; and detecting the time stamp of each frame of image data in the frame buffer to determine the image data to be displayed.

The present disclosure also provides an image display method, including:

the display terminal collects eyeball position coordinates and sends the eyeball position coordinates to the server;

a server acquires a first image set, wherein the first image set comprises multi-viewpoint image data matched with a plurality of viewpoints;

the server acquires eyeball position coordinates, and acquires a first prediction viewpoint matched with the prediction display time based on the eyeball position coordinates and the first prediction duration;

selecting image data matched with the first prediction viewpoint from the first image set to generate a second image set, and sending the second image set to a display terminal;

and the display terminal receives the second image set and displays the second image set.

Optionally, the method further includes:

the playing device acquires the eyeball position coordinates and a second predicted duration;

the playing device determines a second prediction viewpoint based on the eyeball position coordinate and a second prediction duration;

the playing device selects image data matched with the second prediction viewpoint from the second image set to display;

Optionally, the method further includes:

the image processing platform acquires a third image set, the eyeball position coordinates and a third prediction duration;

the image processing platform determines a third prediction viewpoint based on the eyeball position coordinates and the third prediction duration;

the image processing platform selects image data matched with the third predicted duration from the third image set to encode so as to generate the first image set, and sends the first image set to the server;

Optionally, the method further includes:

the image processing platform acquires a fourth image set and a fourth predicted duration;

the image processing platform determines a fourth prediction viewpoint based on the eyeball position coordinates and a fourth prediction duration;

the image processing platform selects image data matched with the fourth prediction viewpoint from the fourth image set to perform image processing to generate the third image set;

Optionally, the method further includes:

the image acquisition device acquires the eyeball position coordinates and a fifth prediction time length, and determines a fifth prediction viewpoint based on the eyeball position coordinates and the fifth prediction time length;

the image acquisition device controls an image acquisition unit matched with the fifth prediction viewpoint to acquire images so as to generate a fifth image set;

The present disclosure provides a non-transitory computer-readable storage medium containing a computer program which, when executed by one or more processors, causes the processors to perform a method as in any one of the above.

The present disclosure provides a computer program product comprising computer program instructions which, when run on a computer, cause the computer to perform the method of any one of the above.

As can be seen from the foregoing, the image display method, system, storage medium, and computer program product provided by the present disclosure collect eyeball position coordinates through a display terminal, and send the eyeball position coordinates to the server; the server predicts a first prediction viewpoint corresponding to the display time of the image to be displayed based on the eyeball position coordinates; and then the server selects image data matched with the first prediction viewpoint from the first image set based on the first prediction viewpoint to generate a second image set and sends the second image set to the display terminal, and the display terminal only needs to display based on the images in the second image set, and does not need to pull all the image data in the first image set to the display terminal to perform operations such as pixel lighting, image rendering, map arrangement display and the like, so that the transmission quantity and the processing quantity of the image data can be reduced.

Drawings

In order to more clearly illustrate the technical solutions in the present disclosure or related technologies, the drawings needed to be used in the description of the embodiments or related technologies are briefly introduced below, and it is obvious that the drawings in the following description are only embodiments of the present disclosure, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.

Fig. 1 is a schematic diagram of a naked-eye 3D display principle of a lenticular lens according to an embodiment of the present disclosure;

fig. 2 is a block diagram showing a configuration of an image display system according to the related art;

FIG. 3 is a block diagram of an image display system according to an embodiment of the disclosure;

fig. 4 is a block diagram of a structure of an image display system applied to the live broadcast field according to an embodiment of the present disclosure;

FIG. 5 is a block diagram of another embodiment of an image display system according to the present disclosure;

FIG. 6 is a block diagram of another configuration of an image display system according to an embodiment of the present disclosure;

FIG. 7 is a schematic diagram of predicted durations, according to an embodiment of the present disclosure;

FIG. 8 is a schematic diagram illustrating eyeball position prediction according to an embodiment of the present disclosure;

FIG. 9 is a diagram illustrating image synchronization according to an embodiment of the present disclosure;

FIG. 10 is a flowchart illustrating an image display method according to an embodiment of the disclosure;

fig. 11 is another schematic flow chart of an image display method according to an embodiment of the disclosure.

Detailed Description

For the purpose of promoting a better understanding of the objects, aspects and advantages of the present disclosure, reference is made to the following detailed description taken in conjunction with the accompanying drawings.

It is to be noted that technical terms or scientific terms used in the embodiments of the present disclosure should have a general meaning as understood by those having ordinary skill in the art to which the present disclosure belongs, unless otherwise defined. The use of "first," "second," and similar terms in the embodiments of the disclosure is not intended to indicate any order, quantity, or importance, but rather to distinguish one element from another. The word "comprising" or "comprises", and the like, means that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items. The terms "connected" or "coupled" and the like are not restricted to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", and the like are used merely to indicate relative positional relationships, and when the absolute position of the object being described is changed, the relative positional relationships may also be changed accordingly.

The naked eye 3D display technology can realize the stereoscopic vision effect without using external tools such as polarized glasses and the like. Currently, the naked-eye 3D display technology is mainly represented by light barrier technology, lenticular lens technology, and distributed optical matrix technology. The lenticular lens technology has no influence on the brightness of the display screen, has good imaging stereoscopy, and is a naked eye 3D display technology which is widely applied at present.

As shown in fig. 1, which is a schematic diagram of a naked-eye 3D display principle of a lenticular lens, pixels at different positions on a display panel are refracted and split by the lenticular lens, and light paths are changed to form different visual areas in space, so that when two eyes of a person are located in a correct visual area (i.e., a left eye receives a left viewpoint image and a right eye also receives a right viewpoint image), the person can feel a stereoscopic sensation.

The traditional lenticular stereoscopic display has dozens (40-60) of viewpoints, and in the content acquisition, manufacturing and transmission links, dozens of paths of video data are required to be processed simultaneously. When the method is applied to live broadcast and video call application scenes, as shown in fig. 2, dozens of paths of video stream data cause great burden on a system and network bandwidth from video image shooting of an image acquisition device, processing and encoding of images by a processing platform, storage of a server or a live broadcast platform and pull stream display of a playing device. Also, the current home network environment is not sufficient to support pull streaming of 60 viewpoints.

Eye Tracking technology (Eye Tracking) is a scientific application technology applied to a near-Eye display device for researching eyeball motion information, eyeball motion information is analyzed by collecting images of human eyes, the Eye gazing direction is determined based on the eyeball motion information, and then the viewpoint of the human eyes on a display screen is determined; when the user views the image data corresponding to two different viewpoints with both eyes, the user can feel stereoscopic impression. When the head or the eyeball of the user moves, the pupil position changes relative to the display device, and therefore the viewpoint corresponding to the human eye changes accordingly.

In view of this, the embodiments of the present disclosure provide an image display system applied to a naked-eye 3D display, which combines a lenticular lens naked-eye 3D display with an eye tracking technology, and can reduce transmission and processing of video stream data, thereby reducing the burden on the system and network bandwidth.

As shown in fig. 3, the image display system includes a server 1 and a display terminal 2. In some embodiments, the server 1 may be a live platform,

wherein the server 1 is configured to: acquiring a first image set, wherein the first image set comprises multi-viewpoint image data matched with a plurality of viewpoints; acquiring eyeball position coordinates, and acquiring a first prediction viewpoint matched with a prediction display moment based on the eyeball position coordinates and a first prediction duration; and selecting image data matched with the first prediction viewpoint from the first image set to generate a second image set, and sending the second image set to a display terminal.

The display terminal 2 is configured to: collecting eyeball position coordinates and sending the eyeball position coordinates to the server; and receiving the second image set and displaying based on the second image set.

In this embodiment, the display terminal 2 may be a device that needs to display content, such as a mobile phone, a computer, a television, a display, a vehicle-mounted display device, a touch-control all-in-one machine, or a conference large screen, and the display terminal 2 includes a display screen. The display terminal 2 can receive the push stream data of the server 1 or obtain the pull stream data from the server to play.

The display terminal 2 may be equipped with an eye tracking device such as a camera, an eye tracker, or the like. Taking a camera as an example, the eyeball tracking device can detect a face image in front of the display screen through a face detection algorithm, then separate information of human eyes according to the detected face image, and finally calculate coordinate information of the human eyes in space relative to the camera by combining parameters of the camera, namely calculate the positions of the human eyes in the space coordinate system. Specifically, the face detection may adopt algorithms such as a template matching technique, AdaBoost-based face detection, and deep learning-based face detection. And finding the interesting positions of human eyes on the human face according to the biological characteristics of the human eyes to reduce the detection range of the human eyes, and then detecting the positions of the human eyes in the image by using the characteristics of the human eyes, for example, a trained xml file provided by an opencv library official party can be adopted. Then, the eyeball position coordinates of the user are obtained, taking the monocular camera for distance measurement as an example, the imaging model of the camera is approximated to a pinhole imaging model, and then an N-point perspective projection model is constructed and solved (PNP solution) according to the relation between the image plane feature point (namely the two-dimensional coordinates of the human eye feature point in the image) and the object physical plane feature point (namely the three-dimensional coordinates of the human eye feature point in the space), so that the positions of human eyes in a camera coordinate system (namely the space coordinate system) are calculated, and the eyeball position coordinates are determined. After obtaining the eyeball position coordinates, the display terminal 2 may transmit the eyeball position coordinates to the server 1.

In this embodiment, the server 1 may be a live platform. In order to realize naked eye 3D display on the display terminal 2, the server 1 needs to include a first image set for realizing naked eye 3D display. Wherein the first image set includes multi-view image data matched with a plurality of views, the image data corresponding to each view may form video stream data corresponding to the view, and when the user is at the corresponding view at a position before the display terminal 2, the video stream data corresponding to the view may be viewed through the display terminal 2.

The server 1 obtains a first predicted time length after receiving the eyeball position coordinate sent by the display terminal 2, and obtains a first predicted viewpoint based on the eyeball position coordinate and the first predicted time length, wherein the first predicted viewpoint is a viewpoint corresponding to the predicted position of the user eyeball when displaying a certain frame of image data at a certain display moment in the future. Wherein the number of views of the first predicted view is greater than 2.

Then, the server 1 selects image data matching each viewpoint included in the first prediction viewpoint from the first image set, generates a second image set, and transmits the second image set to the display terminal 2, so that the display terminal 2 can perform layout display based on the image data in the second image set.

Wherein the second set of images is a subset of the first set of images, the second set of images including only image data matching the first predicted view; and the first set of images includes image data of other viewpoints in addition to the image data matching the first predicted viewpoint. In this way, the viewpoint prediction based on the eye tracking technology can reduce the transmission of image data that is not matched with the first predicted viewpoint when the image data is transmitted from the server 1 to the display terminal 2, and the subsequent display terminal 2 does not need to process the data when performing pixel lighting, image rendering and layout display, so that the transmission amount and the processing amount of the image data can be reduced, and the delay of data transmission can be further reduced.

The image display system of the embodiment collects the eyeball position coordinates through the display terminal and sends the eyeball position coordinates to the server; the server predicts a first prediction viewpoint corresponding to the display time of the image to be displayed based on the eyeball position coordinates; and then the server selects image data matched with the first prediction viewpoint from the first image set based on the first prediction viewpoint to generate a second image set and sends the second image set to the display terminal, and the display terminal only needs to display based on the images in the second image set, and does not need to pull all the image data in the first image set to the display terminal to perform operations such as pixel lighting, image rendering, map arrangement display and the like, so that the transmission quantity and the processing quantity of the image data can be reduced.

Alternatively, the image display system can be applied to one-to-one video application scenes, such as video calls, video conferences and the like.

Optionally, the image display system may also be applied in a one-to-many video application scenario, such as webcast. Since the webcast system is oriented to a large number of viewing users, the general home network environment is not sufficient to support 60-view pull streams. By the image display system of the embodiment, the image data matched with the first prediction viewpoint only needs to be pulled to the display terminal for displaying.

As shown in fig. 4, when only one user watches the display screen, only two-viewpoint image data corresponding to the left and right eyes of the user can be pulled to realize naked-eye 3D display. At present, the eyeball tracking device can simultaneously track two target users to the maximum extent, and at the moment, naked-eye 3D display can be realized only by drawing four viewpoint image data corresponding to the left eye and the right eye of the two users.

In some embodiments, the display terminal 2 is further configured to: acquiring the eyeball position coordinates of a user in real time; and determining an eyeball motion vector based on the eyeball position coordinate of the current collection time and the eyeball position coordinate of the previous collection time in the eyeball position coordinates.

The server 1 is further configured to: receiving eyeball position coordinates and eyeball motion vectors at the current acquisition time, which are sent by the display terminal; determining a first eyeball position prediction coordinate according to the eyeball position coordinate at the current acquisition time, the first prediction time length, the eyeball tracking acquisition interval time length and the eyeball motion vector, and determining the first prediction viewpoint according to the first eyeball position prediction coordinate.

In this embodiment, the display terminal 2 collects the eyeball position coordinates of the user in front of the display screen in real time. The eyeball position coordinates of the user comprise the eyeball position coordinates at the current collection time and the eyeball position coordinates at the previous collection time. The eyeball position coordinate at the current collection time is the eyeball position coordinate of the user when the user watches the current frame image data, the eyeball position coordinate at the previous collection time is the eyeball position coordinate of the user when the display terminal 2 last collects the image watched by the user, and the two are adjacent two collection results when the display terminal 2 carries out the eyeball position coordinate collection respectively. In the time of playing one frame of image information, the display terminal 2 can acquire eyeball position coordinates for many times; or, one or more frames of image information are played in the acquisition gap between two adjacent eyeball position coordinates, which is not limited in this embodiment.

As shown in fig. 8, an XYZ coordinate system is established with the central point of the display screen of the display terminal as the origin, and the eye at the previous acquisition timeThe coordinates of the ball position are (x)_last，y_last，z_last) The eyeball position coordinate at the current acquisition time is (x)_now，y_now，z_now) Then eyeball motion vector

Satisfies the following conditions:

then, the display terminal 2 displays the eyeball position coordinates (x) at the current acquisition time_now，y_now，z_now) And eye movement vector

To the server 2.

The server receives the eyeball position coordinate (x) of the current acquisition moment_now，y_now，z_now) And eye movement vector

Obtaining a first predicted time duration Deltat_display1And an eyeball tracking acquisition interval duration Deltat, and calculating a first eyeball position prediction coordinate (x) corresponding to the first prediction viewpoint based on the information_display1，y_display1，z_display1) Wherein the first eyeball position prediction coordinate (x)_display1，y_display1，z_display1) Satisfies the following conditions:

in this way, the first predicted viewpoint may be determined based on the determined first eye position predicted coordinates.

In some embodiments, the first predicted duration Δ t_display1The acquisition method comprises the following steps: as shown in fig. 7, assume thatAt time T0, the display terminal 2 acquires the eyeball position coordinates of the current acquisition time, and predicts that a certain frame of image data a will be displayed on the display terminal 2 at time T1 in a row map manner, where time T0 is before time T1. The frame of image data a arrives at the server 1 at time T3, and time T3 is before time T1. Thus, the first prediction period Δ t_display1＝T1-T3。

After the first image set including the frame of image data a arrives at the server 1, image data matched with the first prediction viewpoint needs to be selected from the first image set to generate a second image set including the frame of image data a, and the second image set is sent to the display terminal 2, and the display terminal 2 needs to perform layout display based on the second image set including the frame of image data a, so that the time required by the process can be predicted through the server 1 to obtain the first prediction time length Δ t_display1。

Alternatively, the first prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display1. Since the time required to realize the above-described processing is known by the system since the last frame of image data B has been displayed completely, i.e., T1 and T3 have been obtained, the first predicted duration Δ T can be obtained based on T1 and T3 of the last frame of image data B_display1. The previous frame of image data B may be an adjacent image frame to the frame of image data a, or may not be an adjacent image frame, which is not limited in this embodiment.

In other embodiments, the display terminal 2 is further configured to: acquiring the eyeball position coordinate and a second predicted time length; determining a second predicted viewpoint based on the eyeball position coordinates and a second predicted duration; selecting image data matched with the second prediction viewpoint from the second image set for display; wherein the first predicted duration is greater than the second predicted duration, the first predicted view comprising the second predicted view.

In this embodiment, after the display terminal 2 acquires the eyeball position coordinates, a second predicted time length is acquired, and then a second predicted viewpoint is acquired based on the eyeball position coordinates and the second predicted time length, where the second predicted viewpoint is a viewpoint corresponding to the predicted position of the user's eyeball when displaying a frame of image data at a future display time. Then, the display terminal 2 selects image data matching each viewpoint included in the second predicted viewpoint from the second image set, and performs operations such as pixel lighting, image rendering, and layout display. Wherein the number of views of the second predicted view is greater than or equal to 2, and the number of views of the second predicted view is less than the number of views of the first predicted view, the finally displayed image data may be a subset of the second image set. Thus, based on the viewpoint prediction realized by the eyeball tracking technology, the display terminal 2 can only process the image data matched with the second predicted viewpoint without processing all the image data when displaying the image data, so that the processing amount of the image data can be reduced, and the time delay of image playing is further reduced.

In some embodiments, when there is only one user in front of the display screen, the number of views of the second predicted view may be only 2; when there are only two users in front of the display screen, the number of views of the second prediction view may be only 4, thereby further reducing the amount of transmission and processing of image data.

Optionally, the image display system may be applied to a one-to-one video application scenario, such as a video call, a video conference, and the like; the method can also be applied to one-to-many video application scenes, such as network live broadcasting. With the image display system of the embodiment, the display terminal only needs to process and display the image data matched with the second predicted viewpoint.

In some embodiments, the display terminal 2 is further configured to: acquiring a second predicted duration; and determining a second eyeball position prediction coordinate according to the eyeball position coordinate at the current acquisition time, the second prediction time length, the eyeball tracking acquisition interval time length and the eyeball motion vector, and determining a second prediction viewpoint according to the second eyeball position prediction coordinate.

In the present embodiment, the display terminal 2 acquires the eyeball position coordinate (x) at the current acquisition time_now，y_now，z_now) And eye movement vector

Obtaining a second predicted time duration Deltat_display2And an eyeball tracking acquisition interval duration Deltat, and calculating a second eyeball position prediction coordinate (x) corresponding to the second prediction viewpoint based on the information_display2，y_display2，z_display2) Wherein the second eyeball position prediction coordinate (x)_display2，y_display2，z_display2) Satisfies the following conditions:

in this way, the second predicted viewpoint can be determined based on the determined second eye position predicted coordinates.

In some embodiments, the second predicted duration Δ t_display2The acquisition method comprises the following steps: as shown in fig. 7, it is assumed that at time T0, the display terminal 2 acquires the eyeball position coordinates at the current acquisition time, and predicts that a certain frame of image data a will be displayed on the display terminal 2 in a line at time T1, where time T0 is before time T1. The frame of image data a arrives at the display terminal 2 at time T2, and time T2 is before time T1. Thus, the first prediction period Δ t_display1＝T1-T2。

After the second image set including the frame image data a reaches the display terminal 2, image data matched with the second prediction viewpoint needs to be selected from the second image set for layout display, so that the time required by the above process can be predicted through the display terminal 2 to obtain the second predicted time duration Δ t_display2。

Alternatively, the second prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display2. Since the time required to realize the above-described processing is known by the system since the last frame of image data B has been displayed completely, i.e., T1 and T2 have been obtained, the first predicted duration Δ T can be obtained based on T1 and T2 of the last frame of image data B_display2. Wherein the previous frame of image data B can be associated with the frame of image dataA is an adjacent image frame, or may be a non-adjacent image frame, which is not limited in this embodiment.

In some embodiments, the image display system further comprises an image processing platform 3. As shown in fig. 5. The image processing platform is configured to: acquiring a third image set, the eyeball position coordinates and a third prediction duration; determining a third predicted viewpoint based on the eyeball position coordinates and the third predicted duration; selecting image data matched with the third prediction viewpoint from the third image set to be coded so as to generate the first image set, and sending the first image set to the server; wherein the third predicted duration is greater than the first predicted duration, and the third predicted view comprises the first predicted view.

In the present embodiment, the image processing platform 3 is used to process and encode the captured image. The image processing includes format conversion, distortion correction and the like on the acquired image, and the image coding includes image compression coding and the like on the image after format conversion and distortion correction. In order to realize naked eye 3D display on the display terminal 2, the encoding plug flow unit of the image processing platform 3 needs to include a third image set for realizing naked eye 3D display. Wherein the third image set includes multi-view image data matched with a plurality of views.

And after receiving the eyeball position coordinate sent by the display terminal 2, the coding stream pushing unit of the image processing platform 3 acquires a third predicted time length, and acquires a third predicted viewpoint based on the eyeball position coordinate and the third predicted time length. Wherein the number of views of the third predicted view is greater than the number of views of the first predicted view.

Then, the image processing platform 3 selects image data matching each viewpoint included in the third prediction viewpoint from the third image set, performs encoding processing to generate a first image set, and transmits the first image set to the server 1.

Wherein the first set of images is a subset of a third set of images comprising image data of other viewpoints in addition to image data matching the third predicted viewpoint. In this way, the viewpoint prediction based on the eye tracking technology can reduce not only the image data that needs to be encoded but also the image data that is sent from the image processing platform 3 to the server 1, and therefore the amount of transmission and processing of the image data can be reduced, thereby reducing the delay in data transmission.

The image processing platform 3 is further configured to: receiving eyeball position coordinates and eyeball motion vectors at the current acquisition time, which are sent by the display terminal; and determining a third eyeball position prediction coordinate according to the eyeball position coordinate at the current acquisition time, the third prediction time length, the eyeball tracking acquisition interval time length and the eyeball motion vector, and determining a third prediction viewpoint according to the third eyeball position prediction coordinate.

In the present embodiment, the image processing platform 3 receives the eyeball position coordinate (x) at the current acquisition time_now，y_now，z_now) And eye movement vector

Obtaining a third predicted time duration Deltat_display3And an eyeball tracking acquisition interval duration Deltat, and calculating a third eyeball position prediction coordinate (x) corresponding to the third prediction viewpoint based on the information_display3，y_display3，z_display3) Wherein the third eyeball position prediction coordinate (x)_display3，y_display3，z_display3) Satisfies the following conditions:

in this way, the third predicted viewpoint may be determined based on the determined third eye position predicted coordinates.

In some embodiments, the third predicted duration Δ t_display3The acquisition method comprises the following steps: as shown in fig. 7, it is assumed that at time T0, the display terminal 2 acquires the currentThe eyeball position coordinates at the time of acquisition are acquired, and it is predicted that a certain frame of image data a will be displayed on the display terminal 2 in a row at time T1, where time T0 is before time T1. The frame of image data a arrives at the encoding stream unit of the image processing stage 3 at time T4, and time T4 is before time T1. Thus, the third prediction period Δ t_display3＝T1-T4。

After the third image set including the frame of image data a reaches the encoding stream pushing unit of the image processing platform 3, image data matched with the third prediction viewpoint needs to be selected from the third image set to generate image data including the frame of image data a, the image data is encoded by the encoding stream pushing unit to generate the first image set, and the first image set is sent to the server 1 and is streamed to the display terminal for arranging and displaying, so that the time required by the process can be predicted by the image processing platform 3 to obtain the third prediction duration Δ t_display3。

Alternatively, the third prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display3. Since the time required to realize the above-described processing is known by the system since the last frame of image data B has been displayed completely, i.e., T1 and T4 have been obtained, the third prediction time period Δ T can be obtained based on T1 and T4 of the last frame of image data B_display3. The previous frame of image data B may be an adjacent image frame to the frame of image data a, or may not be an adjacent image frame, which is not limited in this embodiment.

In some embodiments, the image processing platform 3 is further configured to: acquiring a fourth image set and a fourth predicted duration; determining a fourth predicted viewpoint based on the eyeball position coordinates and a fourth predicted duration; selecting image data from the fourth set of images that match the fourth predicted viewpoint for image processing to generate the third set of images; wherein the fourth predicted duration is greater than the third predicted duration, the fourth predicted view comprising the third predicted view.

In this embodiment, in order to implement naked eye 3D display on the display terminal 2, the image processing unit of the image processing platform 3 needs to include a fourth image set for implementing naked eye 3D display. Wherein the fourth image set includes multi-view image data matched with a plurality of views.

And after receiving the eyeball position coordinate sent by the display terminal 2, the image processing unit of the image processing platform 3 acquires a fourth predicted time length, and acquires a fourth predicted viewpoint based on the eyeball position coordinate and the fourth predicted time length. Wherein the number of views of the fourth predicted view is greater than the number of views of the third predicted view.

Then, the image processing platform 3 selects image data matched with each viewpoint included in the fourth prediction viewpoint from the fourth image set, performs image processing steps such as format conversion, distortion correction and the like to generate a third image set, and sends the third image set to the image processing platform 3 for encoding processing by an encoding stream pushing unit.

Wherein the third set of images is a subset of a fourth set of images comprising image data of other viewpoints in addition to image data matching the fourth predicted viewpoint. In this way, based on the viewpoint prediction by the eye tracking technique, image data requiring image processing such as format conversion and distortion correction can be reduced, and therefore the amount of transmission and processing of image data can be reduced, and the delay in data transmission can be reduced.

In the present embodiment, the image processing unit of the image processing platform 3 receives the eyeball position coordinate (x) at the current acquisition time_now，y_now，z_now) And eye movement vector

Obtaining a fourth predicted time duration Deltat_display4And an eyeball tracking acquisition interval duration Deltat, and based on the information, calculating a fourth eyeball position prediction coordinate (x) corresponding to the fourth prediction viewpoint_display4，y_display4，z_display4) Wherein the fourth eye position predictionCoordinate (x)_display4，y_display4，z_display4) Satisfies the following conditions:

in this way, the fourth predicted viewpoint may be determined based on the determined fourth eye position predicted coordinates.

In some embodiments, the fourth predicted duration Δ t_display4The acquisition method comprises the following steps: as shown in fig. 7, it is assumed that at time T0, the display terminal 2 acquires the eyeball position coordinates at the current acquisition time, and predicts that a certain frame of image data a will be displayed on the display terminal 2 in a line at time T1, where time T0 is before time T1. The frame of image data A arrives at the image processing unit of the image processing stage 3 at time T5, and time T5 is before time T1. Thus, the fourth predicted time period Δ t_display4＝T1-T5。

The fourth image set including the frame image data a needs to be sequentially subjected to image processing, encoding and streaming to the server, streaming to the display terminal through the server 1, and image arrangement and display through the display terminal, so that the time required by the above process can be predicted through the image processing platform 3 to obtain a fourth predicted time duration Δ t_display4。

Alternatively, the fourth prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display4. Since the time required to realize the above-described processing is known by the system since the last frame of image data B has been displayed completely, i.e., T1 and T4 have been obtained, the third prediction time period Δ T can be obtained based on T1 and T5 of the last frame of image data B_display4. The previous frame of image data B may be an adjacent image frame to the frame of image data a, or may not be an adjacent image frame, which is not limited in this embodiment.

In some embodiments, the image display system further comprises an image acquisition device 4. As shown in fig. 6, the image pickup apparatus includes an image pickup unit for picking up multi-viewpoint image data; is configured to: acquiring the eyeball position coordinate and a fifth prediction duration, and determining a fifth prediction viewpoint based on the eyeball position coordinate and the fifth prediction duration; controlling an image acquisition unit matched with the fifth prediction viewpoint to acquire images so as to generate a fourth image set, and sending the fourth image set to the image processing platform; wherein the fifth predicted duration is greater than the fourth predicted duration, the fifth predicted view comprising the fourth predicted view.

In this embodiment, the image capturing device 4 may include a plurality of image capturing units, and the image capturing units may include cameras, video cameras, and other devices for capturing image data and video data. The image acquisition device 4 can acquire image data and video data of an object to be photographed based on various angles to generate image data for realizing multi-implementation display. In order to realize naked eye 3D display on the display terminal 2, image data or video stream data capable of realizing matching with 40-60 viewpoints is included in the image acquisition device 4. In this embodiment, an example in which the image capturing device 4 can capture image data or video stream data of 60 viewpoints is described.

And the image acquisition device 4 acquires a fifth predicted time length after receiving the eyeball position coordinate sent by the display terminal 2, and acquires a fifth predicted viewpoint based on the eyeball position coordinate and the fifth predicted time length. Wherein the number of views of the fifth predicted view is greater than the number of views of the fourth predicted view.

Then, the image capturing device 4 controls the image capturing unit matched with each viewpoint included in the fifth predicted viewpoint to perform multi-viewpoint image capturing to generate a fourth image set, and then sends the fourth image set to the image processing platform 3.

Thus, the viewpoint prediction based on the eyeball tracking technology can not only reduce the image data to be collected, but also reduce the image data sent from the image collecting device 4 to the image processing platform 3, thereby reducing the transmission amount and the processing amount of the image data and further reducing the delay of data transmission.

The image acquisition apparatus 4 is further configured to: receiving eyeball position coordinates and eyeball motion vectors at the current acquisition time, which are sent by the display terminal; and determining a fifth eyeball position prediction coordinate according to the eyeball position coordinate at the current acquisition time, the fifth prediction time length, the eyeball tracking acquisition interval time length and the eyeball motion vector, and determining a fifth prediction viewpoint according to the fifth eyeball position prediction coordinate.

In the present embodiment, the image pickup device 4 receives the eyeball position coordinates (x) at the current pickup time_now，y_now，z_now) And eye movement vector

Obtaining a fifth predicted time period Deltat_display5And an eyeball-tracking acquisition interval duration Deltat, and calculating a fifth eyeball position prediction coordinate (x) corresponding to the fifth prediction viewpoint based on the information_display5，y_display5，z_display5) Wherein the fifth eyeball position prediction coordinate (x)_display5，y_display5，z_display5) Satisfies the following conditions:

in this way, the fifth predicted viewpoint can be determined based on the determined fifth eyeball position predicted coordinates.

In some embodiments, the fifth predicted duration Δ t_display5The acquisition method comprises the following steps: as shown in fig. 7, it is assumed that at time T0, the display terminal 2 acquires the eyeball position coordinates at the current acquisition time, and predicts that a certain frame of image data a will be displayed on the display terminal 2 in a line at time T1, where time T0 is before time T1. The frame of image data a arrives at the image capturing device 4 at time T6, and time T6 is before time T1. Thus, the fifth prediction period Δ t_display5＝T1-T6。

Wherein the image acquisition device 4 selects a view matching with the fifth prediction viewpointThe image acquisition unit acquires image data to generate a first image set comprising the frame of image data A, and then sequentially performs image processing, encoding and streaming to the server, streaming to the display terminal through the server 1, and layout display on the display terminal, so that the time required by the process can be predicted through the image acquisition device 4 to obtain a fifth predicted time length delta t_display5。

Alternatively, the fifth prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display5. Since the time required to realize the above-described processing is known by the system since the last frame of image data B has been displayed completely, i.e., T1 and T6 have been obtained, the third prediction time period Δ T can be obtained based on T1 and T6 of the last frame of image data B_display5. The previous frame of image data B may be an adjacent image frame to the frame of image data a, or may not be an adjacent image frame, which is not limited in this embodiment.

In further embodiments, the server 1 is further configured to: determining monocular eyeball position prediction coordinates according to the eyeball position coordinates, and acquiring monocular prediction viewpoints according to the monocular eyeball position prediction coordinates; and acquiring the pupil distance of the two eyes, determining a predicted viewpoint of the other eye according to the pupil distance of the two eyes and the single-eye predicted viewpoint, and determining the first predicted viewpoint according to the single-eye predicted viewpoint and the predicted viewpoint of the other eye.

In this embodiment, the viewpoints of the left and right eyes are matched in pairs according to the imaging principle of the lenticular stereoscopic display. As shown in fig. 9, for example, when the left eye observes viewpoint 1, the right eye certainly observes viewpoint 3 because the interpupillary distance of the observer is within a wide range. Therefore, at the time of prediction of the first prediction viewpoint, prediction of the eyeball position prediction coordinates can be performed based on only the eyeball position coordinates of a single eye, thereby obtaining a single eye viewpoint. Then, the viewpoint of the other eye is determined based on the pupillary distance of the two eyes of the user, so that the synchronization of the left eye and the right eye is ensured. When live broadcasting plug flow is carried out, image data corresponding to the left eye and the right eye are spliced into one image to be processed, so that absolute synchronization of the left eye and the right eye is guaranteed.

Optionally, the calculation manners of the second predicted viewpoint, the third predicted viewpoint, the fourth predicted viewpoint and the fifth predicted viewpoint are the same as those in the above embodiment, that is, monocular eyeball position predicted coordinates are determined according to the eyeball position coordinates, and a monocular predicted viewpoint is obtained according to the monocular eyeball position predicted coordinates; and obtaining the pupil distance of the two eyes, determining a predicted viewpoint of the other eye according to the pupil distance of the two eyes and the single-eye predicted viewpoint, and determining the second predicted viewpoint, the third predicted viewpoint, the fourth predicted viewpoint or the fifth predicted viewpoint according to the single-eye predicted viewpoint and the predicted viewpoint of the other eye.

In some embodiments, the server 1 is further configured to: determining an eyeball motion track according to the eyeball position coordinates and the eyeball position prediction coordinates at the current acquisition moment, and acquiring a viewpoint which is coincident with the eyeball motion track as the first prediction viewpoint.

As shown in fig. 8, based on the eyeball position coordinate at the current acquisition time, connecting the eyeball and the origin of a coordinate system XYZ into a first straight line, wherein the projection included angle of the first straight line and the Z axis in the horizontal direction is ≦ α; and based on the eyeball position prediction coordinate, connecting the eyeball and the origin of a coordinate system XYZ into a second straight line, wherein the projection included angle of the included angle between the second straight line and the Z axis in the horizontal direction is between the included angle alpha and the angle beta, and the eyeball motion track is between the included angle alpha and the angle beta. Correspondingly, the first prediction viewpoint is all viewpoints with the included angle between alpha and beta.

Optionally, the second predicted view, the third predicted view, the fourth predicted view or the fifth predicted view is also predicted by using the method shown in this embodiment. Because each predicted viewpoint is related to the predicted time length, the longer the predicted time length is, the larger the span of the included angle alpha to beta is, and the more viewpoints are contained; the shorter the prediction time is, the smaller the span from the included angle alpha to the angle beta is, and the fewer the contained viewpoints are; thus, the closer to the arrangement display of the display terminal, the smaller the number of predicted viewpoints, and the less image data that needs to be transmitted and processed.

Alternatively, in the image display system shown in fig. 6, the data processing amount and the transmission amount may be reduced by performing gaze point prediction based on only any one of the first predicted viewpoint, the second predicted viewpoint, the third predicted viewpoint, the fourth predicted viewpoint and the fifth predicted viewpoint, or may be reduced by performing gaze point prediction based on a plurality of predicted points or all predicted points of the five predicted points, which is not particularly limited in this embodiment.

In some embodiments, the display terminal 2 is further configured to: selecting image data matched with the first prediction viewpoint from the second image set to establish a frame buffer; and detecting the time stamp of each frame of image data in the frame buffer to determine the image data to be displayed.

In this embodiment, as shown in fig. 9, when the display terminal 2 detects that the eyeball of the user is about to move to the eyeball position prediction coordinate, for example, when it is detected that the eyeball position starts to move from the eyeball position coordinate at the previous acquisition time and the eyeball motion vector is greater than the preset system error value, a frame buffer may be established on the display terminal 2 based on any one or more prediction results in the above embodiments, so as to facilitate subsequent image display.

When detecting that the eyeball of the user is about to move to the eyeball position prediction coordinate, verifying the time stamp of each frame of image data in the frame cache, and accordingly selecting to start playing from a correct frame, and avoiding playing errors caused by switching video stream data. As shown in fig. 9, for example, in the current position, the left-eye viewpoint displays the tenth frame image data of the viewpoint (view)2, and the prediction viewpoint is the viewpoint (view)3 and the eleventh frame image data should be displayed; when the user realizes that the user is about to move to the position corresponding to the view (view)3, the time stamp of each frame image corresponding to the view (view)3 needs to be verified, and the eleventh frame image data is selected from the time stamps for playing.

Based on the same inventive concept, the present disclosure also provides an image display method corresponding to any of the above embodiments. As shown in fig. 10, the image display method includes:

step S101, a display terminal collects eyeball position coordinates and sends the eyeball position coordinates to the server.

In this embodiment, the display terminal obtains the eyeball position coordinates through an eyeball tracking technology and then sends the eyeball position coordinates to the server.

In step S102, a server acquires a first image set including multi-viewpoint image data matched with a plurality of viewpoints.

In this embodiment, the server 1 needs to include a first image set for realizing naked-eye 3D display.

Step S103, the server acquires eyeball position coordinates, and acquires a first prediction viewpoint matched with the prediction display time based on the eyeball position coordinates and the first prediction duration. The server 1 receives the eyeball position coordinate sent by the display terminal 2, acquires a first prediction time length, acquires a first prediction viewpoint based on the eyeball position coordinate and the first prediction time length,

and step S104, selecting image data matched with the first prediction viewpoint from the first image set to generate a second image set, and sending the second image set to a display terminal. The server 1 selects image data matching each viewpoint included in the first prediction viewpoint from the first image set, generates a second image set, and transmits the second image set to the display terminal 2, so that the display terminal 2 can perform layout display based on the image data in the second image set.

Step S105, the display terminal receives the second image set and displays the second image set.

According to the image display method, the eyeball position coordinates are collected through the display terminal and sent to the server; the server predicts a first prediction viewpoint corresponding to the display time of the image to be displayed based on the eyeball position coordinates; and then the server selects image data matched with the first prediction viewpoint from the first image set based on the first prediction viewpoint to generate a second image set and sends the second image set to the display terminal, and the display terminal only needs to display based on the images in the second image set, and does not need to pull all the image data in the first image set to the display terminal to perform operations such as pixel lighting, image rendering, map arrangement display and the like, so that the transmission quantity and the processing quantity of the image data can be reduced.

As shown in fig. 8, an XYZ coordinate system is established with the central point of the display screen of the display terminal as the origin, and the eyeball position coordinate at the previous acquisition time is (x)_last，y_last，z_last) The eyeball position coordinate at the current acquisition time is (x)_now，y_now，z_now) Then eyeball motion vector

Satisfies the following conditions:

To the server 2.

In some embodiments, the first predicted duration Δ t_display1The acquisition method comprises the following steps: as shown in fig. 7, it is assumed that at time T0, the display terminal 2 acquires the eyeball position coordinates at the current acquisition time, and predicts that a certain frame of image data a will be displayed on the display terminal 2 in a line at time T1, where time T0 is before time T1. The frame of image data a arrives at the server 1 at time T3, and time T3 is before time T1. Thus, the first prediction period Δ t_display1＝T1-T3。

Alternatively, the first prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display1. The system is known to implement since the last frame of image data B has been displayedThe time required for the above-described processing, i.e., T1 and T3, has been obtained, and thus the first prediction time period Δ T can be obtained based on T1 and T3 of the previous frame image data B_display1. The previous frame of image data B may be an adjacent image frame to the frame of image data a, or may not be an adjacent image frame, which is not limited in this embodiment.

In some embodiments, as shown in fig. 11, the image display method further includes:

step S201, the playing device obtains the eyeball position coordinate and a second predicted duration.

Step S202, the playing device determines a second prediction viewpoint based on the eyeball position coordinate and a second prediction duration.

Step S203, the playing device selects image data matched with the second prediction viewpoint from the second image set to display;

Alternatively, the second prediction time period Δ t may be calculated based on the last frame image data B for which the above-described processing steps have been completed_display2. Since the time required to realize the above-described processing is known by the system since the last frame of image data B has been displayed completely, i.e., T1 and T2 have been obtained, the first predicted duration Δ T can be obtained based on T1 and T2 of the last frame of image data B_display2. The previous frame of image data B may be an adjacent image frame to the frame of image data a, or may not be an adjacent image frame, which is not limited in this embodiment.

In some embodiments, the image display method, the image display system further comprises an image processing platform; the method further comprises the following steps:

step S301, the image processing platform acquires a third image set, the eyeball position coordinates and a third predicted time length.

Step S302, the image processing platform determines a third prediction viewpoint based on the eyeball position coordinates and the third prediction duration.

Step S303, the image processing platform selects image data matched with the third predicted duration from the third image set to encode so as to generate the first image set, and sends the first image set to the server;

The coding plug flow unit of the image processing platform 3 needs to include a third image set for realizing naked eye 3D display. And after receiving the eyeball position coordinate sent by the display terminal 2, the coding stream pushing unit of the image processing platform 3 acquires a third predicted time length, and acquires a third predicted viewpoint based on the eyeball position coordinate and the third predicted time length. Wherein the number of views of the third predicted view is greater than the number of views of the first predicted view.

In some embodiments, the image display method further includes:

step S401, the image processing platform acquires a fourth image set and a fourth predicted time length.

Step S402, the image processing platform determines a fourth prediction viewpoint based on the eyeball position coordinates and a fourth prediction duration.

Step S403, the image processing platform selects image data matching the fourth predicted viewpoint from the fourth image set to perform image processing to generate the third image set;

In some embodiments, the image display method further includes:

step S501, the image acquisition device acquires the eyeball position coordinate and a fifth prediction time length, and determines a fifth prediction viewpoint based on the eyeball position coordinate and the fifth prediction time length.

In step S502, the image capturing apparatus controls the image capturing unit matched with the fifth predicted viewpoint to perform image capturing to generate a fifth image set.

Optionally, the image display method further includes:

the display terminal acquires the eyeball position coordinates of the user in real time, and determines eyeball motion vectors based on the eyeball position coordinates of the current acquisition time and the eyeball position coordinates of the previous acquisition time in the eyeball position coordinates.

The server receives the eyeball position coordinate and the eyeball motion vector of the current acquisition time sent by the display terminal, determines a first eyeball position prediction coordinate according to the eyeball position coordinate, the first prediction time length, the eyeball tracking acquisition interval time length and the eyeball motion vector of the current acquisition time, and determines the first prediction viewpoint according to the first eyeball position prediction coordinate.

Optionally, the server determines a monocular eyeball position prediction coordinate according to the eyeball position coordinate, and obtains a monocular prediction viewpoint according to the monocular eyeball position prediction coordinate; and acquiring the pupil distance of the two eyes, determining a predicted viewpoint of the other eye according to the pupil distance of the two eyes and the single-eye predicted viewpoint, and determining the first predicted viewpoint according to the single-eye predicted viewpoint and the predicted viewpoint of the other eye.

Optionally, the server determines an eye movement trajectory according to the eye position coordinates and the eye position prediction coordinates at the current acquisition time, and acquires a viewpoint coinciding with the eye movement trajectory as the first prediction viewpoint.

Optionally, the display terminal selects image data matched with the first prediction viewpoint from the second image set to establish a frame buffer; and the display terminal detects the time stamp of each frame of image data in the frame buffer to determine the image data to be displayed.

It should be noted that the method of the embodiments of the present disclosure may be executed by a single device, such as a computer or a server. The method of the embodiment can also be applied to a distributed scene and completed by the mutual cooperation of a plurality of devices. In such a distributed scenario, one of the devices may only perform one or more steps of the method of the embodiments of the present disclosure, and the devices may interact with each other to complete the method.

It should be noted that the above describes some embodiments of the disclosure. Other embodiments are within the scope of the following claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments described above and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing may also be possible or may be advantageous.

Based on the same inventive concept, the present disclosure also provides a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method according to any of the above embodiments, corresponding to any of the above-described embodiment methods.

Computer-readable media of the present embodiments, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

The computer instructions stored in the storage medium of the above embodiment are used to enable the computer to execute the method according to any of the above embodiments, and have the beneficial effects of the corresponding method embodiment, and are not described herein again.

Those of ordinary skill in the art will understand that: the discussion of any embodiment above is meant to be exemplary only, and is not intended to intimate that the scope of the disclosure, including the claims, is limited to these examples; within the idea of the present disclosure, also technical features in the above embodiments or in different embodiments may be combined, steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present disclosure as described above, which are not provided in detail for the sake of brevity.

In addition, well-known power/ground connections to Integrated Circuit (IC) chips and other components may or may not be shown in the provided figures for simplicity of illustration and discussion, and so as not to obscure the embodiments of the disclosure. Furthermore, devices may be shown in block diagram form in order to avoid obscuring embodiments of the present disclosure, and this also takes into account the fact that specifics with respect to implementation of such block diagram devices are highly dependent upon the platform within which the embodiments of the present disclosure are to be implemented (i.e., specifics should be well within purview of one skilled in the art). Where specific details (e.g., circuits) are set forth in order to describe example embodiments of the disclosure, it should be apparent to one skilled in the art that the embodiments of the disclosure can be practiced without, or with variation of, these specific details. Accordingly, the description is to be regarded as illustrative instead of restrictive.

While the present disclosure has been described in conjunction with specific embodiments thereof, many alternatives, modifications, and variations of these embodiments will be apparent to those of ordinary skill in the art in light of the foregoing description. For example, other memory architectures (e.g., dynamic ram (dram)) may use the discussed embodiments.

The disclosed embodiments are intended to embrace all such alternatives, modifications and variances which fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalents, improvements, and the like that may be made within the spirit and principles of the embodiments of the disclosure are intended to be included within the scope of the disclosure.

Claims

1. An image display system, applied to naked-eye 3D display, comprising:

2. The system of claim 1,

the display terminal is further configured to: acquiring the eyeball position coordinate and a second predicted time length; determining a second predicted viewpoint based on the eyeball position coordinates and a second predicted duration; selecting image data matched with the second prediction viewpoint from the second image set for display;

3. The system of claim 2, further comprising:

4. The system of claim 3,

an image processing platform further configured to: acquiring a fourth image set and a fourth predicted duration; determining a fourth predicted viewpoint based on the eyeball position coordinates and a fourth predicted duration; selecting image data from the fourth set of images that match the fourth predicted viewpoint for image processing to generate the third set of images;

5. The system of claim 4, further comprising:

6. The system of claim 5,

the display terminal is further configured to: acquiring the eyeball position coordinates of a user in real time; determining an eyeball motion vector based on the eyeball position coordinate of the current collection time and the eyeball position coordinate of the previous collection time in the eyeball position coordinates;

7. The system of claim 6, further comprising:

8. The system of claim 6,

the server further configured to: determining an eyeball motion track according to the eyeball position coordinates and the eyeball position prediction coordinates at the current acquisition moment, and acquiring a viewpoint which is coincident with the eyeball motion track as the first prediction viewpoint.

9. The system of claim 6,

the display terminal is further configured to: selecting image data matched with the first prediction viewpoint from the second image set to establish a frame buffer; and detecting the time stamp of each frame of image data in the frame buffer to determine the image data to be displayed.

10. An image display method, comprising:

the server selects image data matched with the first prediction viewpoint from the first image set to generate a second image set, and sends the second image set to a display terminal;

11. The method of claim 10, further comprising:

12. The method of claim 11, further comprising:

13. The method of claim 12, further comprising:

14. The method of claim 13, further comprising:

15. A non-transitory computer-readable storage medium containing a computer program, which, when executed by one or more processors, causes the processors to perform the method of any one of claims 10-14.

16. A computer program product comprising computer program instructions to cause a computer to perform the method of any one of claims 10 to 14 when the computer program instructions are run on the computer.