TWI773439B - Computational display and operating method thereof - Google Patents

Abstract

A computational display and an operating method thereof are disclosed. The computational display includes a temporal filtering module, a decomposition calculating module and a display module. The temporal filtering module is used to perform temporal filtering module on light field frames of an initial dynamic light field video to obtain a filtered light field frame. The decomposition calculating module is coupled to the temporal filtering module and used to apply a decomposition algorithm to decompose the filtered light field frame into decomposed frames, wherein the frame rate of the decomposed frames is larger than the frame rate of the light field frames. The display module includes multi-layer display units and used to display the decomposed frames at multiple observation times to be observed as a result dynamic light field video.

Description

Computational display and method of operation

The present invention relates to displays, and more particularly, to a computing display and a method of operation thereof.

In recent years, with the evolution of display technology, computational displays such as Multi-layer LCD Factorized Displays can expand applications such as wider dynamic range and wider color gamut, especially factorized light fields Displays have attracted widespread attention because they are significantly superior to traditional light field displays in terms of space, viewing angle resolution, display brightness, and composition volume.

Different from the traditional light field display, before the multi-layer liquid crystal decomposed light field display displays the light field image, the light field image needs to be decomposed into the content played on each layer of liquid crystal of the multi-layer liquid crystal decomposed light field display. In the application of light field display, in order to improve the stereoscopic viewing effect, the display often uses a much higher update rate than the human eye can perceive to play the picture. Flicker fusion mechanism to fuse multiple frames of images into a single Fused frame.

However, in a dynamic light field movie, due to the difference between the two groups of decomposed frames decomposed by different two light field images, the human eye will see an indefinable fusion light at an unspecified time point. Field frame (Undefined fused frame). For example, as shown in Figure 1, although the human eye can correctly see the fused frame formed by the fusion of the same set of decomposed frames {a0, a1, a2, a3} at time point t0, it will be seen at time point t2. The fusion frames that are formed by interweaving different components of the decomposition frames {a2, a3, b0, b1}, because these fusion frames are not defined in the decomposition algorithm, they belong to the fused light field frames that cannot be defined.

In practical applications, compared with the original light field film, these undefined fused light field frames have many image artifacts (Artifact). Furthermore, due to the large differences between these undefined fused lightfield frames, severe flickering is felt when viewed continuously by the human eye. As the display update speed is different, the flickering phenomenon may cause discomfort to the human eye, which needs to be improved.

In view of this, the present invention provides a computing display and an operation method thereof to effectively solve the above-mentioned problems encountered in the prior art.

One embodiment according to the present invention is a computing display. In this embodiment, the computational display includes a time domain filter module, a decomposition calculation module and a display module. The time domain filtering module is used for performing time domain filtering on a plurality of light field frames in the original dynamic light field film to obtain filtered light field frames. The decomposition algorithm module is coupled to the time-domain filtering unit, and is used for decomposing the filtered light field frame into a plurality of decomposition frames through a decomposition algorithm, wherein the update rate of the plurality of decomposition frames is greater than the update rate of the plurality of light field frames . The display module includes a multi-layer display unit for displaying the plurality of decomposed frames at a plurality of observation time points respectively, so as to be observed as a result dynamic light field film.

In one embodiment, the computing display is a multi-layer LCD factorized display and at least one display unit of the multi-layer display units is a liquid crystal layer.

In one embodiment, the decomposition algorithm adopted by the decomposition calculation module is a light field video decomposition algorithm, and the computational display further includes: a weight estimation module, coupled to the time domain filter module, for according to the initial dynamic light field video. The weights are estimated for the plurality of light field frames, so that the temporal filtering module can perform temporal filtering according to the weights.

In one embodiment, the decomposition algorithm module performs a decomposition algorithm on the filtered light field frame under the restriction of fixing other decomposition frames of the plurality of decomposition frames to optimize the current decomposition frame of the plurality of decomposition frames .

In one embodiment, the weight is calculated according to the time proportion of the current decomposed frame to the affected light field frame of the plurality of light field frames.

In one embodiment, the decomposition algorithm adopted by the decomposition calculation module is a light field film decomposition algorithm, and the decomposition calculation module includes: a segmentation processing unit for dividing the dynamic light field video into multiple segments with fixed lengths. After that, the decomposition algorithm is executed by the decomposition calculation module.

In one embodiment, the segment processing unit determines the fixed lengths of the plurality of segments according to the capacity of the memory, so as to avoid frequent reading and writing between the memory and the disk.

In one embodiment, any two adjacent segments in the plurality of segments partially overlap each other, so as to avoid the degradation of the quality of the decomposition algorithm caused by the segmentation process.

In one embodiment, the decomposition calculation module first initializes each decomposition frame of the plurality of decomposition frames with random random numbers, and then cyclically performs the optimization calculation of each decomposition frame according to the time sequence until the set number of iterations is reached. .

In one embodiment, the time domain filter module and the decomposition calculation module are disposed on the computing platform, and the computing platform reads the current light field frame from the plurality of light field frames stored in the disk of the host through a channel, The decomposed frames updated by the computing platform are written back to the host's disk through the channel.

Another embodiment according to the present invention is a method for operating a computing display. In this embodiment, the computing display operation method is used to operate a computing display. The operation method of the computational display includes the following steps: (a) performing temporal filtering on a plurality of light field frames in an initial dynamic light field film to obtain filtered light field frames; (b) using a decomposition algorithm to obtain the filtered light field frames; The field frame is decomposed into a plurality of decomposed frames, wherein the update rate of the plurality of decomposed frames is greater than the update rate of the plurality of light field frames; and (c) respectively displaying the plurality of decomposed frames at a plurality of observation time points to be observed Dynamic light field movie for the result.

In one embodiment, the computing display is a multi-layer liquid crystal decomposition display and at least one display unit of the multi-layer display units is a liquid crystal layer.

In one embodiment, the decomposition algorithm is a light field video decomposition algorithm, and the computing display operation method further includes: estimating weights according to the plurality of light field frames in the initial dynamic light field video, for step (a) according to the weights time domain filtering.

In one embodiment, the method for operating a computational display further comprises: performing a decomposition algorithm on the filtered light field frame to optimize the number of decomposition frames in the plurality of decomposition frames under the constraint of fixing other decomposition frames of the plurality of decomposition frames. the current decomposed frame.

In one embodiment, the weight is calculated according to the time proportion of the current decomposed frame to the affected light field frame of the plurality of light field frames.

In one embodiment, the decomposition algorithm is a light field video decomposition algorithm, and the operation method of the computational display includes: dividing the dynamic light field video into a plurality of segments with fixed lengths, and then executing the decomposition algorithm.

In one embodiment, the fixed length of the plurality of segments is determined according to the capacity of the memory, so as to avoid frequent reading and writing between the memory and the disk.

In one embodiment, any two adjacent segments in the plurality of segments partially overlap each other, so as to avoid the degradation of the quality of the decomposition algorithm caused by the segmentation process.

In one embodiment, the computing display operation method is to initialize each of the plurality of decomposed frames with random random numbers, and then cyclically perform the optimization calculation of each decomposed frame according to the time sequence, until the set until the number of iterations.

In one embodiment, steps (a) and (b) are performed on the computing platform, and the computing platform reads the current light field frame from the plurality of light field frames stored in the disk of the host through the channel, and then calculates the current light field frame. The decomposed frames after the platform update are written back to the host's disk through the channel.

Compared with the prior art, the computational display and the operation method thereof of the present invention use the weighted time domain filtering technology to preprocess the target light field video, so that the light field video decomposition algorithm can be calculated more efficiently, and the light field video is overlapped The segment is the optimization unit, and the decomposition algorithm is executed independently for each segment to improve the execution speed of the decomposition algorithm, so it can effectively solve the image defects and flickering that are prone to occur when the traditional decomposition light field display is playing videos. , to increase the comfort of the human eye.

The advantages and spirit of the present invention can be further understood from the following detailed description of the invention and the accompanying drawings.

Reference will now be made in detail to the exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings. Elements/components using the same or similar numbers in the drawings and the embodiments are intended to represent the same or similar parts.

One embodiment according to the present invention is a computing display. In this embodiment, the computing display may be any kind of factorized display, such as a multi-layer LCD factorized display, and the multi-layer display unit may include at least one liquid crystal layer and may include a backlight layer, But not limited to this.

It should be noted that, unlike the prior art, where each light field frame in the light field film is independently decomposed into a set of decomposed frames as a constraint condition, the present invention starts from the film observation time point, for every possible light field film in the light field film. Fused light field frames that will be observed at the observation time point are considered.

Taking the time-domain modulation of rank N (every N decomposed frames are observed as a fusion frame) as an example, each light field frame in the prior art light field film is decomposed into a set of N decomposed frames independently, Therefore, when the decomposed display plays a light field movie, only one frame every N frames allows the observer to observe the correct fused light field frame. As for the present invention, the concept of the same group of decomposed frames corresponding to a single light field frame in the prior art is broken. The present invention limits the fused light field frames that should be correctly observed by the human eye according to the playback time of each decomposed frame, so that all decomposed frames can be correctly observed. Any N adjacent decomposed frames in the frame can be observed by the human eye as a defined fused light field frame. As a result, the human eye no longer observes flicker caused by indefinable fused light field frames because its viewing time is out of sync with the split display.

Please refer to FIG. 2 , which is a functional block diagram of the computing display in this embodiment. As shown in FIG. 2 , the computing display 1 includes a loading module 10 , a weight estimation module 12 , a time-domain filtering module 14 , a decomposition calculation module 16 and a display module 18 . The loading module 10 is coupled to the weight estimation module 12 . The weight estimation module 12 is coupled to the time domain filtering module 14 . The time domain filter module 14 is coupled to the decomposition calculation module 16 . The decomposition calculation module 16 is coupled to the display module 18 .

The loading module 10 is used for loading, for example, the light field video LV stored in the external memory and the content of the plurality of light field frames LF in the initial multi-frame multi-layer decomposition video V0 to, for example, a processing platform. The weight estimation module 12 is used for estimating the weight W according to the plurality of light field frames LF, and then the time domain filtering module 14 performs time domain filtering on the plurality of light field frames LF according to the weight W to obtain the filtered light field. Frame FLF. The decomposition algorithm module 16 decomposes the filtered light field frame FLF into a plurality of decomposed frames FF through a decomposition algorithm, wherein the update rate of the plurality of decomposed frames FF is greater than the update rate of the plurality of light field frames LF. The display module 18 displays the plurality of decomposed frames FF at a plurality of observation time points, respectively. When the human eye views the display module 18 at the multiple observation time points, the human eye will see the fusion of the multiple decomposed frames FF at the multiple observation time points due to the visual flicker fusion mechanism. The resulting multi-frame multi-layer decomposition film V1 is formed by multiple fused light field frames.

Please refer to FIG. 3 . FIG. 3 is a schematic diagram illustrating that light field frames at different time points are respectively decomposed into decomposed frames, and any four adjacent decomposed frames can be observed as defined fused light field frames.

As shown in Figure 3, taking the time domain modulation of rank 4 (every four decomposed frames are observed as a fusion frame) as an example, it is assumed that the light field frames at different time points t0, t1, t2, ... are respectively The sequence is LF0, LF1, LF2, ..., wherein the light field frame LF0 is decomposed into four decomposed frames FF0~FF3, the light field frame LF1 is decomposed into four decomposed frames FF1~FF4, and the light field frame LF2 is decomposed into four decomposed frames. Decomposition frames FF2~FF5, ..., the rest can be deduced by analogy.

Therefore, when the display module 18 of the computing display 1 plays a plurality of decomposed frames FF0, FF1, FF2, FF3, FF4, FF5, . On the Flicker fusion mechanism, the observer's eyes will see a single fusion light field formed by the fusion of any four adjacent decomposed frames (such as FF0~FF3, FF1~FF4, FF2~FF5, ...). frame. Since the adjacent decomposed frames FF0 to FF3 may correspond to the light field frame LF0, the adjacent decomposed frames FF1 to FF4 may correspond to the light field frame LF1, and the adjacent decomposed frames FF2 to FF5 may correspond to the light field frames LF2, . . . , therefore, any four adjacent decomposed frames (such as FF0~FF3, FF1~FF4, FF2~FF5, ...) can be observed as a defined fused light field frame, without the An undefined fused lightfield frame event occurs.

Please refer to FIG. 4 . FIG. 4 is a schematic diagram of calculating the contribution according to the time ratio of the current decomposed frame to the affected light field frame and assigning weights respectively.

As shown in FIG. 4 , assuming that the current decomposed frame in the decomposed frames FF0 to FF7 is FF2, the weight estimation module 12 of the computational display 1 can calculate the time of the affected light field frames LF0, LF1 and LF2 according to the current decomposed frame FF2. The contribution is calculated in proportion and given weights W0, W1, and W2, respectively, for the time-domain filtering module 14 to perform subsequent time-domain filtering.

Please refer to FIG. 5 . FIG. 5 is a schematic diagram illustrating that the light field frame is decomposed into decomposed frames after time series filtering. As shown in FIG. 5 , the time-domain filtering module 14 of the computational display 1 can perform time-domain filtering on the light field frames LF0 , LF1 , and LF2 at different time points t0 , t1 , and t2 according to the weights, so as to obtain the filtered light field. Frame FLF. Next, the decomposition algorithm module 16 of the computing display 1 can decompose the filtered light field frame FLF into a plurality of decomposed frames FF0-FF7 through a decomposition algorithm, wherein the update rate of the plurality of decomposed frames FF0-FF7 is greater than that of the plurality of decomposed frames FF0-FF7 The update rate of light field frames LF0~LF2.

In addition, the decomposition algorithm module 16 can also perform a decomposition algorithm on the filtered light field frame FLF under the restriction of fixing other decomposition frames FF0 to FF1 and FF3 to FF7 in the plurality of decomposition frames FF0 to FF7, so as to optimize the The current decomposed frame FF2 among the plurality of decomposed frames FF0 to FF7.

Please refer to FIG. 6 . FIG. 6 is a schematic diagram of video clips in different memories of the host and the parallel processing platform.

As shown in FIG. 6 , the host HM and the parallel processing platform PP transmit data to each other through the channel PCIE. The host HM includes a disk DK and a main memory MM. The parallel processing platform PP includes a memory MR and a computing unit CE. The disk DK (such as a hard disk) of the host HM can store a complete light field movie and the complete light field movie can be divided into multiple video segments of a fixed length. The main memory MM can be loaded from the disk DK and decode the current The fixed length of the video clip can be determined according to the capacity of the main memory MM of the host HM, so as to avoid frequent reading and writing between the main memory MM and the disk DK. The host HM loads the current video segment to the parallel processing platform PP through the channel PCIE and stores it in the memory MR for the computing unit CE to perform subsequent computing processing.

Next, it will be described in detail through the following several embodiments.

(1) The first embodiment: it is used to illustrate how the present invention applies the decomposition algorithm under the film frame to obtain a stable light field film decomposition result. This embodiment implements a decomposition algorithm for a dynamic light field video with a target size of 9x9x384x512 and 60 frames per second. The result is a set of decomposed videos of rank 4 and two-layer LCD. The update rate of the equivalent decomposition video is per second 240 frames. The decomposition algorithm is set to perform 100 iterations.

The decomposition algorithm can be executed on the parallel computing platform PP of FIG. 6 in the form of a software program. In the running phase, the host HM reads and decodes the current video segment from its disk DK, and then transmits it to the memory MR of the parallel computing platform PP through the channel PCIE. After the parallel computing platform PP completes the operation, the updated current video clip is sent back to the main memory MM of the host through the channel PCIE and written back to the disk DK to complete an iterative update of the current video clip. As shown in FIG. 6 , since the data required for each frame operation includes images of multiple light field frames and multiple decomposed frames, the main memory MM of the host HM and the memory MR of the parallel computing platform PP can only store a part of the data. scene. The detailed steps are described as follows:

(a) Initialize each decomposed frame with random random numbers;

(b) cyclically perform the optimization calculation for each frame in time sequence (steps (c) to (g)) until the set number of iterations;

(c) Load the relevant decomposition frame and light field frame content of the currently processed frame from the disk of the host computer to the parallel computing platform;

(d) Calculate the contribution and give weights according to the time proportion of the current decomposition frame to the affected light field frame;

(e) temporally filtering the target light field frame according to the weight to obtain the filtered light field frame;

(f) subjecting the filtered lightfield frame to a decomposition algorithm to optimize the current decomposed frame, subject to the constraints of fixing other decomposed frames (without optimization updates); and

(g) Write the updated decomposed frame back to the hard disk of the host computer.

When the decomposed frames obtained according to the implementation of the present embodiment are played on the decomposed light field display, a smooth playback result can be obtained, and the image defects and flickering phenomenon across frames can be effectively avoided. On the test image, a light-field video reconstruction result with an average Peak Signal-to-Noise Ratio (PSNR) of 37 decibels (dB) can be obtained. At the same time, the difference in the peak signal-to-noise ratio of successive light field frames does not exceed 0.2 decibels (dB), thus presenting a smooth playback result.

(2) The second embodiment: based on the first embodiment, it is further proposed to reduce operation steps and complicated memory access by means of preprocessing, so as to improve the execution efficiency of the decomposition algorithm.

Since in the aforementioned first embodiment, each frame operation needs to read the relevant light field frame and decomposed frame from the disk of the host computer, and then transmit it to the parallel computing platform through the channel, the frequent transmission process is seriously reduced. The overall execution efficiency of the decomposition algorithm. Since each light field frame contains a total of 81 images with a 9x9 viewing angle, it occupies most of the memory capacity and data transmission bandwidth. Therefore, the second embodiment proposes that the parallel computing platform pre-processes the temporal filtering operation of the light field frame and stores it in its memory. In this way, when the parallel computing platform performs subsequent decomposition operations, it is not necessary to re-read the light field frame and the time domain filtering operation in each iteration update, so the overall execution efficiency can be effectively improved. The detailed steps are described as follows:

(h) Initialize each decomposed frame with random random numbers;

(i) preprocessing each frame in chronological order (steps (j)~(l));

(j) Read the current light field frame from the host's disk to the parallel computing platform;

(k) Calculate the contribution and give a weight according to the time proportion of the current decomposition frame to the affected light field frame;

(1) performing time filtering on the target light field frame according to the weight, to obtain the filtered light field frame, and store in the magnetic disk;

(m) cyclically perform the optimization calculation for each frame according to the time sequence (steps (n)~(p)) until the set number of iterations;

(n) reading out the current decomposed frame and the corresponding filtered light field frame from the disk of the host to the parallel computing platform;

(o) performing a decomposition algorithm on the filtered lightfield frame to optimize the current decomposed frame, subject to the constraints of fixing other decomposed frames (without optimization updates); and

(p) Write the updated decomposed frame back to the host's disk.

The image quality of the decomposed frame obtained by the implementation of this embodiment is not affected and is the same as that obtained in the first embodiment, but the memory bandwidth can be reduced by about 30% due to preprocessing, and this bandwidth reduction can save about 32% of the memory bandwidth. operation time.

(3) The third embodiment: it further proposes a segmentation processing method based on the first embodiment and the second embodiment. First, the complete light field film is divided into fixed-length film segments, and then the first embodiment is executed. and the decomposition algorithm of the second embodiment to improve its execution efficiency.

Since parallel computing platforms (such as image processing units GPU) are usually limited by memory capacity and cannot fully accommodate a complete light field movie, it is necessary to repeatedly read from the host's disk many times during iterative optimization. The data is stored in the memory of the parallel computing platform, resulting in a decrease in the computing speed. Therefore, this embodiment divides the complete light field video into a size that can be accommodated by the memory of the parallel computing platform, thus avoiding access to the host's disk during the segmented iterative calculation process, thereby greatly improving the execution of the decomposition algorithm efficiency.

In addition, in order to avoid the degradation of the quality of the decomposition algorithm caused by the segmentation calculation of the light field film, this embodiment can be performed by overlapping segmentation. As shown in FIG. 7 , the complete light field film can be divided into different segments SG1~SG1~ SG3 and adjacent segments (eg, SG1 and SG2, SG2 and SG3) partially overlap each other, for example, each segment includes 32 frames and overlaps with the preceding and following segments by 4 frames, but not limited thereto. The detailed steps are described as follows:

(q) segment the complete light field movie according to the memory capacity limit of the parallel computing platform;

(r) performing the preprocessing in the second embodiment on each light field frame in time sequence (steps (s1) to (s3));

(s1) Read the current light field frame from the disk of the host to the parallel computing platform;

(s2) Calculate the contribution and give a weight according to the time proportion of the current decomposition frame to the affected light field frame;

(s3) performing time-domain filtering on the target light field frame according to the weight to obtain the filtered light field frame, and storing it in the disk of the host computer;

(t) cyclically perform the optimization calculation for each segment in chronological order;

(u) Read out each decomposed frame of the current segment and the corresponding filtered light field frame from the disk of the host to the main memory;

(v) performing an optimization operation on each decomposed frame in the current segment until the set number of iterations;

(w) write each decomposed frame in the current segment back to the hard disk;

(x) transmitting the relevant decomposition frame of the currently processed frame and the corresponding filtered light field frame to the parallel computing platform;

(y) performing a decomposition algorithm on the filtered lightfield frame to optimize the current decomposition frame, subject to the constraints of fixing other decomposition frames (without optimization updates); and

(z) Write the updated decomposed frame back to main memory.

Compared with the first embodiment, although the image quality of the decomposed frames obtained by this embodiment is reduced by about 0.48 decibels (dB) on average, the peak signal-to-noise ratio (PSNR) image reconstruction quality is The access bandwidth is reduced by 100 times, which greatly reduces the disk access time, which previously accounted for a considerable proportion, so the overall execution time can be further reduced by 43%.

Another embodiment according to the present invention is a method for operating a computing display. In this embodiment, the operation method of the computing display can be applied to any decomposition type display, such as a multi-layer liquid crystal decomposition type display, and at least one display unit of the multi-layer display units is a liquid crystal layer, but not limited thereto.

Please refer to FIG. 8 . FIG. 8 is a flowchart illustrating an operation method of the computing display in this embodiment. As shown in FIG. 8, the operation method of the computing display may include the following steps:

Step S10: performing time domain filtering on a plurality of light field frames in the initial dynamic light field film to obtain filtered light field frames;

Step S12: Decompose the filtered light field frame into a plurality of decomposed frames through a decomposition algorithm, wherein the update rate of the plurality of decomposed frames is greater than the update rate of the plurality of light field frames; and

Step S14 : Display the multiple decomposed frames at multiple observation time points respectively, so as to be observed as the resulting dynamic light field movie.

It should be noted that step S10 and step S12 can be executed on the computing platform, and the computing platform can read the current light field frame from the plurality of light field frames stored in the disk of the host through the channel, and the updated light field frame by the computing platform The split frame can be written back to the host's disk through the channel, but not limited to this.

In one embodiment, before step S10 , the computational display operation method may further estimate weights according to the plurality of light field frames in the initial dynamic light field video, for step S10 to perform temporal filtering according to the weights. The decomposition algorithm in step S12 is a light field film decomposition algorithm. Step S12 may perform a decomposition algorithm on the filtered light field frame under the restriction of fixing other decomposition frames in the plurality of decomposition frames to optimize the current decomposition frame among the plurality of decomposition frames, and the weight may be based on the current decomposition The frame is calculated and contributed to the time proportion of the affected light field frame among the plurality of light field frames, but is not limited thereto.

In another embodiment, before step S12, the computing display operation method may further divide the complete dynamic light field video into a plurality of segments with a fixed length, and then execute the decomposition algorithm. The fixed length of the plurality of segments can be determined according to the capacity of the memory, so as to avoid frequent reading and writing between the memory and the disk. In addition, any two adjacent segments in the plurality of segments may partially overlap each other, so as to avoid the degradation of the quality of the decomposition algorithm caused by the segmentation process.

In practical applications, step S12 may firstly initialize each of the plurality of decomposed frames with random numbers, and then cyclically perform the optimization calculation of each decomposed frame in time sequence until the set number of iterations is reached.

Compared with the prior art, the computational display and the operation method thereof of the present invention use the weighted time domain filtering technology to preprocess the target light field video, so that the light field video decomposition algorithm can be calculated more efficiently, and the light field video is overlapped The segment is the optimization unit, and the decomposition algorithm is executed independently for each segment to improve the execution speed of the decomposition algorithm, so it can effectively solve the image defects and flickering that are prone to occur when the traditional decomposition light field display is playing videos. , to increase the comfort of the human eye.

a: light field frame

b: light field frame

a0~a3: Decomposition frame

b0~b3: decomposed frame

x: undefined fused frame

1: Computing Display

10: Load the mod

12: Weight Estimation Module

14: Time domain filter module

16: Decomposition calculation module

18: Display module

LV: Light Field Film

V0: Initial multi-frame multi-layer decomposition video

V1: Resulting multi-frame multi-layer decomposition video

LF: light field frame

W: weight

FLF: filtered light field frame

FF: break down frames

t0~t2: time point

LF0~LF2: Light field frame

FF0~FF7: Fragmentation frame

W0~W2: Weight

HM: host

DK: Disk

MM: main memory

PP: Parallel Processing Platform

PCIE: channel

MR: memory

CE: arithmetic unit

SG1~SG3: Fragments

S10~S14: Steps

The accompanying drawings of the present invention are described as follows: FIG. 1 is a schematic diagram illustrating the flickering phenomenon that is easily caused by the indefinable merged light field frames when a conventional split light field display plays a dynamic light field video. FIG. 2 is a functional block diagram of a computing display according to a preferred embodiment of the present invention. FIG. 3 is a schematic diagram illustrating that light field frames at different time points are respectively decomposed into decomposed frames, and any four adjacent decomposed frames can be observed as a defined fused light field frame. FIG. 4 is a schematic diagram of calculating the contribution according to the time ratio of the current decomposed frame to the affected light field frame and assigning weights respectively. FIG. 5 is a schematic diagram illustrating that a light field frame is decomposed into decomposed frames after time series filtering. FIG. 6 is a schematic diagram of video clips in different memories of the host and the parallel processing platform. FIG. 7 is a schematic diagram illustrating that a complete light field video is divided into different video segments and adjacent video segments partially overlap each other. FIG. 8 is a flowchart illustrating an operation method of a computing display according to another preferred embodiment of the present invention.

1: Computing Display

10: Load the mod

12: Weight Estimation Module

14: Time domain filter module

16: Decomposition calculation module

18: Display module

LV: Light Field Film

V0: Initial multi-frame multi-layer decomposition video

V1: Resulting multi-frame multi-layer decomposition video

LF: light field frame

W: weight

FLF: filtered light field frame

FF: break down frames

Claims (20)

A computational display, comprising: a time-domain filter module for performing time-domain filtering on a plurality of light field frames in an initial dynamic light field film to obtain a filtered light field frame; a decomposition calculation module, coupled to connected to the temporal filtering module for decomposing the filtered light field frame into a plurality of decomposed frames through a decomposition algorithm, wherein the update rate of the plurality of decomposed frames is greater than the update rate of the plurality of light field frames; and A display module includes a multi-layer display unit for displaying the plurality of decomposed frames at a plurality of observation time points respectively, so as to be observed as a result dynamic light field film. The computing display of claim 1, wherein the computing display is a multi-layer LCD factorized display and at least one display unit in the multi-layer display unit is a liquid crystal layer. The computational display of claim 1, wherein the decomposition algorithm adopted by the decomposition calculation module is a light field film decomposition algorithm, and the computational display further comprises: a weight estimation module coupled to the time domain filter The module is used for estimating a weight according to the plurality of light field frames in the initial dynamic light field video, so that the temporal filtering module can perform temporal filtering according to the weight. The computational display of claim 3, wherein the decomposition algorithm module performs the decomposition algorithm on the filtered light field frame under the constraint of fixing other decomposition frames in the plurality of decomposition frames to optimize the The current decomposed frame among the plurality of decomposed frames. The computational display of claim 4, wherein the weight is calculated according to the time proportion of the current decomposed frame to the affected light field frame of the plurality of light field frames. The computing display according to claim 1, wherein the decomposition algorithm adopted by the decomposition calculation module is a light field film decomposition algorithm, and the decomposition calculation module includes: a segment processing unit for the initial dynamic After the light field film is divided into a plurality of segments with a fixed length, the decomposition algorithm is executed by the decomposition calculation module. The computing display of claim 6, wherein the segment processing unit determines the fixed length of the plurality of segments according to a capacity of a memory, so as to avoid frequent communication between the memory and a disk read and write. The computational display of claim 6, wherein any two adjacent segments of the plurality of segments partially overlap each other, so as to avoid degradation of the decomposition algorithm quality caused by segmentation processing. The computing display of claim 1, wherein the decomposition algorithm module first initializes each decomposition frame of the plurality of decomposition frames with random numbers, and then cyclically performs optimization of each decomposition frame according to time sequence Calculated until the set number of iterations. The computational display of claim 1, wherein the time-domain filter module and the decomposition calculation module are disposed on a computing platform, and the computing platform stores the plurality of data from a disk of a host through a channel The current light field frame is read from the light field frame, and the decomposed frame updated by the computing platform is written back to the disk of the host computer through the channel. A computing display operating method for operating a computing display, including The following steps are included: (a) performing time-domain filtering on a plurality of light field frames in an initial dynamic light field film to obtain a filtered light field frame; (b) applying a decomposition algorithm to the filtered light field frame Decomposed into a plurality of decomposed frames, wherein the update rate of the plurality of decomposed frames is greater than the update rate of the plurality of light field frames; and (c) respectively displaying the plurality of decomposed frames at a plurality of observation time points to be observed as a The resulting dynamic light field movie. The method for operating a computing display as claimed in claim 11, wherein the computing display is a multi-layer liquid crystal decomposing display and includes a multi-layer display unit, and at least one display unit in the multi-layer display unit is a liquid crystal layer. The computing display operation method of claim 11, wherein the decomposition algorithm is a light field video decomposition algorithm, and the computing display operation method further comprises: according to the plurality of light field frames in the initial dynamic light field video A weight is estimated for step (a) to perform temporal filtering based on the weight. The computing display operation method of claim 13, further comprising: performing the decomposition algorithm on the filtered light field frame to optimize the multiple decomposition frames under the constraint of fixing other decomposition frames of the plurality of decomposition frames. The current decomposed frame among the decomposed frames. The method for operating a computational display as claimed in claim 14, wherein the weight is calculated according to the time proportion of the current decomposed frame to the affected light field frame of the plurality of light field frames. The computing display operation method of claim 11, wherein the decomposition algorithm is a light field video decomposition algorithm, and the computing display operation method comprises: dividing the dynamic light field video into a plurality of segments with fixed lengths , and then execute the decomposition algorithm. The computing display operation method of claim 16, wherein the fixed length of the plurality of segments is determined according to a capacity of a memory, so as to avoid frequent reading and writing between the memory and a disk. The computing display operation method of claim 16, wherein any two adjacent segments of the plurality of segments partially overlap each other, so as to avoid the degradation of the quality of the decomposition algorithm caused by segmentation processing. The computing display operation method as claimed in claim 11, wherein each of the plurality of decomposed frames is initialized with random numbers, and then the optimization calculation of each decomposed frame is performed cyclically according to time sequence, until up to the set number of iterations. The computing display operation method of claim 11, wherein steps (a) and (b) are performed on a computing platform and the computing platform stores the plurality of lights from a disk of a host through a channel The current light field frame is read from the field frame, and the decomposed frame updated by the computing platform is written back to the disk of the host through the channel.

Images