KR102428831B1 - Method for transmitting data and transmitter therefor - Google Patents

Abstract

A data transmission method according to an embodiment is a method of transmitting video data for a display panel by a transmitter in electronic communication with a receiver through a wireless communication channel, the method comprising: receiving, by the transmitter, a data signal from a data source; receiving, by the transmitter, a return signal from the receiver; encoding, by the transmitter based on the return signal, the data signal using a plurality of encoder blocks to generate a layered encoded data stream, and decoding and displaying, by the transmitter, on the display panel transmitting the layered encoded data stream to the receiver for the purpose, wherein a first encoder block of the plurality of encoder blocks encodes the data signal, and each subsequent encoder block is an input of a preceding encoder block. and the output of the quantizer of the preceding encoder block.

Description

METHOD FOR TRANSMITTING DATA AND TRANSMITTER THEREFOR

The present disclosure relates to a data transmission method and a transmitter.

The demand for video transmission over the air is increasing due to the emergence of new applications and use cases. Technological advances in high-resolution display screens and the advent of high-quality video (HD, FHD, UHD, etc.) have increased bandwidth requirements for high-throughput transmissions in recent years. For example, uncompressed Ultra High Definition (UHD) video requires a bandwidth of 12 Gbps.

In addition to high data rate constraints, wireless video transmission is also time/delay sensitive. When providing 60 frames per second, the inter-frame time is 1/60 = 16.6 ms. Thus, any portion of a frame that is not received within 16.6 ms must be dropped so that the display device can start rendering the next frame and retransmitting data is usually not a viable option. In addition to high bandwidth and latency (latency) requirements, wireless channels are susceptible to interference, which can cause the quality of wireless channels to fluctuate unpredictably over time. Therefore, it can be difficult to provide guaranteed quality of service (QoS) for transmitting high quality video data over a wireless channel.

The above described herein is merely for improving the understanding of the background of the described technology, and accordingly, it may include information that does not constitute the prior art already known to those skilled in the art.

An embodiment of the present description is to provide a data transmission method and a system therefor.

A data transmission method according to an embodiment is a method of transmitting video data for a display panel by a transmitter in electronic communication with a receiver through a wireless communication channel, the method comprising: receiving, by the transmitter, a data signal from a data source; receiving, by the transmitter, a return signal from the receiver; encoding the data signal using a plurality of encoder blocks by the transmitter based on the return signal to generate a layered encoded data stream, and decoding and the display by the transmitter sending the layered encoded data stream to the receiver for on-panel display, wherein a first encoder block of the plurality of encoder blocks encodes the data signal, and each subsequent encoder block Encodes the difference between the input of the encoder block and the output of the quantizer of the preceding encoder block.

dynamically changing, by the transmitter, one or more parameters of the plurality of encoder blocks for subsequent encoding and transmission of the data signal based on at least one of a channel quality, a video quality, a codec requirement, a compression ratio requirement, or a data rate requirement It may further include the step of

Changing the one or more parameters of the plurality of encoder blocks may include adjusting the number of the encoder blocks.

Changing the one or more parameters of the plurality of encoder blocks may include adjusting a compression ratio of the encoder block.

Changing the one or more parameters of the plurality of encoder blocks may include adjusting at least one of a transform operation, adjusting a quantization table, or adjusting a variable length encoder algorithm used in the encoder blocks. may include

The method further includes selecting one profile from among a plurality of profiles based on the return signal, wherein each of the profiles may have different predefined parameters corresponding to the plurality of encoder blocks.

The different predefined parameters may include a difference in the number of the encoder blocks for at least two of the plurality of profiles.

The return signal may include an indicator of the quality of the wireless communication channel as measured by the receiver.

The return signal may include an indicator of the visual quality measured by the receiver.

A transmitter according to an embodiment is a transmitter for transmitting data for a display panel to a receiver through a wireless communication channel, the transmitter receiving a data signal from a data source; receive a return signal from the receiver; based on the return signal, decode the data signal using a plurality of encoder blocks to generate a layered encoded data stream; and transmit to the receiver for decoding the layered encoded data stream and display on the display panel, wherein a first encoder block of the plurality of encoder blocks encodes the data signal, and a subsequent encoder block Each encodes the difference between the input of the preceding encoder block and the output of the quantization unit of the preceding encoder block.

The transmitter dynamically changes one or more parameters of the plurality of encoder blocks for subsequent encoding and transmission of the data signal based on at least one of a channel quality, a video quality, a codec requirement, a compression ratio requirement, or a data rate requirement. can be configured to

Changing the one or more parameters of the plurality of encoder blocks may include adjusting a number of the encoder blocks.

Changing the one or more parameters of the plurality of encoder blocks may include adjusting a compression ratio of the encoder block.

Changing the one or more parameters of the plurality of encoder blocks may include adjusting at least one of a transform operation, adjusting a quantization table, or adjusting a variable length encoder algorithm used in the encoder blocks. can

The transmitter is further configured to select one profile from among a plurality of profiles based on the return signal, each of the profiles may have different predefined parameters corresponding to the plurality of encoder blocks.

The different predefined parameters may include a difference in the number of the encoder blocks for at least two of the plurality of profiles.

The return signal may include an indicator of the quality of the wireless communication channel as measured by the receiver.

The return signal may include an indicator of the visual quality measured by the receiver.

A receiver according to an embodiment is a receiver for receiving data for a display panel from a transmitter through a wireless communication channel, comprising: receiving a layered encoded data stream from the transmitter through the wireless communication channel; send a return signal to the transmitter; decoding the layered encoded data stream according to parameters of corresponding encoder blocks of the transmitter using a plurality of decoder blocks to generate a decoded data stream; and send the decoded data stream to the display panel for display, wherein a first layer of the layered encoded data stream is a compressed version of an input data stream, and Each subsequent layer is a compressed version of the difference between the input of the preceding encoder block and the output of the quantizer of the preceding encoder block.

According to the embodiment of the present description, data can be efficiently transmitted with low delay.

1 is a block diagram illustrating a wireless data transmission system illustrating a high level overview description of a cross-layer optimization system, in accordance with some embodiments of the present invention.
2 illustrates an exemplary layer-based compression scheme used in accordance with some embodiments of the present invention.
3 is a block diagram illustrating an exemplary architecture of a cross-layer optimization system and further details of some components, in accordance with some embodiments of the present invention.
4 illustrates an example of partitioning pixels into different types, in accordance with some embodiments of the present invention.
5 illustrates an example of a field of view of one eye in accordance with some embodiments of the present invention.
6 illustrates an example of reconstruction of a packet or layer structure in accordance with some embodiments of the present invention.
7 illustrates a header structure in accordance with some embodiments of the present invention.
8 illustrates an exemplary reconstruction of data in accordance with some embodiments of the present invention.
9 illustrates an exemplary reconstruction of pixel bits and packets in accordance with some embodiments of the present invention.
10 illustrates header fields available under the wireless communication standard.
11 is a flow diagram illustrating a process for cross-layer image optimization, in accordance with some embodiments of the present invention.
12A illustrates a transmitter architecture for a dynamic layering compression system and method in accordance with some embodiments of the present invention.
12B illustrates a receiver architecture for a dynamic layering compression system and method in accordance with some embodiments of the present invention.
13 shows a simplified example of how a block of data of size W x H bits may be processed (eg, encoded and decoded) in parallel, in accordance with an embodiment of the present invention.
14 illustrates various exemplary configurations for different data transmission profiles in accordance with some embodiments of the present invention.

An embodiment of the present invention will be described in detail with reference to the accompanying drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The following detailed description sets forth embodiments of systems and methods for cross-layer image optimization (CLIO) for wireless video transmission over multi-gigabit channels provided in accordance with the present invention. and does not represent the only aspect of the present invention. The description sets forth features of the invention in connection with the illustrated embodiments. It will be understood, however, that the same or equivalent functions and structures may be achieved by different embodiments, which are also intended to be included within the spirit and scope of the present invention. As indicated elsewhere herein, like reference numbers refer to like parts or features.

The future of display technology includes a world full of inexpensive displays provided by various wireless streaming devices (cell phones, set-top boxes, projectors, etc.). Transmission of high-quality video over wireless links has proven to be a challenge. While wireless devices are non-static, wireless links lack bandwidth and are susceptible to various kinds of noise. The degree of delay (latency) is also high and variable, which is particularly detrimental to video. Due to the stringent requirements of video transmission, a common design method in which different layers (eg, an application (APP) layer, a medium access control (MAC) layer, and a physical (PHY) layer) are designed independently is a high data rate It does not enable wireless data transmission. Accordingly, embodiments of the present invention provide a cross-layer approach in which information in one layer is used to change parameters of different layers. This flexibility allows for rapid adaptation to rapid changes in radio links.

The IEEE 802.11ad standard can provide the bitrate required for uncompressed full high definition (FHD) wireless video. 802.11ad operates in the 60 GHz band using channels with a bandwidth of 2.16 GHz and provides bandwidth up to 4.6 Gbps at the physical layer (PHY) using a single carrier, which is sufficient for uncompressed FHD video transmission. However, IEEE 802.11ad can only achieve maximum bandwidth in certain deployments. For example, IEEE 802.11ad requires that the transmitter and receiver be located within a short distance of each other and within line of sight (LoS). Accordingly, embodiments of the present invention provide an improved approach to wireless data transmission.

According to some embodiments, various features of the present invention are used to enhance and ensure QoS for video streaming over wireless networks, including physical (PHY), media access control (MAC) or application (APP) cross-layer solutions. can be used Accordingly, information from one layer (eg, MAC layer) can be used to optimize parameters of another layer (eg, APP layer). For example, in video streaming, the APP layer may use information about the channel quality for rate control (network aware). Lower layers may also be configured to use information about video traffic characteristics. According to various embodiments, the system provides dynamic profile partitioning, dynamic tagging for packets, unequal error protection for different layers in layer-based compression techniques, and importance level aware modulation/coding selection, packetization and bit or pixel Use pruning.

According to one embodiment, a cross-layer method may be used to optimize perceptual quality in delay limited scalable video transmission. In addition, Unequal Error Protection (UEP) may be used according to packet loss visibility of the PHY layer for each video layer. In addition, there is a buffer-aware source adaptation in the APP layer. The rate adaptation scheme is used for QoS-driven seamless handoff scheme based on IEEE 802.11 Media Independent Handover (MIH) framework. To control the rate, the quantization parameter (QP) is adapted for single layer coding (AVC/H.264) and the enhancement layer is dropped for scalable coding (SVC/H.264). Rate and traffic adaptation is also included along with admission control and automatic layer management to optimize QoS in terms of video quality and number of sessions granted in wireless video transmission. Traffic management, path selection and frame filtering are included as cross-layer optimization techniques for video streaming over UDP/RTP in cellular networks. The cross-layer framework optimizes the video codec, optimizes time slicing for layer coded transmission, and optimizes the energy efficiency of the wireless multimedia broadcast receiver considering the user's QoS level and the size and energy constraints of the display device. Adaptive Modulation and Coding Scheme (MCS).

Essentially, compression removes redundancy from the source, and thus is inherently more susceptible to transmission errors. There are several methods that compression systems can use to mitigate the effects of transmission errors on video streams. If the video stream is of quality, spatial or temporal scalability, then we can use the concept of Unequal Error Protection (UEP) to provide a higher level of protection for the more important bits of information. On the receiver side, if the video stream has the characteristics of error tolerance, error propagation is minimized and a good percentage of errors can be ignored. After decoding, error concealment may be applied to the decoded output. One technique using temporal error concealment is to save the previous frame and replay the current frame if it is corrupted. Another possibility is to predict the current pixel using the surrounding pixels of the region in error. In general, error concealment is used as a last resort when other methods fail.

1 is a block diagram illustrating a wireless data transmission system 100 in accordance with some exemplary embodiments of the present invention. The wireless data transmission system 100 includes a transmitter 102 and a receiver 106 . The transmitter 102 receives data (eg, uncompressed video data) from the data source 104 (eg, an external data source such as a video card computer system) at the input terminal 103 . The receiver 106 may be integrated into a user device or electronic device 108 (eg, an electronic display device, smartphone, television, tablet, etc.) having a display panel 110 that displays images and video. The display panel 110 includes a plurality of pixels, each pixel including a plurality of different color components (or sub-pixels) (eg, each pixel may include a red component, a green component, and a blue component) ) may be included.

From a video perspective, each frame of video is displayed (eg, briefly displayed) as an image on the display panel 110 . In some embodiments, the display panel 110 may be any device having an integrated display, such as, for example, a television, monitor, cellular phone, tablet computer, wearable device, augmented reality (AR) headset, or virtual reality (VR) headset. It may be included as part of the electronic device 108 .

In addition, the transmitter 102 and the receiver 106 have wireless radios 112 and 114 integrated therein, and the transmitter 102 is in wireless data communication with the receiver 106 via the wireless radios 112 and 114 . Accordingly, transmitter 102 and receiver 106 send and receive data to and from each other using any suitable wireless data spectrum and standard (eg, Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard). For example, transmitter 102 may send data along with parameters or characteristics of a cross-layer image optimization scheme to receiver 106 via direct (eg, data) channel 130 (eg, a wireless communication channel). and the receiver 106 sends feedback data or a return signal (eg, including channel or visual quality data) to the transmitter 102 via a return channel 132 .

Among other elements, the transmitter 102 includes an application (APP) layer 116 (including, for example, a display protocol and video encoder module), a media access control (MAC) layer 118 and a physical (PHY) layer ( 120) is further included. According to an embodiment of the present invention, various aspects of cross-layer image optimization are performed or implemented in the APP layer 116 , the MAC layer 118 and the physical layer 120 . Likewise, the receiver 106 includes an APP layer 122 , a MAC layer 124 and a PHY layer 126 .

The wireless data transmission system 100 includes a cross-layer image optimization (CLIO) module or engine operating as part of or in cooperation with the transmitter 102 to control the cross-layer image optimization system and method. 134) may be further included. CLIO module 134 may include, for example, a processor and a memory coupled to the processor, the memory storing instructions to be executed by the processor, which instructions cause the processor to cause the cross-layer image described herein. Implement various operations of optimization systems and methods. For example, CLIO module 134 may be in electronic communication with APP layer 116 , MAC layer 118 , and PHY layer 120 to exchange data and commands to implement the cross-layer image optimization systems and methods described herein. can be done

Therefore, as illustrated in FIG. 1 , the wireless data transmission system 100 implements various mechanisms for cross-layer image optimization in the APP layer 116 , the MAC layer 118 , and the PHY layer 126 . , receive data at the transmitter 102 and transmit data to the receiver 106 .

A video compression/decompression system or codec can be evaluated in essentially six different ways: compression, quality (distortion), complexity, delay, quality scalability and error tolerance.

. According to one embodiment, a layer-based compression architecture (same as or similar to the JPEG2000 standard) in which a video or data stream is parsed or divided into multiple quality layers, resolution layers, spatial locations or color components is a cross-layer image. It may be used for data or video compression as part of an optimization system and method (eg, executed by the CLIO module 134 ). A layer-based encoder according to an embodiment of the present invention performs the following steps: color transform for components, wavelet transform on each color channel, quantization of wavelet coefficients and finally, known as code-block. Arithmetic coding of rectangular regions of wavelet coefficients can be used. These encoders provide flexibility for the structure of the encoded data by allowing different sequences of these code-blocks based on color components, resolution levels, spatial block positions or layers. Individual code blocks may be independently decodable, but headers for these code blocks may depend on previous headers and thus be decoded in sequence.

A layer-based compression encoded data stream used in accordance with an embodiment of the present invention may begin with a frame header followed by a packet header and a sequence of data bits. As mentioned, a data stream can be partitioned in multiple quality layers, resolution levels, spatial positions, and color components. The example of FIG. 2 shows an example of a layer-based compression encoded data stream generated in a layer, in one example, the first layer may correspond to a compression level of 8:1; At the end of the second layer (which decoded all the first layer packets), the compression is set to 4:1. Finally, decoding the third layer packet may be equivalent to transmitting the stream with 2:1 compression. Embodiments of the present invention are not limited to the compression ratio or the number of layers described above. Rather, an embodiment of the present invention is such that the first or highest layer has the highest compression ratio, the next layer has data that, when added to the first layer, causes the compression ratio to be lower than the compression ratio of the first layer, and so on. The last layer will have data that, when added to each of the preceding layers, results in a relatively low, visually lossless, or visually near lossless (eg 1:1, 1.5:1, 2:1) compression ratio to the viewer. Any suitable number of layers having any suitable compression ratio may be used depending on the design and functionality of the cross-layer image optimization system and method.

According to some embodiments, when a stereoscopic image is transmitted to a receiver, in the context of a display of a VR display device, an error in the data transmission affects both the left and right images, even if the error is only on one side. can give Thus, in some embodiments of the present invention, a cross-layer image optimization system and method (eg, CLIO module 134 and/or video encoder module of APP layer 116) is {left image + right image difference). } to ensure that the slice is encoded as a multiplexed stream, after which {various left image + difference of right image}. This procedure can reduce (eg, in half) the number of cases where an error in data transmission impacts the visibility of an artifact.

An embodiment of the present invention in a cross-layer algorithm for improving the quality of video streaming is a modulation and coding scheme (MCS) based on one or more characteristics of a PHY layer, e.g., an indicator of a channel quality of the PHY layer. This changing dynamic adaptive modulation and coding (AMC) may be further used. A lower MCS value provides a lower bit error rate, but also achieves lower throughput. In one embodiment, the MAC layer with dynamic AMC may notify the APP layer about the MCS to adjust the compression rate based on the MCS.

Losing different bits and packets has different impact on video quality, embodiments of the present invention provide more protection for most significant bit (MSB) in uncompressed/compressed video, more protection for packet header in JPEG 2000 or It works to protect the more important bits in video reception using the UEP approach, including more protection for I-frames in MPEG-4. Protection can be achieved by using more robust MCS, retransmission or additional error correction codes such as Reed-Solomon.

3 is an exemplary structure of a cross-layer optimization system of the data transmission system 100 including control schemes at the APP, MAC and PHY layers, and further details of some components, according to some exemplary embodiments of the present invention. It is a block diagram illustrating The measurement and feedback block is used to estimate conditions such as bit error rate, video quality, and available bandwidth of a particular class of service. Then, the models of the APP layer and the MAC layer can minimize the distortion of the video quality by optimal bit allocation, reduce the number of bits required for forward error correction, and prioritize packets according to the impact of their loss. .

In some embodiments, as illustrated in FIG. 3 , at the APP layer, the system based on feedback and measurements from the receiver (eg, based on data received on the return channel 132 from the receiver 106 ) ) to enable/disable optical compression. The feedback may be based, for example, on video quality, and a measure of video quality, for example. It may be based on the signal-to-noise ratio (SNR) or peak signal-to-noise ratio (PSNR) reported by the receiver.

The MAC layer of FIG. 3 may include a number of functions, modules or procedures including, for example, pixel partitioning, retransmission and multiple CRC checking, packetization with features such as MSB/LSB separation, and different priority buffers. Based on the feedback received from the PHY layer or parameters of the APP layer, the system (eg, CLIO module 134 ) changes the parameters of these functions at the MAC layer and determines how these functions, modules or procedures should work. can For example, whether or not pixel partitioning is performed, and when pixel partitioning is performed, there is a method in which the partition is divided into packets. Further, the system may be configured to determine or select whether retransmission is enabled.

According to some embodiments, the system (eg, CLIO module 134 ) may utilize these features in a UEP implementation. For example, some of the packets may include such characteristics as retransmissions, or these characteristics may have different parameters for different packets, such as different MCS or MSB/LSB separations.

The PHY layer (eg, the PHY layer 302 along with the corresponding PHY layer 304 in the receiver) may also include multiple layers for implementation of UEP as illustrated in FIG. 3 . Different packets may be transmitted through different layers with different MCS values. Different layers may also be associated with different buffers with different priorities. According to some demonstrative embodiments, the system (eg, CLIO module 134 ) may associate different packets with different layers and priority buffers according to video quality and measurement feedback.

According to some exemplary embodiments of the present invention, the first step in optimizing a cross-layer image for uncompressed or compressed transmission is layering or partitioning. An embodiment of the present invention utilizes a plurality of predefined layering and/or partitioning profiles, which are described in more detail below. As part of a data transmission session, one of the different layering and/or partitioning profiles may be, for example, a type of display device, a different criterion or indicator for video visual quality (including peak signal-to-noise ratio (PSNR)); The selection is made based on one or more predefined parameters including channel quality, codec requirements or selection, MCS and data rate requirements, and the like. Each profile dictates or is defined by a different set of procedures at the transmitter and receiver, including, for example, other error correction techniques, compression or packetization.

In addition, embodiments of the present invention may use Unequal Error Protection (UEP) methods as part of cross-layer image optimization for ordered bit streams of different layers. The ordering of bit streams or packets can be performed on layered-based compressed video or uncompressed video. Different MCS values are used for different layers depending on the channel quality.

According to some exemplary embodiments of the present invention, dynamic tagging for packets (eg, as in IEEE 802.11ad) may be used as part of the cross-layer image optimization system and method of the present invention. have. Since the importance level recognition procedure is implemented in the MAC packetizer for the transmitter and the depacketizer at the receiver, the importance level and corresponding information can be transmitted or signaled to the receiver.

As described above, in embodiments of the present invention, the system may utilize dynamic layering and partitioning profiles as part of the cross-layer image optimization system and method of the present invention. The transmitter and receiver (eg, transmitter 102 and receiver 106 ) may utilize or agree to a plurality of predefined layering/partitioning profiles. These profiles may be selected based on different criteria for video visual quality (including PSNR), channel quality, codec requirements or selection, MCS, data rate requirements, and the like. The receiver then reverses the layering method and selects the corresponding algorithm(s) to be executed to further correct any transmission errors using together/or an error concealment algorithm corresponding to the selected profile.

In one embodiment, the system uses seven profiles for use with various channel conditions, although the number of profiles is not limited thereto, and any suitable number of profiles may be used depending on the design and functionality of the wireless data transmission system. can be

For example, profile “00” may be defined for when the channel operates under ideal or near-ideal conditions, while profile “06” may be defined for when the channel operates under very poor conditions. Profiles between "00" and "06" are then defined for various intermediate conditions.

For example, one profile (eg, profile “00” or ideal/nearly ideal conditional profile) may be defined for uncompressed video transmission when the channel quality is high. For such a profile, the pixel partitioning technique may assume that pixels next to each other are likely to contain very similar content. If any pixel from the group of 4 is missing, it can be recalculated from the other 3 captured pixels. Profile “00” can be selected by dividing the image into four types of pixels as illustrated in FIG. 4 . In one type, three colors of a pixel are packed into one packet. All 4 type packets are sent to the receiver. A receiver that knows that profile "00" is being transmitted can improve the image quality. For example, if the pixel data of any one of the pixels (or any color within the pixel) is missing, the receiver can do so by averaging a group of 4 or 8 surrounding pixels (the number of pixels averaging can be dynamic) Run an averaging method to replace missing pixels or colors. Image compression methods also rely on the idea that pixels next to each other have correlated values. This method is simpler to implement and can be more robust against bit errors compared to entropy-coding compression algorithms.

The second profile (eg profile “01”) may be selected by dividing the image into 4 types of pixels, similar to the first profile (profile “00”). However, three types of pixels are selected for transmission. The system calculates whether the resulting distortion is below a threshold by dropping one type of pixel. When this profile is signaled to the receiver, the receiver will estimate the missing pixel data value from the average of 3 (or more) other pixels within each of the 4 pixel types. The threshold may be selected based on different criteria including, for example, visual quality (eg, PSNR). The selection and threshold of non-transmitted pixel types may also be signaled.

A third profile (eg, profile “02”) may operate on a lower bandwidth than the first two profiles (eg, profiles “00” and “01”) described above (eg, thereby may be selected). For example, the third profile may not transmit the least significant bit (LSB) of a pixel in one or all packets. Then, at the receiver, an averaging algorithm can compensate for this bit. Another threshold level for distortion is a third profile (eg profile “02”) instead of the first or second profile (eg profile “01” or “00”) described above for uncompressed video. ") may be considered.

Other profiles may be considered for compressed video when the quality of the channel is low and the PHY layer signals a lower value for MCS. For example, multiple quality-layering may be used to define a new profile for wireless video transmission. In layered-based compression, multiple layers may be created as described above. According to an embodiment of the present invention, the highest layer may include data having the highest compression ratio among a plurality of layers, and when the lowest layer is added to a higher layer, a relatively low compression ratio (eg, 1:1 or 1:1.5 or any suitable or visually lossless compression ratio). For example, in a three-layer compression scheme (eg, as illustrated in FIG. 2 ), layer 1 may include compressed image data having a compression ratio of 4:1. When layer 2 is added to layer 1, it may include data forming a compressed image having a compression ratio of 2:1. Layer 3 may contain a bitstream where when added to Layer 1 and Layer 2 the resulting image has a compression ratio of 1:1 (or 1.5:1 or other visually lossless compression ratio).

The transmitter selects the layer to transmit according to channel quality, visual quality, codec requirements, etc. We have a fourth profile (eg, profile "03") when transmitting layers 1, 2 and 3 (or all layers), and a fifth profile (eg, profile "03") when transmitting layers 1 and 2 (or a sub-set of layers). For example, a sixth profile (eg, profile "05") may be defined when transmitting the profile "04") and the layer 1 or the layer with the highest compression ratio. For a seventh profile (eg, profile “06”), the transmitter may transmit a portion of one of the layers, eg, both layer 1 and layer 2 and only a portion of layer 3. By recognizing that a seventh profile (eg, profile “06”) is being transmitted, the receiver will either search for missing data information or execute an additional correction algorithm. The sender may indicate the length of the original packet, and also the transmitted packet length in the packet header. The receiver can accordingly recognize that part of the layer has not been received and infer the correct size of the received packet. In addition, the receiver alerts the decoder that part of the packet has not been received and it may request that the extra information be retransmitted or retransmitted by the transmitter.

The additional profile may be set according to the type of data to be transmitted, the nature of the display device on which the image is to be displayed, or the importance of a particular part of the image to other parts of the same image. For example, in accordance with some embodiments, a particular portion or region of an image may indicate that errors in data transmission for bits corresponding to one part of the image are more to the viewer than errors in data transmission for bits corresponding to other parts of the image. In the sense of being more perceptible, it may be more important than other parts of the image.

According to some embodiments, the cross-layer image optimization system and method may be deployed, for example, in the context of a VR display system in which the display device (eg, the display panel 110 ) is a VR display device. In this situation, the majority (eg, 90%) of the viewer's image perception focuses on and around the center of the image (eg, the center of the field of view, 15 degrees), because the user perceives different aspects of the image. This is because they tend to move their head to focus on a part. Thus, as illustrated in FIG. 5 , the cross-layer image optimization system and method (eg, CLIO module 134 ) is a second region of the image (eg, labeled in FIG. 5 with label “B”). the first or central region of the image (e.g., labeled “A”) rather than the region, eg, the region between the first region and the second field of view outside the first region, or between 15 and 20 degrees of the field of view. Data corresponding to the area labeled at 5, for example, an area of 15 degrees of view) may be prioritized. Also, both the first and second regions of the image are outside the third region of the image (eg, the region labeled “C” in FIG. 5 , eg, outside the first and second regions, or 20 degrees outside the field of view) area) can be prioritized.

Another method of identifying regions of an image of higher importance or priority is described in US Patent Application Serial No. 15/288,977, which is incorporated herein by reference in its entirety. For example, according to some embodiments, data corresponding to a region of the image with a higher importance has a higher level of importance or a higher degree of error protection than data corresponding to a region of the image with a lower importance. can be specified to have The region of the image with higher importance may vary depending on the type of display device corresponding to the receiver, for example, as described herein with reference to FIG. 5 .

Because different regions of an image may be prioritized differently, a cross-layer image optimization system and method (eg, CLIO module 134 ) determines that data in a first region, compared to data in other regions of an image, is A profile (e.g., a layering/partitioning profile as described above) or a UEP scheme (described in more detail below), such that it has the lowest level of error, the second region of the image has the second lowest level of error, and so on. ) can be selected. Further, in accordance with some embodiments, cross-layer image optimization systems and methods (eg, CLIO module 134 ) are configured such that errors from external regions or regions are the highest priority (eg, central) regions of the image. Data can be partitioned so that it does not propagate to

According to some embodiments, the cross-layer image optimization system may use a lower number profile (eg, a lower error-prone profile, eg, to prioritize different parts of an image for transmission). . 1st to 7th profiles) may be used. According to some embodiments, a separate profile may be specifically predefined according to the type of display device (eg, television, VR display device, AR display device, wireless medical display device, etc.), so that the data is exchanged with the image Depending on the priority level of the portion of data, transmit (e.g., compress, encode, packetize, bit dropping or pruning) or receive (e.g., decompress, decode, error Correction, de-packetization, retransmission) are transmitted with different procedures.

In one embodiment, selection criteria for a layering or partitioning profile may be based on video visual quality (eg, PSNR) and/or channel quality (eg, SNR). Layering identifies whether uncompressed video, uncompressed video with some packets dropped, or compressed video with different compression ratios should be transmitted. Different layers may be treated differently in terms of queue priority, delay tolerance, error protection and/or reliability. Correspondingly, the transmitter and receiver can transmit (e.g., compress, encode, packetize, bit drop or prune) or receive (e.g., decompress, decoding, error correction, de-packetization) may enable different procedures prior to or as part of it.

According to some embodiments, the selection of the profile may be transmitted to the receiver by bit selection of a MAC or PHY header packet. Based on the bit selection, the receiver can identify a selected method for layering. In addition to the profile number, any other information required for the selected profile may be sent to the receiver, including, for example, the number of layers in compression and/or the number of packets/bytes in each layer. The extra information may be predefined or transmitted by a bit pattern in the packet header or over a secure channel.

Accordingly, one or more embodiments of the present invention provide a system for selecting a set of predefined layering and/or partitioning profiles for transmitting data (eg, uncompressed video data) from a transmitter to a receiver over a wireless data communication channel. and methods. Profiles may be, for example, channel quality, video quality, codec requirements or selections, MCS and data rate requirements, nature or characteristics of transmitted data, characteristics of a display device at the receiver end (eg coupled to the receiver) or It may be selected according to various conditions or parameters including characteristics. The selection of the profile may be transmitted from the transmitter to the receiver by, for example, a bit sequence of a MAC layer and/or PHY layer header packet. Depending on the profile selected, additional information indicating one or more parameters for decompression, decoding, error correction, etc. may also be transmitted from the transmitter to the receiver (eg, in MAC layer and/or PHY layer header packets). Based on the information (eg, bit sequence) transmitted from the transmitter to the receiver, the receiver identifies a method and/or protocol used at the transmitter for data layering and/or partitioning and/or compression, and thus layering and It may become possible to select a corresponding algorithm or procedure to execute to reverse partitioning and/or compression and also correct any errors during transmission using an error concealment algorithm or other procedure corresponding to the selected profile. .

Embodiments of the present invention may further utilize priority-based unequal error protection (UEP) to protect more important bits (eg, to facilitate proper transmission of those bits), for example for video decoding. can For example, in layer-based compression, bits corresponding to or associated with a higher compression rate (e.g., corresponding to a base layer or layer 1) are more important in video decoding and thus require more protection . Also, bit errors in the packet header or frame header prevent the video decoder from decoding subsequent data bits. In one embodiment of a cross-layer image optimization system and method, the packet header is moved to the beginning of each frame as illustrated in FIG. 6 . Further, the packet header may be identified by unique leading and end values as illustrated in FIG. 7 . If the header is moved to the beginning of the frame, the identifier is also moved to separate the headers from each other. The trailing identifier may also be removed according to some example embodiments. The last identifier may be maintained to distinguish between the end of header information and data information of the next layer.

According to some embodiments, cross-layer image optimization systems and methods (eg, CLIO module 134 ) may use different protection techniques against errors or loss of wireless medium for these header and base layers. For example, different MCS values may be used for different layers or different forward error correction (FEC) algorithms may be used with different coding rates, according to some embodiments. In addition, the inequality error protection operation can be performed at the MAC layer, thus enabling embodiments of the present invention to improve or optimize the reconstruction engine and apply it to different codecs. For example, although the APP layer passes the entire data to the MAC layer, the MAC layer has more flexibility to perform the various cross-layer image optimization operations described herein. Also, by implementing UEP at the MAC layer, the system is relatively more dynamic because different functions can be implemented with different parameters depending on the particular situation (eg frame by frame or within a frame) as described herein. Because it can be changed. In addition, the embodiment of the present invention may thus become possible to perform only partial reconstruction based on the state of the radio channel.

The compression technique used according to an embodiment of the present invention may provide additional partitioning techniques in addition to layers of different compression ratios by providing different data components as illustrated in FIG. 8 . For example, various similar pixel bits (eg, pixel bits having the same or similar MCS values) may be reconstructed in a packet to be grouped together. Any suitable compression technique that can also partition its output into multiple layers with different priorities may be used. The same approach can be used for header and data movement. FECs with different MCS values or different coding rates may be used for headers or partitions of data.

In one embodiment, the header contains information about the length of the data packet that follows the header in the frame. In some examples, cross-layer image optimization systems and methods may transmit some of the data bits in each layer or partition according to channel conditions, MCS conditions, bandwidth conditions, and the like. Accordingly, the length of the data stream that can be transmitted can also be appended to the header information to be transmitted to the receiver. For example, a fixed or predetermined number of bytes (N bytes) may be reserved for identifying this length. According to some embodiments, the length of the data stream may be at the end of each header for easy identification.

Accordingly, systems and methods in accordance with some embodiments of the present invention may repartition bits in a layer or level based compression system according to priority, delay tolerance, and/or level of protection or reliability for transmission on a wireless data communication channel, or can be configured to reconfigure. A data frame contains a frame header and packet header bits, which require a higher level of protection compared to the data bits. The data bits of a frame may also be partitioned into different priority, protection and/or reliability levels. The header may be identified according to a predefined or known bit string, and the data bits may be ordered according to the order of importance. According to some embodiments, higher priority or lower delay tolerance bits may be moved or reconstructed at the beginning of the data frame. For example, a frame of data may begin with a frame header (eg, in order of priority or importance), followed by a packet header, followed by bits of data. A packet header may be separated or identified by a predefined or predetermined string of bits. Further, according to some embodiments, the length of the pruned and transmitted data bit stream may be indicated by inserting an indicator of the length corresponding to each packet header into, before or after the packet header. The receiver's decoder reconstructs the frame by, for example, moving the packet header and other data bits to their original location (prior to reconstruction) and removing any additional inserted data, such as data length bits, but adds to reconstructing the frame. use data.

Accordingly, embodiments of the present invention may provide implementations of inequality error protection for any layer and/or level based compression technique for cross-layer image optimization. Header, layer and/or data partition may be reconfigured according to importance or priority level, and such reconfiguration may be performed at the MAC layer of the transmitter. According to the cross-layer image optimization scheme, some portion of data from each packet may be removed or dropped, and information about this data may be incorporated into header information. Additionally, embodiments of the present invention may utilize unequal error protection, for example, by using different MCS values for forward error correction to protect information at different importance levels or differently. As part of inequality error protection in accordance with some exemplary embodiments of the present invention, all headers can be reconfigured to be grouped together (eg, in one location), so that all headers are, for example, in each header of a frame. Rather than using several small forward error corrections individually for the

Losing different packets affects visual quality differently, according to some embodiments of the invention, because different packets can be protected differently using UEP and layer/partition-based compression. For example, if a particular packet in the header or base layer is lost, the system may not be able to reconstruct the video at the receiver side. On the other hand, some packet loss may result in PSNR degradation. Also, as described above, depending on the type of display device used in the system, different portions of the image data may have a higher priority than other portions of the image data in terms of minimizing errors. Accordingly, embodiments of the present invention may further utilize dynamic UEP and importance aware tagging as described in more detail below.

As described above, cross-layer image optimization systems and methods in accordance with embodiments of the present invention operate to partition packets into different importance levels based on priority, delay tolerance, protection or reliability. Each importance level may be treated differently when transmitting and/or receiving.

In one embodiment, each frame is divided into multiple video packets with different importance levels. The system then tags each packet with a corresponding importance level and the corresponding bit is an aggregated MAC protocol data unit (A-MPDU) subframe and a physical layer convergence procedure (PLCP) protocol data unit (PPDU). ) is packed in the frame. In one example, a PPDU may be tagged with an importance level and may be dynamically packed with packets and A-MPDU subframes of the same importance level. When retransmission is enabled, the A-MPDU subframes that need to be retransmitted need to be queued and placed in the appropriate PPDU with the appropriate importance level.

According to another embodiment, each A-MPDU subframe may be tagged according to an importance level. In this case, all PPDUs are considered equally in terms of importance level and they are packed with A-MPDUs with different importance levels. In delay sensitive scenarios requiring retransmission, it may be advantageous to tag the A-MPDU subframe rather than the PPDU packet. Although tagging A-MPDU subframes rather than PPDU packets may use more overhead for tag information, this approach may be easier to manage packets of multiple importance levels.

According to various embodiments, a bit pattern may be added to the header of an A-MPDU or PPDU to tag a packet. The bit pattern may be M bits, where 0...0 represents the highest importance, and 1...1 represents the lowest importance. According to some embodiments, the value M may be predefined or the value M may be signaled as part of the header information. According to some embodiments, the transmitter and receiver may define 2^M importance levels and different procedures for each level.

For guaranteed quality of service for wireless video transmission via IEEE802.11ad or IEEE 802.11ay, a cross-layer image optimization system and method according to some exemplary embodiments of the present invention provides a dynamic importance level for each tagged importance level. UEP can be implemented through level-aware AMC. Different importance levels may be modulated with different MCS values or they may be encoded with different forward error correction (FEC) values or the same FEC values but with different coding rates.

In the related art system, the PHY may recommend an MCS index appropriate for the current channel situation. However, according to some embodiments of the present invention, the cross-layer image optimization system and method can adapt the recommended MCS for most of the bit stream. For example, the system may decrement the MCS index for a more important layer or packet and increase the MCS index for a less important layer or packet. 9 shows an example for uncompressed video transmission where the system selects different MCS values for the MSB/LSB portion of the bits representing the RGB color in a pixel. 4 MSB bits are sent to MCS-1, the next 2 bits are sent to MCS and the last 2 LSB bits are sent to MCS+1. It also includes regrouping of bits into packet/A-MPDU subframes such that bits with the same MCS index are grouped into the same packet/A-MPDU subframe.

For example, in compressed video transmission, as described above with respect to FIGS. 6 and 8 , after layered-based compression is performed and the bits are reconstructed according to the importance level of the transmission, the header and base layer 1 are MCS- 1 may be transmitted, layer 2 may be transmitted as MCS, and finally layer 3 may be transmitted as MCS or MCS+1. The header and layer 1 are thus more protected and transmitted with little or no error. Since these information bits are transmitted in the MCS-1, there is a possibility that the required bandwidth exceeds the supported bandwidth of the channel for the MCS. For compensation, a portion of layer 3 may be dropped and/or layer 3 bits may be modulated with MCS+1.

In IEEE 802.11ad, the MCS index is signaled only for each PPDU packet of 262,143 bytes. Therefore, if the PPDU is tagged with the importance level, the embodiment of the present invention may use the IEEE 802.11ad procedure to use the recommended MCS or modified MCS index for each packet. According to some embodiments, signaling in IEEE 802.11ad may be changed so that MCS is signaled for each A-MPDU subframe. For example, currently reserved bits or unused information in the header of each packet may be used. According to the IEEE 802.11ad standard, the header includes the fields of FIG. 10 . As an example, in a peer-to-peer connection, an embodiment of the present invention uses "address 4" to signal the MCS. An embodiment of the present invention may additionally add a new byte to the header to signal the MCS for each A-MPDU.

Thus, cross-layer image optimization systems and methods according to some exemplary embodiments of the present invention may provide data (e.g., for example, pixel data, packets, etc.). Each video frame may be divided into multiple importance levels and the corresponding bits may be packed into A-MPDU subframes. Different packets (eg, A-MPDU subframes, PPDU subframes, etc.) may be tagged according to their importance level, and according to some embodiments, different packets may be transmitted to the receiver according to their order of importance. Different packets (eg, A-MPDU subframes) may be grouped (eg, into PPDU packets) according to their importance level. A value or tag indicating the importance level may be transmitted from the sender to the receiver and added to the header of the packet. At the receiver end, the receiver may be enabled to determine the tag level, thus further enabling different error recovery and/or concealment procedures depending on the tagging. Also, data with different importance levels can be protected differently by selecting different modulation and coding scheme (MCS) values for each packet. The corresponding MCS value may be inserted into the header for the packet in addition to the tag.

Accordingly, embodiments of the present invention provide dynamic partitioning and/or tagging (e.g., in IEEE multi-gigabit 802.11 ) and importance recognition procedures. Embodiments of the present invention may enable tagging of each small packet within an A-MPDU (data) packet using multiple content driven tags. A tag may be transmitted and signaled to a receiver. Different procedures or parameters may be associated with tags to incorporate various aspects of the cross-layer image optimization mechanism described herein. In addition, embodiments of the present invention may use unequal error protection based tags such as AMC (MCS adaptation for tags).

According to various embodiments of the present invention, the system may continuously monitor various parameters and factors for cross-layer image optimization and adjust accordingly. For example, the system may monitor the return signal from the receiver, and as the environment changes, for example, by selecting a different profile, compression or error correction technique, etc. as described above for subsequent data transmission, cross- Layer image optimization can be adjusted accordingly.

11 is an exemplary flow diagram illustrating various operations of a method for cross-layer image optimization, in accordance with some exemplary embodiments of the present invention. However, the number and order of the operations illustrated in FIG. 11 are not limited to the illustrated flowchart. For example, according to some embodiments, the order of operations may be changed (unless otherwise indicated), or processes may include both additional or fewer operations without departing from the spirit and scope of the present invention. have. For example, various other features and operations described in this disclosure and the corresponding figures may be incorporated into a process, or certain operations may be omitted in accordance with some exemplary embodiments of the present invention.

Once the process begins, in step 1200 the system (eg, the wireless data transmission system 100 and/or the transmitter 102 ) receives data (eg, including uncompressed video data) from a data source. video data signal). In step 1202, return data is sent to the receiver (e.g., receiver 106) including various information regarding data transmission to the receiver, including, for example, channel quality, visual quality, and type of indication device coupled to the receiver. )) is received by Next, at step 1204, the system may identify the type of display device. In step 1206 , the layering and/or partitioning profile is determined, for example, by type of display device, different criteria or indicators for video visual quality (including peak signal-to-noise ratio (PSNR)), channel quality, codec It is selected and then transmitted based on one or more predefined parameters including requirements or selections, MCS and data rate requirements, and the like.

At step 1208 , layer or level-based compression may be performed on the data according to, for example, a selected layering/partitioning profile. Then, in step 1210, the layers, partitions, packets, or bits of data are reconstructed into groups or packets according to their importance level as described above.

At step 1212, data may be tagged to indicate a packet of data and/or a level of importance of the data. Then, in step 1214, unequal error protection may be performed to transmit the data to the receiver based on the importance level of the different types of data. In step 1216, data is transmitted to a receiver, for example, for display on a display panel.

As described above, in accordance with some embodiments of the present invention, the cross-layer image optimization system provides various predetermined data transmission profiles or parameters (eg, profile “00”) based on, for example, the channel quality between the transmitter and the receiver. from "07"), compression techniques, and/or error correction techniques for data transmission. To this end, according to an embodiment of the present invention, as a part of a cross-layer image optimization system, a dynamic layering compression structure for data compression and decompression may be used.

In the prior art, layer-based compression schemes can be iterative in nature and can be complex and difficult to implement in real-time hardware, especially in the context of cross-layer designs with multiple quality layers. Accordingly, some embodiments of the present invention may provide a system and method for real-time implementation of layer-based compression. Encoding and decoding data using one or more layers may require relatively low latency and operate in a cross-layer image optimization system (eg, as part of the wireless data transmission system 100 shown in FIG. 1 ). ) can be implemented relatively easily. Further, the dynamic layering compression system may use configurable block compression parameters in real time (eg, frame by frame) according to cross-layer image optimization and/or compression profiles.

12A and 12B illustrate transmitter and receiver structures, respectively, for a dynamic layering compression system and method in accordance with some embodiments of the present invention. 12A , the transmitter 102 receives data (uncompressed video data) from a data source 104 (eg, an external data source such as a video card computer system), and a wireless data transmission system ( 100) compresses the data according to one or more parameters. For example, as indicated by the parameters of a given data transmission profile (eg, profiles “00” through “07”) and/or data received by the return channel 132, as described above. Depending on the channel or video quality, the system can dynamically change how data is compressed and decompressed for transmission.

According to some embodiments, the CLIO module 134 of the transmitter 102 and/or the encoder of the APP layer 116 may adjust the compression ratio of the compressed data and the number of compression layers according to the parameters of the data transmission profile ( For example as described with respect to FIG. 2).

12A , depending on the selected data transmission profile, the system may use M (M is an integer greater than or equal to 1) compression or encoder blocks or layers 1220-1 through 1220-M. For example, according to some embodiments, the selected data transmission profile may use a single compression block 1220-1. Each encoder block 1220-1 to 1220-M includes a transform module or wavelet module 1222-1 to 1222-M, a quantization module 1224-1 to 1224-M and a variable length coder (VLC) module ( 1226-1 to 1226-M), respectively. In the transform modules 1222-1 to 1222-M, a transform operation using any suitable transform algorithm known in the art is performed on input data on a data block of size W x H bits, so that the output is sent to the quantization modules 1224-1 to 1224-M. The quantization modules 1224-1 to 1224-M perform a quantization operation on the transformed data, and an output thereof is provided to the VLC modules 1226-1 to 1226-M, where the VLC operation is performed. For data from VLC modules 1226-1 to 1226-M, forward error correction in corresponding forward error correction modules 1227-1 to 1227-M according to parameters of the selected data transmission profile is applied, and the corresponding compressed data layers 1228-1 to 1227-M may be transmitted to a compressed stream multiplexer 1230 and transmitted to a receiver, respectively.

Outputs of quantization modules 1224-1 to 1224-M are also sent to inverse quantization modules 1232-2 to 1232-M, where an inverse quantization operation is performed. Outputs of inverse quantization modules 1232-2 to 1232-M are sent to inverse wavelet modules 1234-2 to 1234-M.

The difference between the input data of each of the encoder blocks 1220-1 to 1220-(M-1) and the output of the inverse wavelet modules 1234-2 to 1234-M is, for example, a difference module or difference It is computed in operations 1240-2 to 1240-M and provided as input data to the next or subsequent encoder block. Thus, in each operation 1240-2 to 1240-M, the system identifies the missing data after the quantization operation by comparing it with the original input of the preceding encoder block, and assigns the missing difference data to the subsequent input encoder block. can be provided as input to

As shown in FIG. 12A , each encoder block 1220-1 to 1220-M has a corresponding compression ratio (eg, N:1, P:1, Q:1), and compresses W x H data. The total compression ratio for the block may be the sum of compression ratios. For example, for a data transmission profile with 4 encoder blocks, the first encoder block compresses the data to 1/8, the second to 1/8, the third to 1/4, and the fourth to 1 When compressed by /4, the total compression ratio may be 3/4 (1/8 + 1/8 + 1/4 + 1/4), and the compression ratio of uncompressed data to compressed data may be 1.333:1 .

According to some embodiments of the present invention, each of the encoder blocks 1220-1 to 1220-M is independently configured as part of the hardware of the encoder, so that the MAC layer matches the bitrate required to properly transmit data to the receiver. data can be encoded. Thus, as long as the system specifies or identifies which functional encoder block is used in the bitstream, the system can switch parameters of the encoder in real time (eg, frame by frame). For example, depending on the compression ratio requirements from the MAC layer, additional encoder layers may be added or removed, a specific transform, quantization table and variable length encoder may be changed, and/or the compression ratio parameters of each encoder block may be changed. The compression ratio may be changed according to requests from the MAC layer, the type of data to be transmitted, and/or the selected data transmission profile.

As shown in FIG. 12B , the receiver 106 and/or the decoder of the APP layer 122 of the receiver 106 , for example, via the data channel 130 in the compressed stream multiplexer 1244 , the transmitter 102 ) to start decoding by receiving the layered encoded data stream. Initially, forward error correction may be performed for each layer, eg, in the corresponding forward error correction modules 1246-1 to 1246-M, depending on whether forward error correction has been performed at the transmitter. Then, for each encoder module 1220-1 to 1220-M, the corresponding decoder module 1250-1 to 1250-M performs a decoding operation on each compressed layer of data. For each decoder module 1250-1 to 1250-M, the corresponding layer of compressed data is first sent to the VLC module 1252-1 to 1252-M, where the VLC operation is performed, and its output is inverse Each of the quantization modules 1254-1 to 1254-M are transmitted to perform an inverse quantization operation. Outputs of the inverse quantization modules 1254-1 to 1254-M are sent to the inverse transform or wavelet modules 1256-1 to 1256-M where an inverse transform operation is performed. Finally, the output of each of the decoder blocks 1250-1 to 1250-M is combined into a data stream for display on the display panel 110 , for example in combining modules or operations 1260 and 1262 . .

According to embodiments, at the receiver side, the layered encoded compressed data stream received by the receiver may include information and parameters regarding the encoding scheme used at the transmitter side. For example, the layered encoded compressed data stream may include parameters (eg, compression ratio, transform algorithm, etc.) used to generate the layered encoded compressed data and the number of encoder block layers. . According to some embodiments, such information and parameters regarding the encoding scheme may be conveyed in a header file of the data stream and/or information regarding the selected data transmission profile. When information is transmitted by transmitting the selected data transmission profile, the parameters of the encoding method may be stored in advance in the memory of the receiver side.

13 shows a simple example of how data blocks of size W x H bits can be processed (eg, encoded and decoded) in parallel. For example, as shown in FIG. 13 , in an embodiment where the system uses encoder and decoder modules of three layers (first layer, second layer, third layer), a data block is processed by one encoder/decoder layer After that, the subsequent encoder/decoder layer can process the corresponding block while at the same time the previous encoder/decode layer process another data block. Thus, multiple data blocks can be processed in parallel by different encoder/decoder layers, and thus the overall delay of the layered compression system can be very low.

14 illustrates various exemplary configurations for different data transmission profiles in accordance with some embodiments of the present invention. The abscissa or x-axis direction represents frame time and the vertical or y-axis direction represents data rate (e.g. bits per pixel), so the total area of each rectangle is the relative total transmitted per layer. Indicates the number of bits. For example, in the case of the first configuration (configuration 1), the third compression layer (layer 3) may have half the total number of bits of the first and second compression layers (layer 1, layer 2). The first configuration can be used, for example, in an environment where there is a very high reliability wireless channel for transmitting video data of relatively high quality. In the case of the second configuration (configuration 2), the total number of bits of the data of the first layer (layer 1) and the forward error correction (layer 1FEC) of the layer 1 are the second layer (layer 2) and the third layer (layer 3) may be the same as the number of bits of data of The second configuration (configuration 2) can be used, for example, in a situation where there is a high-reliability radio channel, but sometimes bad channel statistics and transmitting relatively high-quality video data. In the third configuration (configuration 3), with the same amount of forward error correction data for the first and second layers (Layer 1, Layer 2), the second layer (Layer 2) is the first layer (Layer 1). It has twice the number of bits of data, and there are no layers of other data. The third configuration (configuration 3) can be used, for example, in situations where there is a low reliability channel for transmitting medium quality video.

Accordingly, embodiments of the present invention may provide an efficient layered data compression system compatible with the cross-layer image optimization system described above, including an encoder at the transmitter and a decoder at the receiver. For example, encoding of a layer of compressed data can be achieved with a relatively low delay, and can also be implemented relatively easily in hardware. In addition, block compression parameters can be configured in real time (eg, frame by frame) to accommodate cross-layer image compression goals.

A wireless data transmission system and/or any other related device or component in accordance with an embodiment of the invention described herein may include any suitable hardware, firmware (eg, application specific integrated circuit), software, or software, firmware, hardware It can be implemented using a suitable combination of For example, the various components of a wireless data transmission system may be formed on one integrated circuit (IC) chip or on separate IC chips. In addition, various components of the display device may be implemented on a flexible printed circuit film, a tape carrier package (TCP), a printed circuit board (PCB), or formed on the same substrate as the display device. In addition, various components of the wireless data transmission system may execute on one or more processors, on one or more computing devices, processes or threads that execute computer program instructions, and interact with other system components to perform the various functionality described herein. can be The computer program instructions are stored in a memory that may be implemented in a computing device using a standard memory device, such as, for example, random access memory (RAM). The computer program instructions may also be stored in other non-transitory computer readable media, such as, for example, a CD-ROM, flash drive, and the like. Furthermore, those skilled in the art will appreciate that the functionality of various computing devices may be integrated or combined into a single computing device, or that the functionality of a particular computing device may be distributed across one or more computing devices, without departing from the scope of the exemplary embodiments of the present invention. should be aware that

While the present invention has been described with respect to specific embodiments, those skilled in the art will not encounter difficulties in devising variations to the described embodiments without departing from the scope and spirit of the invention. Moreover, the present invention itself may suggest solutions for other tasks and adaptations for different applications to those skilled in the various technical fields. The intention of the present application is to cover all such uses of the present invention and changes and modifications that may be made to the embodiments of the present invention, which may be chosen for the purposes of the present disclosure without departing from the spirit and scope of the present invention, to be covered by the claims. can Accordingly, the present embodiments are to be regarded in all respects as illustrative rather than restrictive, and the scope of the present invention will be indicated by the appended claims and their equivalents rather than by the above description.

Claims (10)

A method of transmitting video data for a display panel by a transmitter in electronic communication with a receiver via a wireless communication channel, the method comprising:
receiving, by the transmitter, a data signal from a data source;
receiving, by the transmitter, a return signal from the receiver;
encoding, by the transmitter based on the return signal, the data signal using a plurality of encoder blocks to generate a layered encoded data stream; and
sending, by the transmitter, the layered encoded data stream to the receiver for decoding and display on the display panel;
A first encoder block of the plurality of encoder blocks encodes the data signal, and each subsequent encoder block encodes a difference between an input of a preceding encoder block and an output of a quantizer of the preceding encoder block.
data transmission method. In claim 1,
dynamically changing, by the transmitter, one or more parameters of the plurality of encoder blocks for subsequent encoding and transmission of the data signal based on at least one of a channel quality, video quality, codec requirement, compression ratio requirement, or data rate requirement Further comprising the step of, data transmission method. In claim 2,
and changing the one or more parameters of the plurality of encoder blocks comprises adjusting a number of the encoder blocks. In claim 2,
and changing the one or more parameters of the plurality of encoder blocks comprises adjusting a compression ratio of the encoder block. In claim 2,
Changing the one or more parameters of the plurality of encoder blocks may include adjusting at least one of a transform operation, adjusting a quantization table, or adjusting a variable length encoder algorithm used in the encoder blocks. A data transmission method comprising: In claim 1,
and selecting one profile from among a plurality of profiles based on the return signal, wherein each of the profiles has different predefined parameters corresponding to the plurality of encoder blocks. In claim 6,
and the different predefined parameters include a difference in the number of the encoder blocks for at least two of the plurality of profiles. In claim 6,
and the return signal includes an indicator of the quality of the wireless communication channel as measured by the receiver. In claim 2,
and the return signal includes an indicator of the visual quality measured by the receiver. A transmitter for transmitting data for a display panel to a receiver via a wireless communication channel, the transmitter comprising:
receive a data signal from a data source;
receive a return signal from the receiver;
based on the return signal, decode the data signal using a plurality of encoder blocks to generate a layered encoded data stream;
send to the receiver for decoding the layered encoded data stream and display on the display panel;
a first encoder block of the plurality of encoder blocks encodes the data signal, and each subsequent encoder block encodes a difference between an input of a preceding encoder block and an output of a quantization unit of the preceding encoder block
transmitter.

Images