CN114915842A

CN114915842A - Video data processing method, module, chip and storage medium

Info

Publication number: CN114915842A
Application number: CN202110186240.5A
Authority: CN
Inventors: 陈实; 任敏鸿
Original assignee: Amlogic Shanghai Co Ltd
Current assignee: Amlogic Shanghai Co Ltd
Priority date: 2021-02-08
Filing date: 2021-02-08
Publication date: 2022-08-16

Abstract

The invention provides a video data processing method, a module, a chip and a storage medium, wherein the video data processing method uses reference compressed data corresponding to a reference position to replace compressed data at an error position, and decodes the replaced compressed data in real time, so that image information at the reference position can replace image information at the error position in real time, and a viewer does not feel abrupt on the image information after the error position replacement by using a vision pause phenomenon (Persistence of vision) in the video playing process, thereby being beneficial to improving the picture quality of a video and improving the quality feeling of the viewer on the video picture.

Description

Video data processing method, module, chip and storage medium

Technical Field

The embodiment of the invention relates to the technical field of video image processing, in particular to a video data processing method, a module, a chip and a storage medium.

Background

With the progress of high-definition digital video and image processing, people have increased the visual quality requirements of video and image, and high-definition digital video and image have come into daily life. At the same time, the amount of video and image data is also increasing dramatically, and efficient compression of data is a common way to facilitate data transmission and storage.

The decoder is a hardware device for restoring the audio and video code stream into the digital signal, and needs a large memory read-write bandwidth and space, the higher specification video or higher code rate code stream easily occupies more memory read-write bandwidth and space during decoding, and in order to reduce the memory read-write bandwidth and space occupied by the decoder, the standard code stream needs to be compressed in real time, so that the memory read-write bandwidth and space occupied by the decoder can be greatly reduced.

However, the prior art has problems in the process of decoding the compressed code stream.

Disclosure of Invention

The embodiment of the invention provides a video data processing method, a module, a chip and a storage medium, which are used for improving the video quality.

To solve the foregoing problems, an embodiment of the present invention provides a video data processing method, which processes original data of a video, and includes: compressing the original data of the video to form compressed data; decoding the compressed data, detecting the decoding process, and acquiring an error frame and an error position; acquiring a reference frame based on the error frame; acquiring a reference position of the reference frame based on the error position; acquiring reference compressed data corresponding to the reference position; replacing compressed data corresponding to the error position of the error frame with the reference compressed data; and decoding the replaced compressed data.

Accordingly, an embodiment of the present invention further provides a video data processing module, which processes original data of a video, and is characterized by including: the compression module is suitable for compressing the original data of the video to form compressed data; a decoder adapted to decode the compressed data; the detection module is suitable for detecting the decoding process and acquiring an error frame and an error position; a calculation module adapted to obtain a reference frame based on the error frame; and adapted to obtain a reference position of the reference frame based on the error position; and is suitable for obtaining the reference compressed data corresponding to the reference position; and is adapted to replace the compressed data corresponding to the error position of the error frame with the reference compressed data; and adapted to transmit the compressed data after replacement to the decoder for decoding.

Correspondingly, the embodiment of the invention also provides a chip which comprises the video data processing module.

Correspondingly, the embodiment of the present invention further provides a storage medium, where one or more computer instructions are stored, and the one or more computer instructions are used to implement the foregoing video data processing method.

Compared with the prior art, the technical scheme of the embodiment of the invention has the following advantages:

the embodiment of the invention provides a video data processing method, which comprises the steps of detecting the decoding process of compressed data, acquiring an error frame and an error position, and acquiring a reference frame based on the error frame; acquiring a reference position of a reference frame based on the error position; the reference compressed data corresponding to the reference position is obtained, the reference compressed data is used for replacing the compressed data corresponding to the error position of the error frame, and the replaced compressed data is decoded, that is, the reference compressed data corresponding to the reference position is used for replacing the compressed data corresponding to the error position, and the replaced compressed data is decoded in real time, so that the image information at the reference position can replace the image information at the error position in real time.

In an alternative, the step of compressing the original data of the video to form compressed data comprises: extracting volume data in the original data; storing the volume data; and acquiring the head address information corresponding to the volume data according to the storage address and the storage length of the volume data. In the embodiment of the invention, the header address information and the volume data are stored separately, and then the decoding process of the compressed data is detected to obtain the header address information corresponding to the error position of the error frame, and after the header address information corresponding to the reference position of the reference frame is obtained according to the header address information of the error position, the corresponding reference volume data can be quickly obtained by using the header address information corresponding to the reference position, so that the addressing efficiency is greatly improved, the real-time decoding is facilitated, the effect of concealing the decoding error position in real time is realized, and the video quality is improved.

Drawings

FIG. 1 is a flow chart illustrating a video data processing method according to an embodiment of the present invention;

FIG. 2 is a schematic flow chart of step 1 of forming compressed data according to an embodiment of the present invention;

FIG. 3 is a schematic diagram illustrating that volume data is stored at 4Kpage (m) in step 1 according to the embodiment of the present invention;

FIG. 4 is a schematic diagram of storing volume data at 4Kpage (m +1) in step 1 according to the embodiment of the present invention;

fig. 5 is a schematic diagram illustrating that header address information is stored in a 4K page in step 1 according to the embodiment of the present invention;

FIG. 6 is a diagram illustrating first header address information and first header address information in an error frame in step 4 according to an embodiment of the present invention;

FIG. 7 is a diagram illustrating first header address information and second header address information in a reference frame in step 4 according to an embodiment of the present invention;

FIG. 8 is a schematic flow chart of step 6 according to an embodiment of the present invention;

FIG. 9 is a functional block diagram of a video data processing module according to an embodiment of the present invention;

FIG. 10 is a functional block diagram of the compression module of FIG. 9;

FIG. 11 is a functional block diagram of the detection module of FIG. 9;

FIG. 12 is a functional block diagram of the computing module of FIG. 9.

Detailed Description

As can be seen from the background art, the prior art has a problem in the process of decoding the compressed code stream.

Specifically, the redundancy of the compressed data is greatly reduced, and errors may occur after the compressed data is transmitted through an unreliable channel; errors can be generated after decoding by a decoder, which affects video quality, however, errors occurring in decoding of a compressed code stream are more obvious than those occurring in decoding of a standard code stream, and decoding errors caused by compressed data errors need to be error-covered.

In the prior art, decoding errors caused by errors possibly occurring in compressed code stream data are not subjected to error concealment, because the video error positions are very obvious after the code stream is compressed and decoded, the video quality is lower than that of the video with the decoding errors occurring in single compression, and error concealment measures are not adopted, so that the video quality is seriously reduced.

The embodiment of the invention provides a video data processing method, which is used for processing original data of a video and comprises the following steps: compressing the original data of the video to form compressed data; decoding the compressed data, detecting the decoding process, and acquiring an error frame and an error position; acquiring a reference frame based on the error frame; acquiring a reference position of the reference frame based on the error position; acquiring reference compressed data corresponding to the reference position; replacing compressed data corresponding to the error position of the error frame by the reference compressed data; and decoding the replaced compressed data.

In order to solve the technical problem, embodiments of the present invention provide a video data processing method, a module, a chip, and a storage medium, where the video data processing method detects a decoding process of compressed data, acquires an error frame and an error position, and then acquires a reference frame based on the error frame; acquiring a reference position of a reference frame based on the error position; obtaining the reference compressed data corresponding to the reference position, replacing the compressed data corresponding to the error position of the error frame with the reference compressed data, and decoding the replaced compressed data, that is, in the embodiment of the present invention, the reference compressed data corresponding to the reference position is used to replace the compressed data corresponding to the error position, and the replaced compressed data is decoded in real time, so that the image information at the reference position can replace the image information at the error position in real time.

Fig. 1 is a flowchart illustrating a video data processing method according to an embodiment of the invention. The video data processing method is used for processing the original data of the video.

The original data of the video contains color information and redundant information. Because there is a large amount of data redundancy in the original data of the video, there is a large amount of space that can be compressed. Specifically, the color information refers to a color space (YUV); the redundant information includes: temporal redundancy, spatial redundancy, coding redundancy, and visual redundancy information.

In this embodiment, the video format includes MPEG-1, MPEG-2, MPEG-4, etc., MPEG1 is required to comply with ISO/IEC11172, and MPEG2 is required to comply with ISO/IEC 13818.

Specifically, the step of providing the original data of the video comprises: and coding and compressing the video data to form code stream data.

The video data is in a pixel representation form of a dynamic image, the data volume is huge, the storage space and the transmission bandwidth can not meet the requirements of storage and transmission at all, and the purpose of coding and compressing the video data is to compress the data so as to facilitate transmission. Specifically, the amount of data after compression is usually about 5% before compression. Moreover, the encoding compression can also enable information in the video data to be represented by a specified code stream according to a certain rule.

Specifically, in the process of encoding and compressing video data, the adopted encoding and compressing standard includes: AV1, H.264/AVC, H.265/HEVC, AVS2, VP9, and the like. As an example, h.265/HEVC is used to encode and compress video data, which has high resolution and can transmit a higher quality network video under a limited bandwidth, compared with h.264/AVC, the h.265/HEVC can play a video with the same quality by only half of the original bandwidth, thereby improving compression efficiency, robustness and error recovery capability, reducing real-time delay, reducing channel acquisition time and random access delay, and reducing complexity.

The step of providing raw data of the video further comprises: and decoding the code stream data to form original data of the video.

And decoding the code stream data to form original data of the video, and preparing for compressing the original data of the video to form compressed data subsequently.

It should be noted that, after the video data is encoded and compressed, code stream data is formed, the code stream data is expressed according to the type of the data in a specified form, and after the code stream data is decoded to form original data of the video, the original data of the video is arranged according to the content of the data in a certain syntax. For example, the color information and the redundant information in the original data have different syntax, which facilitates extracting useful data information in the subsequent data compression process.

As shown in fig. 2, step S1 is executed to compress the original data of the video to form compressed data.

And compressing the original data of the video to form compressed data, wherein compared with the original data of the video, the compressed data has the advantages of reduced data size, small occupied storage space, small required bandwidth in the transmission process and convenience for data transmission and storage. Correspondingly, in the process of subsequently adopting a decoder to restore the compressed data into a digital signal and generating a multi-frame dynamic image, the occupied memory read-write bandwidth and space are smaller.

In this embodiment, the compression protocol includes a compression algorithm, and in the step of compressing the original data of the video by the compression protocol to form compressed data, according to the compression algorithm of the compression protocol, the original data of the video is divided into a plurality of frames, an image of each frame is divided into a plurality of units, and each unit is divided into a plurality of compression blocks (blocks).

In this embodiment, the size of the compression block is related to the bandwidth of the hardware, and also depends on the output of the Loop Filter (Loop Filter). As an example, the compressed block is 64 pixels by 64 pixels.

Referring to fig. 2 in combination, in this embodiment, the step of compressing the original data of the video to form compressed data includes:

in step S11, the volume data (Body) in the raw data is extracted.

And extracting volume data in the original data for subsequent storage in a first position of a memory.

The step of extracting the volume data in the original data comprises: extracting color data in the original data; tree-coding and rearranging the extracted color data as the volume data. The proportion of the color data in the number of the original data is small, the data volume of the corresponding volume data is small, and the storage space and the transmission bandwidth required by the volume data are small.

In this embodiment, the color information and the redundant information in the original data have different grammars, and the process of extracting the volume data is a process of determining the color data according to a difference between the grammars of the color information and the redundant information. In other embodiments, the volume data in the original data may also be extracted by performing data transformation on the original data.

Step S12, storing the volume data.

And storing the volume data for decoding the volume data subsequently and restoring the volume data into dynamic image pixels.

In the step of forming the compressed data, the volume data is stored in a first location of a memory.

In this embodiment, a paged Memory Management Unit (MMU) is used to divide the Memory into 4K pages, and specifically, the volume data is stored in the 4K pages. The 4K page is larger, the corresponding memory occupying a page table (page table) is small, and under the condition of the same number of items of an address Translation cache (TLB), a larger memory can be tracked, so that the hit rate of the address Translation cache is improved; the number of times of writing into the disk can be reduced, and the disk I/O can be increased; in addition, the times of accessing the partner system can be reduced, and the cache utilization rate can be improved. In other embodiments, 8K pages may also be used for storage.

In the process of storing the bank data, the paged memory management unit preferentially divides a blank 4K page with a physical address smaller than the current physical address, so that the physical addresses of two 4K pages (e.g., 4K page (m) and 4K page (m +1)) connected in sequence can be continuous or discontinuous, and thus, the effects of fully utilizing the memory space and saving the memory space are achieved.

As an example, the volume data is stored in a compression unit of 32 pixels × 4 pixels. The larger the area of the storage unit is, the larger the power consumption of the correspondingly manufactured chip during operation is, the smaller the area of the storage unit is, the smaller the power consumption of the correspondingly manufactured chip during operation is, the consideration is given to the area of the subsequently manufactured chip and the power consumption design requirement, the consideration is given to the Loop filter (Loop filter) in the subsequent processing module to be generally 4 rows output, and the volume data takes 32 pixels × 4 pixels as the storage unit. In other embodiments, the length and width of the volume data may be other values. For example: 64 pixels × 4 pixels.

In this embodiment, in the step of storing the volume data, 32 bits are used as a storage unit. 4K is 4096 bits, and the 4K is divided by 32 bits, so that 128 storage units can be divided. The volume data includes different data amounts, and the data amount of the volume data usually occupies 1 to 8 memory units. In other embodiments, in the step of storing the volume data, 64 bits may also be used as a storage unit.

In this embodiment, the volume data is continuously stored in the first location of the memory. The volume data is continuously stored in the first position of the memory, which is beneficial to saving the memory space. Specifically, the volume data is stored continuously in 4K pages.

As shown in fig. 3, the continuous storage of the volume data in the first location of the memory is illustrated as an example.

The first volume data in the 4K page (m) is body (x), body (x) is stored from the first storage unit of the 4K page, and the data amount of body (x) occupies 5 storage units of the first row of the 4K page (m). Body (x +1) is the next volume data of Body (x), because the principle of storing volume data is continuous storage, Body (x +1) is stored next to Body (x), that is, Body (x +1) is stored from the sixth storage unit of the first row of 4K page (m).

In this embodiment, in the process of continuously storing volume data, when the space of the current 4K page is not enough to put down Body, the volume data is stored in the next 4K page. And subsequently, after the volume data is stored in the first position of the memory, acquiring the head address information corresponding to the volume data according to the storage address and the storage length of the volume data. Because the 4K pages are divided by using a paging Memory Management Unit (MMU), if a volume data is continuously stored in two 4K pages in the process of storing the volume data, correspondingly, a volume data corresponds to two Header address information headers, memory space is wasted, and in the subsequent decoding process, the data amount of the volume data of one storage unit can be obtained only by reading two Header address information, which tends to slow down the decoding rate. Therefore, the volume data is stored in one 4K page, only one head address information is correspondingly generated, and in the subsequent decoding process, the data information of one volume data can be obtained by reading one head address information, so that the improvement of the decoded data is facilitated.

Specifically, the step of storing the volume data in the first location of the memory includes: judging whether the residual space of the first 4K page can contain volume data or not; when the residual space of the first 4K page can contain the volume data, writing the volume data into the residual space of the first 4K page; and writing the volume data into the space of the second 4K page when the residual space of the first 4K page is judged not to be capable of containing the volume data.

As an example, as shown in fig. 3 and 4, in the process of storing the volume data, after the volume data Body (y) is stored, if 3 pixel units of 32 × 4 pixels remain in the 4K page (m) for storing the volume data and the data amount of Body (y +1) is more than 3 pixel units of 32 × 4 pixels, the volume data is stored in the page of 4K page (m + 1).

Step S13, obtaining Header address information (Header) corresponding to the volume data according to the storage address and the storage length of the volume data.

The head address information comprises a storage address and a storage length of corresponding volume data, so that the volume data corresponding to the head address information can be conveniently and quickly read according to the head address information in the follow-up process, and the addressing efficiency is improved.

It should be noted that the header address information is stored in a second location of the memory. Specifically, the header address information is stored in a 4K page.

In an embodiment of the present invention, the second location of the memory and the first location of the memory are in different areas of the memory. The head address information and the volume data are stored separately, the decoding process of the compressed data is detected subsequently, the head address information corresponding to the error position of the error frame is obtained, after the head address information corresponding to the reference position of the reference frame is obtained according to the head address information of the error position, the head address information corresponding to the reference position can be used for quickly obtaining the corresponding reference volume data, the addressing efficiency is greatly improved, the real-time decoding is facilitated, the effect of decoding the error position is covered in real time, and the video quality is improved.

In this embodiment, the step of storing the header address information in the second location of the memory includes: the header address information is stored in a matrix arrangement of 16 rows per column and every two columns. As an example, as shown by a dotted line box in fig. 5, the zeroth and first header address information are located in the first row, the second and third header address information are located in the second row, the second header address information is located below the zeroth header address information, and the third header address information is located below the first header address, and thus they are arranged. In the step of storing the volume data, most of the units of compressed blocks are 64 pixels × 64 pixels, so the header address information is stored in a manner of being arranged in a matrix of 16 rows per column and per two columns, and in the decoding process, the sum of the widths of two volume data is equal to 64 pixels, and the sum of the heights of 16 individual data is equal to 64 pixels, so that 64 pixels × 64 pixels can be formed, and particularly, the decoding efficiency can be improved in the decoding process.

In this embodiment, the memory space occupied by each piece of header address information is 32 bits.

Step S2 is executed to decode the compressed data, detect the decoded data, and obtain an error frame and an error position.

And decoding the compressed data to obtain dynamic image pixels of the video.

The step of decoding the compressed data comprises: and sequentially reading the volume data corresponding to the head address information and decoding the volume data. Because the volume data and the head address information are in one-to-one correspondence, the volume data corresponding to the head address information can be quickly read according to the head address information, and the addressing efficiency is high.

In this embodiment, a decoder is used to decode the compressed data. The decoder can decode the code stream conforming to the H.265 protocol, the VP9 protocol, the AV1 protocol or the AVS2 protocol.

In this embodiment, in the process of decoding the compressed data, resolution data of the video, the number of frames after decoding, and the first address information corresponding to each frame are obtained.

The resolution of the video is ready for subsequent acquisition of the reference position. The resolution data of the video is information originally existing in original data of the video, and the information of the video resolution can be obtained in the process of decoding the compressed data by adopting a decoder.

And the number of the decoded frames and the first address information corresponding to each frame are used for acquiring reference frames and reference positions subsequently. In the process of decoding the compressed data by adopting the decoder, the number of the decoded frames and the first address information corresponding to each frame can be obtained.

Specifically, in the process of decoding the compressed data, the first address information of each frame is read according to a decoder.

In this embodiment, the first header address information refers to header address information corresponding to the first 32 × 4 pixels compressed in each frame of image.

In this embodiment, the step of detecting the decoding process and acquiring the error frame and the error position includes: when the volume data corresponding to the first header address information x1 is decoded, an error occurs in the decoding, wherein the first header address information x1 is used as an error position; and using the first header address information closest to the first header address information x1 in the decoded first header address information as first header address information x0, and obtaining the error frame according to the first header address information x 0.

In this embodiment, in the step of detecting the decoding process of the compressed data, one or more of microcode detection, hardware decoder detection, location information detection, and number detection are adopted.

The microcode detection is used for detecting syntax errors of the Header, such as: the addresses do not match or the read length exceeds the 4K boundary.

The hardware decoder detects errors that are used to detect that a macroblock does not conform to a standard, such as a standard of 32x4 pixels compressed, decoded to other sizes.

The position information detection and the number detection are used for judging whether the position and the number of the decoded macro block generate errors or not.

Step S3 is executed to obtain a reference frame based on the error frame. And acquiring the reference frame to be used for acquiring the reference position of the reference frame for the subsequent reference frame according to the error position.

In this embodiment, the reference frame includes a frame previous to the error frame. The decoded compressed data can be converted into a video, the video playing is based on the phenomenon of visual pause of people, the video is composed of multiple frames of images, the change of two adjacent frames of images is small, therefore, the previous frame of the error frame is selected as the reference frame, the difference between the position of the reference frame and the image information of the error position is small, the image restoration degree of the error position is high, and the video quality is improved.

Specifically, the step of acquiring the reference frame based on the error frame includes: and in the decoded first head address information, the first head address information adjacent to the first head address information x0 is used as second first head address information y0, and a frame where the second first head address information y0 is located is the reference frame.

In other embodiments, the reference frame may further include a second frame preceding the error frame.

Step S4 is executed to obtain the reference position of the reference frame based on the error position, as shown in fig. 6 and 7.

And acquiring the reference position of the reference frame, and compressing the reference data corresponding to the reference position for subsequent acquisition.

Specifically, the step of obtaining the reference position of the reference frame based on the error position includes: second header address information y1 (shown in fig. 7) corresponding to the first header address information x1 in a reference frame is obtained according to the positional relationship between the first header address information x1 (shown in fig. 6), the first header address information x0 (shown in fig. 6), and the second header address information y0 (shown in fig. 7), and the second header address information y1 is used as the reference position.

Specifically, the relationship among the first header address information x1, the first-head address information x0, the second header address information y1, and the second-head address information y0 is:

x1-x0＝y1-y0；

accordingly, the second header address information y1 is y0+ x1-x 0.

Step S5 is executed: and acquiring reference compressed data corresponding to the reference position.

The reference compressed data refers to reference volume data corresponding to a reference position, and the reference volume data is subsequently used for replacing the compressed data corresponding to the error position.

In this embodiment, the step of obtaining the reference compressed data corresponding to the reference position includes: reference volume data corresponding to the second header address information y1 is acquired.

The head address information corresponds to the volume data, and correspondingly, the reference volume data corresponding to the second head address information y1 can be called quickly according to the second head address information y1, so that the data size processed in the addressing process is reduced, the addressing efficiency is greatly improved, the real-time decoding is realized, the effect of concealing the decoding error position in real time is realized, and the imaging quality of the video can be improved.

Step S6 is executed: and replacing the compressed data corresponding to the error position of the error frame by the reference compressed data.

The embodiment of the invention replaces the compressed data of the error position by the reference compressed data corresponding to the reference position and decodes the replaced compressed data in real time, so that the image information of the reference position can replace the image information of the error position in real time.

Specifically, with reference to fig. 8, step S6 is executed to replace the compressed data corresponding to the error position of the error frame with the reference compressed data, and step S6 includes:

in step S61, the reference volume data is sequentially stored in a First-in First-out queue (FIFO). First-in-first-out (FIFO) means that the first instruction to enter completes and retires before the second instruction is executed.

Step S62, sequentially writing the reference volume data in the fifo queue into the memory.

Specifically, the reference volume data is sequentially written into the first position of the memory, so that the memory space can be fully utilized, and the effect of saving the memory space is achieved.

In step S63, third header address information x3 is generated according to the storage address of the reference volume data in the memory and the storage length of the volume data.

The third header address information x3 is used to subsequently overwrite the first header address information x 1.

Step S64, overwriting the first header address information x1 with the third header address information.

The third header address information covers the first header address information x1, and in the subsequent decoding process, the reference volume data corresponding to the third header address information is called correspondingly and decoded into data information.

Step S7 is executed to decode the replaced compressed data.

And decoding the replaced compressed data in real time, so that the image information of the reference position can replace the image information of the error position in real time, and in the video playing process, the quality sense of the video image is improved, and the quality feeling of a viewer on the video image is improved.

In this embodiment, the decoder is used to decode the replaced compressed data. The decoder may decode reference compressed data compliant with an h.265 protocol, a VP9 protocol, an AV1 protocol, or an AVs2 protocol.

The decoder calls the replaced compressed data for decoding according to the third header address information x 3.

It should be noted that, the video data processing method further includes: and after the replaced compressed data is decoded, detecting the decoding process of the replaced compressed data.

And detecting the decoding process of the replaced compressed data to ensure that the image quality of the decoded video is higher.

And in the step of detecting the decoding process of the replaced compressed data, if no problem occurs, decoding the volume data corresponding to the next header address information.

If a problem occurs in the process of detecting the decoding process of the replaced compressed data, the steps S1 to S7 of the video processing method are performed again until the decoding process does not have a problem.

Correspondingly, the embodiment of the invention also provides a video data processing module. Fig. 9 is a functional block diagram of a video data processing module according to an embodiment of the present invention.

A video data processing module, configured to process raw data of a video, where the video data processing module includes: a compression module 10, adapted to compress original data of a video to form compressed data; a decoder 20 adapted to decode the compressed data; the detection module 30 detects the decoding process to obtain an error frame and an error position; a calculation module 40 adapted to obtain a reference frame based on the erroneous frame; and adapted to obtain a reference position of the reference frame based on the error position; and is suitable for obtaining the reference compressed data corresponding to the reference position; and is suitable for replacing the compressed data corresponding to the error position of the error frame by the reference compressed data; and adapted to transmit the compressed data after replacement to the decoder for decoding.

In this embodiment, the reference compressed data corresponding to the reference position is used to replace the compressed data at the error position, and the replaced compressed data is decoded in real time, so that the image information at the reference position can replace the image information at the error position in real time.

The original data of the video is arranged according to a certain syntax according to the content of the data. For example, the color information, the redundant information and the tree coding information in the original data have different syntaxes, which facilitates extracting useful data information in the subsequent data compression process.

And the compression module 10 is suitable for compressing the original data of the video to form compressed data.

The compression module 10 compresses the original data of the video to form compressed data, and compared with the original data of the video, the compressed data has the advantages of reduced data size, small occupied storage space, small bandwidth required in the transmission process, and convenience in data transmission and storage. Correspondingly, in the process of subsequently adopting a decoder to restore the compressed data into a digital signal and generating a multi-frame dynamic image, the occupied memory read-write bandwidth and space are smaller.

The compression protocol comprises a compression algorithm, in the step of compressing original data of the video to form compressed data through the compression protocol, the original data of the video is divided into a plurality of frames according to the compression algorithm of the compression protocol, an image of each frame is divided into a plurality of units, and each unit is divided into a plurality of compression blocks (blocks).

As shown in fig. 10, the compression module 10 includes: an extracting unit 11 adapted to extract volume data in the raw data.

The extraction unit 11 tree-encodes and rearranges original data of a video as volume data. And extracting volume data in the original data for subsequent storage in a first position of a memory.

The volume data includes: tree-coded and rearranged color data.

The proportion of the color data in the number of the original data is small, the data volume of the corresponding volume data is small, and the storage space and the transmission bandwidth required by the volume data are small.

In this embodiment, the color information and the redundant information tree code in the original data have different grammars, and the process of extracting the volume data is a process of judging the color data according to the difference between the grammars of the color information and the redundant information. In other embodiments, the volume data in the original data may also be extracted by performing data transformation on the original data.

As shown in fig. 10, the compression module 10 includes: a volume data storage unit 12 adapted to store the volume data.

The volume data storage unit 12 stores the volume data for subsequent decoding of the volume data to restore to dynamic image pixels.

In this embodiment, the volume data is stored in a first location of the memory. Specifically, the volume data is stored in the first location of the memory through a Central Processing Unit (CPU) and a bus.

In this embodiment, a paged Memory Management Unit (MMU) is used to divide the Memory into 4K pages, and specifically, the volume data is stored in the 4K pages. The 4K page is larger, the corresponding memory occupying a page table (page table) is small, and under the condition of the same number of items of an address Translation cache (TLB), a larger memory can be tracked, so that the hit rate of the address Translation cache is improved; the times of writing into the disk can be reduced, and the disk I/O can be increased; in addition, the times of accessing the partner system can be reduced, and the cache utilization rate can be improved. In other embodiments, 8K pages may also be used for storage.

As an example, the volume data is stored in a compression unit of 32 pixels × 4 pixels. The larger the area of the storage unit is, the larger the power consumption of the correspondingly manufactured chip during operation is, the smaller the area of the storage unit is, the smaller the power consumption of the correspondingly manufactured chip during operation is, and in consideration of the area of the subsequently manufactured chip and the power consumption design requirements, in combination with consideration of a Loop filter (Loop filter) in a subsequent processing module, the Loop filter is usually output in 4 lines, and the volume data takes 32 pixels × 4 pixels as the storage unit. In other embodiments, the length and width of the volume data may be other values. For example: 64 pixels × 4 pixels.

In this embodiment, in the step of storing the volume data, 32 bits are used as a storage unit. 4K is 4096 bits, and the 4K is divided by 32 bits, so that 128 storage units can be divided. The volume data has different data amounts, and the data amount of the volume data usually occupies 1 to 8 memory units. In other embodiments, in the step of storing the volume data, 64 bits may also be used as a storage unit.

Referring to fig. 3, as an illustration, the volume data is continuously stored in the first location of the memory.

In this embodiment, in the process of continuously storing volume data, when the space of the current 4K page is not enough to put down Body, the volume data is stored in the next 4K page.

After storing the volume data in the first position of the memory, the head address information corresponding to the volume data is obtained according to the storage address and the storage length of the volume data. Because the 4K pages are divided by using a paging Memory Management Unit (MMU), if a volume data is continuously stored in two 4K pages in the process of storing the volume data, correspondingly, a volume data corresponds to two Header address information headers, memory space is wasted, and in the subsequent decoding process, the data amount of the volume data of one storage unit can be obtained only by reading two Header address information, which tends to slow down the decoding rate. Therefore, the volume data is stored in one 4K page, only one piece of head address information is correspondingly generated, and in the subsequent decoding process, the data information of one volume data can be obtained by reading one piece of head address information, so that the improvement of the decoded data is facilitated.

The volume data storage unit 12 includes: a judging subunit 121, configured to judge whether the remaining space of the first 4K page can accommodate the reference volume data; a writing subunit 122, configured to write the reference volume data into the remaining space of the first 4K page when it is determined that the remaining space of the first 4K page can accommodate the reference volume data; and the memory is also used for writing the reference volume data into the space of the second 4K page when the residual space of the first 4K page is judged not to be capable of containing the reference volume data.

As an example, referring to fig. 3 and 4, in the process of storing the volume data, after the volume data (y) is stored, if 3 pixel units of 32 pixels × 4 pixels remain in the 4K page (m) for storing the volume data and the data amount of Body (y +1) is greater than 3 pixel units of 32 pixels × 4 pixels, the volume data is stored in the page of 4K page (m + 1).

As shown in fig. 10, the compression module 10 includes: a Header address information obtaining unit 13, adapted to obtain Header address information (Header) corresponding to the volume data according to the storage address and the storage length of the volume data.

The head address information comprises a storage address and a storage length corresponding to the volume data, so that the volume data corresponding to the head address information can be read quickly according to the head address information, and the addressing efficiency is improved.

It should be noted that the header address information is stored in a second location of the memory. In this embodiment, the header address information is stored in a 4K page.

In the embodiment of the invention, the second position of the memory and the first position of the memory are in different areas of the memory, the header address information and the volume data are stored separately, the decoding process of the compressed data is detected subsequently, the header address information corresponding to the error position of the error frame is obtained, and after the header address information corresponding to the reference position of the reference frame is obtained according to the header address information of the error position, the corresponding reference volume data can be quickly obtained by using the header address information corresponding to the reference position, so that the addressing efficiency is greatly improved, the real-time decoding is facilitated, the effect of concealing the decoding error position in real time is realized, and the video quality is improved.

In this embodiment, the header address information is stored in a matrix arrangement of 16 rows in each two columns. As an example, as shown by a dotted line box in fig. 5, the zeroth and first header address information are located in the first row, the second and third header address information are located in the second row, the second header address information is located below the zeroth header address information, and the third header address information is located below the first header address, and they are arranged. In the step of storing the volume data, most of the unit compressed blocks are 64 pixels × 64 pixels, so the header address information is stored in a manner of being arranged in a matrix of 16 rows per column and every two columns, and in the decoding process, the sum of the widths of two volume data is equal to 64 pixels, and the sum of the heights of 16 individual data is equal to 64 pixels, so that 64 pixels × 64 pixels can be formed, and particularly, the decoding efficiency can be improved in the decoding process.

A decoder 20 adapted to decode the compressed data. The decoder 20 decodes the compressed data to obtain dynamic image pixels of the video.

The decoder 20 is adapted to sequentially read the volume data corresponding to the header address information and decode the volume data. Because the volume data and the head address information are in one-to-one correspondence, the volume data corresponding to the head address information can be quickly read according to the head address information, and the addressing efficiency is high.

In this embodiment, the decoder 20 is used to decode the compressed data. The decoder 20 may decode a code stream conforming to a protocol such as h.265 protocol, VP9 protocol, AV1 protocol, or AVs 2.

In this embodiment, the decoder 20 is adapted to obtain resolution data of a video, the number of decoded frames, and first address information corresponding to each frame.

The resolution of the video provides for subsequent calculation module 40 to obtain the reference position. The resolution data of the video is information originally existing in the original data of the video, and the information of the video resolution can be obtained in the process of decoding the compressed data by using the decoder 20.

And the number of the decoded frames and the first address information corresponding to each frame are used for acquiring reference frames and reference positions subsequently. In the process of decoding the compressed data by using the decoder 20, the number of decoded frames and the first address information corresponding to each frame can be obtained.

Specifically, in the process of decoding the compressed data, the first address information of each frame is read according to the decoder 20 and the bus.

The detection module 30 is adapted to detect a decoding process and obtain an error frame and an error position.

The error frame is used for acquiring the reference frame subsequently, and the error position and the error frame are used for acquiring the reference position of the reference frame subsequently.

As shown in fig. 11, the detection module 30 includes an error frame acquisition unit 41 and an error position acquisition unit 42: an error position obtaining unit 41, adapted to decode a volume data corresponding to first header address information x1, where the first header address information x1 is used as the error position, and an error occurs in the decoding; the error frame obtaining unit 42 is adapted to obtain the error frame according to the first head address information x0 by using the head address information closest to the first head address information x1 in the decoded head address information as the first head address information x 0.

In this embodiment, the detection performed by the detection module 30 includes: one or more of microcode detection, hardware decoder detection, location information detection, and number detection.

The hardware decoder detects errors that are not compliant with a standard, such as compression of 32x4 pixels, for macroblocks decoded to other sizes.

And the position information detection and the number detection are used for judging whether the position and the number of the decoded macro block have errors or not.

It should be noted that, if no error is found in the process of detecting the decoding process by the detection module 30, the decoding of the address information of the next header is continued.

As shown in fig. 12, the calculation module 40 includes: a reference frame acquiring unit 51 adapted to acquire a reference frame based on the error frame; and a reference position acquiring unit 52 adapted to acquire a reference position of the reference frame based on the error position; and a reference compressed data acquisition unit 53 adapted to acquire reference compressed data corresponding to the reference position; and a data replacing unit 54 adapted to replace the compressed data corresponding to the error position of the error frame with the reference compressed data.

The embodiment of the invention replaces the compressed data of the error position by the reference compressed data corresponding to the reference position and decodes the replaced compressed data, so that the image information of the reference position can replace the image information of the error position, and in the video playing process, the visual pause phenomenon (Persistence of vision) is utilized to decode in real time and cover the effect of decoding the error position in real time, so that a viewer does not feel abrupt on the image information replaced by the error position, the picture quality of the video is favorably improved, and the quality of the video picture of the viewer is improved.

A reference frame acquiring unit 51 adapted to acquire the reference frame in preparation for subsequently acquiring a reference position of the reference frame based on the error position.

In this embodiment, the reference frame includes a frame previous to the error frame. The decoded compressed data can be converted into a video, the video playing is based on the phenomenon of visual pause of people, the video is composed of multiple frames of images, and the change of two adjacent frames of images is small, so that the previous frame of the error frame is selected as a reference frame, correspondingly, the difference between the position of the reference frame and the image information of the error position is small, the image restoration degree of the error position is high, and the video quality is favorably improved.

Specifically, the reference frame obtaining unit 51 is adapted to use, as the second first address information y0, the first address information adjacent to the first address information x0 in the decoded first address information, where a frame where the second first address information y0 is located is the reference frame.

In other embodiments, the reference frame may further include a second frame preceding the error frame

A reference position obtaining unit 52 adapted to obtain a reference position of the reference frame based on the error position.

And the reference position of the reference frame is used for acquiring the reference compressed data corresponding to the reference position.

Specifically, the reference position obtaining unit 52 is adapted to obtain, according to a position relationship between the first header address information x1 (shown in fig. 6) and the first header address information x0 (shown in fig. 6), and in combination with the second first header address information y0 (shown in fig. 7), the second header address information y1 (shown in fig. 7) corresponding to the first header address information x1 in the reference frame, and the second header address information y1 as the reference position.

x1-x0＝y1-y0；

accordingly, the second header address information y1 is y0+ x1-x 0.

A reference compressed data obtaining unit 53 adapted to obtain reference compressed data corresponding to the reference position.

The reference compressed data refers to reference volume data corresponding to a reference position, and the reference volume data is compressed data corresponding to an error position which is subsequently used for replacing the error frame.

A reference compressed data acquiring unit 53 adapted to acquire reference volume data corresponding to the second header address information y 1.

The header address information corresponds to the volume data, and correspondingly, the reference volume data corresponding to the second header address information y1 can be called quickly according to the second header address information y1, so that the data volume processed in the addressing process is reduced, the addressing efficiency is greatly improved, real-time decoding is realized, the effect of concealing the decoding error position in real time is realized, and the imaging quality of the video can be improved.

In this embodiment, the central processing unit reads the reference compressed data corresponding to the second header address information y1 through a bus.

A data replacing unit 54 adapted to replace the compressed data corresponding to the error position of the error frame with the reference compressed data.

As shown in fig. 12, the data replacement unit 54 includes: the reference volume data storing unit 541 is adapted to store the reference volume data in a first-in first-out queue in sequence. First-in-first-out (FIFO) refers to the first instruction to complete and retire before executing the second instruction.

The data replacement unit 54 includes: the reference volume data writing unit 542 is adapted to sequentially write the reference volume data in the first-in first-out queue into the memory.

The data replacement unit 54 includes: the third header address information obtaining unit 543 is adapted to generate third header address information x3 according to the storage address and the storage length of the reference volume data in the memory.

The data replacement unit 54 includes: an overwriting unit 544 adapted to overwrite the first header address information x1 with the third header address information x 3.

The third header address information x3 covers the first header address information x1, and in the subsequent decoding process, the reference volume data corresponding to the third header address information x3 is called accordingly to be decoded into data information.

In this embodiment, the replaced compressed data is transmitted to the decoder 20 through a bus for decoding.

The decoder 20 is further adapted to decode the replaced compressed data.

In this embodiment, the decoder 20 is used to decode the replaced compressed data. The decoder 20 may decode reference compressed data conforming to the h.265 protocol, the VP9 protocol, the AV1 protocol, or the AVs2 protocol.

The decoder 20 calls the replaced compressed data for decoding according to the third header address information x 3.

It should be noted that the video data processing method further includes: and after the replaced compressed data is decoded, detecting the decoding process of the replaced compressed data.

And detecting the decoding process of the replaced compressed data, and if no problem occurs, decoding the volume data corresponding to the next header address information.

And in the process of detecting the decoding process of the replaced compressed data, if a problem occurs, processing the compressed data by using the video processing module again until the decoding process does not have the problem.

Correspondingly, the embodiment of the invention also provides a chip, and the chip comprises the video data processing module. For the video data processing module, please refer to the description in the previous section, which is not described again.

Correspondingly, the embodiment of the invention also provides a storage medium, wherein one or more computer instructions are stored in the storage medium and used for realizing the video data processing method provided by the embodiment of the invention.

The storage medium is a computer-readable storage medium, and the storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a usb disk, a removable hard disk, a magnetic disk, or an optical disk, which may store various program codes.

The embodiments of the present invention described above are combinations of elements and features of the present invention. Unless otherwise mentioned, the elements or features may be considered optional. Each element or feature may be practiced without being combined with other elements or features. In addition, the embodiments of the present invention may be configured by combining some elements and/or features. The order of operations described in the embodiments of the present invention may be rearranged. Some configurations of any embodiment may be included in another embodiment, and may be replaced with corresponding configurations of the other embodiment. It is obvious to those skilled in the art that claims that are not explicitly cited in each other in the appended claims may be combined into an embodiment of the present invention or may be included as new claims in a modification after the filing of the present application.

Embodiments of the invention may be implemented by various means, such as hardware, firmware, software, or a combination thereof. In a hardware configuration, the method according to an exemplary embodiment of the present invention may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, and the like.

In a firmware or software configuration, embodiments of the present invention may be implemented in the form of modules, procedures, functions, and the like. The software codes may be stored in memory units and executed by processors. The memory unit is located inside or outside the processor, and may transmit and receive data to and from the processor via various known means.

The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the invention. Thus, the present invention is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A video data processing method for processing raw data of a video, comprising:

compressing the original data of the video to form compressed data;

decoding the compressed data, detecting the decoding process, and acquiring an error frame and an error position;

acquiring a reference frame based on the error frame;

acquiring a reference position of the reference frame based on the error position;

acquiring reference compressed data corresponding to the reference position;

replacing compressed data corresponding to the error position of the error frame by the reference compressed data;

and decoding the replaced compressed data.

2. The video data processing method of claim 1, wherein the step of compressing original data of the video to form compressed data comprises:

extracting volume data in the original data;

storing the volume data;

and acquiring the head address information corresponding to the volume data according to the storage address and the storage length of the volume data.

3. The video data processing method of claim 2, wherein the step of extracting the volume data in the raw data comprises: extracting color data in the original data; tree-coding and rearranging the extracted color data as the volume data.

4. The method of claim 2, wherein in the step of forming the compressed data, the volume data is stored in a first location of a memory, and the header address information is stored in a second location of the memory, the second location of the memory being different from the first location of the memory.

5. The video data processing method according to claim 2, wherein in the step of storing the volume data in a first location of a memory, the memory is a 4K page;

the step of storing the volume data in a first location of a memory comprises: judging whether the residual space of the first 4K page can contain volume data or not;

when the residual space of the first 4K page can contain the volume data, writing the volume data into the residual space of the first 4K page;

and writing the volume data into the second 4K page when the residual space of the first 4K page is judged not to be capable of containing the volume data.

6. The method of claim 4, wherein the step of storing the header address information in a second location of the memory comprises: the header address information is stored in a matrix arrangement of 16 rows per column and every two columns.

7. The video data processing method of claim 2, wherein resolution data of the video, the number of decoded frames, and first address information corresponding to each frame are obtained during decoding of the compressed data.

8. The video data processing method of claim 7, wherein the step of decoding the compressed data comprises: and sequentially reading the volume data corresponding to the head address information and decoding the volume data.

9. The video data processing method of claim 8, wherein the step of detecting the decoding process and obtaining the error frame and the error location comprises:

when the volume data corresponding to the first head address information is decoded, errors occur in decoding, and the first head address information is used as an error position;

and taking the first head address information which is closest to the first head address information in the decoded first head address information as first head address information, and acquiring the error frame according to the first head address information.

10. The video data processing method of claim 9, wherein the step of obtaining the reference frame based on the erroneous frame comprises:

and taking the first address information adjacent to the first address information in the decoded first address information as second first address information, wherein the frame where the second first address information is located is the reference frame.

11. The video data processing method of claim 10, wherein the step of obtaining the reference position of the reference frame based on the error position comprises:

according to the position relation between the first head address information and in combination with the second head address information, obtaining second head address information corresponding to the first head address information in a reference frame;

the step of obtaining the reference compressed data corresponding to the reference position comprises the following steps: and acquiring reference volume data corresponding to the second head address information.

12. The video data processing method of claim 11, wherein the step of replacing the compressed data corresponding to the error position of the error frame with the reference compressed data comprises:

sequentially storing the reference volume data into a first-in first-out queue;

sequentially writing the reference volume data in the first-in first-out queue into a memory;

generating third head address information according to the storage address of the reference volume data in the memory and the storage length of the volume data;

and utilizing the third header address information to cover the first header address information.

13. The method of claim 1, wherein the step of detecting the decoding of the compressed data comprises one or more of microcode detection, hardware decoder detection, location information detection, and number detection.

14. A video data processing module, configured to process raw data of a video, comprising:

the compression module is suitable for compressing original data of the video to form compressed data;

a decoder adapted to decode the compressed data;

the detection module is suitable for detecting the decoding process to acquire an error frame and an error position;

a calculation module adapted to obtain a reference frame based on the error frame; and a reference position adapted to obtain the reference frame based on the error position; and is suitable for obtaining the reference compressed data corresponding to the reference position; and is adapted to replace the compressed data corresponding to the error position of the error frame with the reference compressed data; and adapted to transmit the compressed data after replacement to the decoder for decoding.

15. The video data processing module of claim 14, wherein the compression module comprises:

an extraction unit adapted to extract volume data in the raw data;

a volume data storage unit adapted to store the volume data;

and the head address information acquisition unit is suitable for acquiring the head address information corresponding to the volume data according to the storage address and the storage length of the volume data.

16. The video data processing module of claim 15, wherein the volume data comprises tree-coded and rearranged color data.

17. The video data processing module according to claim 15, wherein the volume data is stored in a first location of a memory, the header address information is stored in a second location of the memory, and the second location of the memory is different from the first location of the memory.

18. The video data processing module of claim 15, wherein the volume data is stored in a memory, the memory including 4K pages;

the volume data storage unit includes:

the judging subunit is used for judging whether the residual space of the first 4K page can accommodate the reference volume data;

the writing subunit is used for writing the reference volume data into the residual space of the first 4K page when judging that the residual space of the first 4K page can contain the reference volume data; and the memory is also used for writing the reference volume data into the space of the second 4K page when the residual space of the first 4K page is judged not to be capable of containing the reference volume data.

19. The video data processing module of claim 15, wherein the header address information is stored in a matrix arrangement of 16 rows per column and per column.

20. The video data processing module according to claim 15, wherein the decoder is adapted to obtain resolution data of the video, a number of frames completed in decoding, and first header address information corresponding to each frame.

21. The video data processing module according to claim 20, wherein the decoder is adapted to sequentially read the volume data corresponding to the header address information and decode the volume data.

22. The video data processing module of claim 21, wherein the detection module comprises an error frame acquisition unit and an error location acquisition unit:

an error position obtaining unit, adapted to decode the volume data corresponding to the first header address information, when an error occurs in the decoding, the first header address information being used as the error position;

and the error frame acquisition unit is suitable for taking the first head address information closest to the first head address information in the decoded first head address information as first head address information and acquiring the error frame according to the first head address information.

23. The video data processing module according to claim 22, wherein the calculation module includes a reference frame obtaining unit adapted to use, as second first-head address information, first-head address information adjacent to the first-head address information in first-head address information for which decoding is completed, and a frame in which the second first-head address information is located is the reference frame.

24. The video data processing module according to claim 23, wherein the calculation module includes a reference position obtaining unit adapted to obtain second header address information corresponding to the first header address information in a reference frame based on a positional relationship between the first header address information, the first-first header address information, and in combination with the second-first header address information;

the calculation module includes the reference compressed data acquisition unit and is adapted to acquire reference volume data corresponding to the second header address information.

25. The video data processing module of claim 24, wherein the calculation module including a data replacement unit comprises:

the reference volume data storing unit is suitable for sequentially storing the reference volume data into a first-in first-out queue;

the reference volume data writing unit is suitable for sequentially writing the reference volume data in the first-in first-out queue into the memory;

the third head address information acquisition unit is suitable for generating third head address information according to the storage address of the reference volume data in the memory and the storage length of the volume data;

an overwriting unit adapted to overwrite the first header address information with the third header address information.

26. The video data processing module of claim 14, wherein the detection module employs a detection comprising: one or more of microcode detection, hardware decoder detection, location information detection, and number detection.

27. A chip, characterized by: the method comprises the following steps: a video data processing module according to any one of claims 14 to 26.

28. A storage medium storing one or more computer instructions for implementing the video data processing method of any one of claims 1 to 13.