WO2022206217A1 - 在视频编码装置中进行图像处理的方法、装置、介质及系统 - Google Patents

在视频编码装置中进行图像处理的方法、装置、介质及系统 Download PDF

Info

Publication number
WO2022206217A1
WO2022206217A1 PCT/CN2022/077311 CN2022077311W WO2022206217A1 WO 2022206217 A1 WO2022206217 A1 WO 2022206217A1 CN 2022077311 W CN2022077311 W CN 2022077311W WO 2022206217 A1 WO2022206217 A1 WO 2022206217A1
Authority
WO
WIPO (PCT)
Prior art keywords
block
memory
encoded
image data
search window
Prior art date
Application number
PCT/CN2022/077311
Other languages
English (en)
French (fr)
Inventor
赵娟萍
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2022206217A1 publication Critical patent/WO2022206217A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/156—Availability of hardware or computational resources, e.g. encoding based on power-saving criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/423—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation characterised by memory arrangements

Definitions

  • the present application belongs to the technical field of electronic devices, and in particular, relates to a method, device, storage medium, electronic device and system for image processing in a video encoding device.
  • Video Encoder VENC
  • the video encoding apparatus may encode video images. When encoding one frame of video image, it is usually necessary to read the data amount of multiple frames of encoded video images.
  • Embodiments of the present application provide a method, an apparatus, a storage medium, an electronic device, and a system for performing image processing in a video encoding apparatus.
  • an embodiment of the present application provides a method for image processing in a video encoding device, the method comprising:
  • the to-be-encoded block is encoded according to the relative relationship between the matched block and the to-be-encoded block.
  • an embodiment of the present application provides an apparatus for performing image processing in a video encoding apparatus, the apparatus comprising:
  • a first determining module configured to determine the block to be encoded from the current frame image
  • the second determining module is configured to determine, from the current frame image, a first area that needs to be read repeatedly for multiple times, and store the image data of the first area in a preset memory, where the power consumption of the preset memory less than the preset power consumption threshold;
  • a reading module for reading the image data of the first area from the preset memory
  • a third determining module configured to determine a matching block matching the block to be encoded from the first region according to the image data of the first region
  • An encoding module configured to encode the to-be-encoded block according to the relative relationship between the matched block and the to-be-encoded block.
  • embodiments of the present application provide a storage medium on which a computer program is stored, and when the computer program is executed on a computer, causes the computer to execute the image encoding in a video encoding apparatus provided by the embodiments of the present application method of processing.
  • the embodiments of the present application further provide an electronic device, including a memory, a processor, and a video encoding apparatus.
  • the processor is configured to execute the computer program stored in the memory by invoking the computer program provided in the embodiments of the present application.
  • a method for image processing in a video encoding device is configured to execute the computer program stored in the memory by invoking the computer program provided in the embodiments of the present application.
  • an embodiment of the present application further provides an image processing system, including a video encoding device, a first memory, and a second memory, wherein the power consumption of the second memory is greater than a first preset of the power consumption of the first memory.
  • the video encoding device includes a third memory, the read speed of the third memory is greater than the second preset multiple of the read speed of the first memory, the first memory and the second memory respectively store In the reconstructed frame image of the current frame image, the image data that needs to be read repeatedly for many times, when the video encoding device encodes, from the first memory according to a first preset number of times and from the first memory according to a second preset number of times.
  • the second memory reads the image data that needs to be read repeatedly for many times, and determines the image data in the search window (Search Window, SWin) from the image data that needs to be read repeatedly.
  • the image data in the window is stored in the third memory, and the video encoding device reads the image data in the search window from the third memory, and determines a matching block that matches the block to be encoded,
  • the coding is performed according to the block vector (Block Vector) and residual of the matching block and the block to be coded.
  • FIG. 1 is a schematic flowchart of a method for performing image processing in a video encoding apparatus provided by an embodiment of the present application.
  • FIG. 2 is a schematic structural diagram of a video compression system in the related art.
  • FIG. 3 is a schematic diagram of data storage in a video encoding apparatus in the related art.
  • FIG. 4 is a schematic diagram of increasing the number of channels of a dynamic random access memory (DRAM) for data access in the related art.
  • DRAM dynamic random access memory
  • FIG. 5 is a schematic diagram of an alignment of square blocks in the related art.
  • FIG. 6 is a schematic diagram of a hierarchical search provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a non-hierarchical search provided by an embodiment of the present application.
  • Inter-frame ME inter-frame motion estimation
  • IntraBC intra-frame block copy
  • FIG. 9 is another schematic flowchart of a method for performing image processing in a video encoding apparatus provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of a scene of searching in a reconstructed frame image of a current frame image according to an embodiment of the present application.
  • FIG. 11 is a schematic diagram illustrating the comparison of power consumption when reading data between a static random-access memory (SRAM) and a dynamic random-access memory according to an embodiment of the present application.
  • SRAM static random-access memory
  • FIG. 12 is a schematic structural diagram of a video compression system using a system cache (Sys$) provided by an embodiment of the present application.
  • FIG. 13 is another schematic structural diagram of a video compression system using a system cache (Sys$) provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a video compression system using a system buffer memory (System Buffer, SysBuf) provided by an embodiment of the present application.
  • System Buffer System Buffer, SysBuf
  • FIG. 15 is a schematic diagram of a scenario when one coding block row is moved down according to an embodiment of the present application.
  • FIG. 16 is a schematic diagram of a power consumption curve when reading data from a multi-channel DRAM according to an embodiment of the present application.
  • FIG. 17 is a schematic diagram of a power consumption curve when data is read from Sys$ or SysBuf and DRAM, respectively, according to an embodiment of the present application.
  • FIG. 18 is a schematic diagram of a scene of a search range of a search window in a reconstructed frame image of a current frame image provided by an embodiment of the present application.
  • FIG. 19 is a schematic diagram of a scene encoded by a video encoding apparatus provided by an embodiment of the present application.
  • FIG. 20 is a schematic structural diagram of an apparatus for performing image processing in a video encoding apparatus provided by an embodiment of the present application.
  • FIG. 21 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 22 is another schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 23 is a schematic structural diagram of an image processing system provided by an embodiment of the present application.
  • FIG. 24 is another schematic structural diagram of an image processing system provided by an embodiment of the present application.
  • An embodiment of the present application provides a method for image processing in a video encoding device, wherein the method includes:
  • the to-be-encoded block is encoded according to the relative relationship between the matched block and the to-be-encoded block.
  • the video encoding apparatus may determine the block to be encoded from the current frame image, determine the first area that needs to be read repeatedly from the reconstructed frame image of the current frame image, and convert the first area into
  • the image data is stored in a preset memory, and the power consumption of the preset memory is less than the preset power consumption threshold. Then, the image data of the first area is read from the preset memory, and a matching block matching the block to be encoded is determined from the first area according to the image data of the first area. Then, according to the relative relationship between the matching block and the block to be encoded, the block to be encoded is encoded.
  • the image data of the first region is stored in a preset memory with low power consumption, so as to achieve the purpose of reducing the power consumption of the video encoding apparatus. Therefore, the embodiments of the present application can reduce the power consumption of the video encoding apparatus.
  • the determining of the block to be coded from the current frame image includes:
  • the block to be encoded is determined from the row of blocks to be encoded.
  • the preset memory includes a first memory and a second memory, and the power consumption of the second memory is greater than a first preset multiple of the power consumption of the first memory, so The first area that needs to be read repeatedly for multiple times is determined from the reconstructed frame image of the current frame image, and the image data of the first area is stored in a preset memory, including:
  • the image data of the first area is stored in the first memory and the second memory, respectively.
  • the first area includes a plurality of block rows
  • the storing the image data of the first area in the first memory and the second memory respectively includes:
  • the image data of the moved block line is stored in the first memory and the second memory respectively;
  • Block lines that are not used in encoding the next block line to be encoded are removed from the first memory.
  • the first area includes a plurality of block rows, each of the block rows includes a plurality of blocks, and the image data of the first area is respectively stored in the first area.
  • a memory and a second memory including:
  • the image data of the newly added block is stored in the first memory and the second memory respectively;
  • Blocks that are not used in encoding the next block to be encoded are removed from the first memory.
  • the reading the image data of the first region from the preset memory includes:
  • the image data of the first region is read from the first memory block by line for a first preset number of times, and the image data of unread block lines in the first region is read from the second memory block by line The second preset number of times.
  • the determining a matching block matching the block to be encoded from the first region according to the image data of the first region includes:
  • the image data of the search window is determined from the read image data of the first area.
  • the search window is located in the first area, and the left side of the block to be encoded is the lowermost one of the search window.
  • the right side of the block row is adjacent;
  • the image data of the search window is stored in a third memory, and the read-write speed of the third memory is greater than the second preset multiple of the read-write speed of the first memory;
  • the image data of the search window is read from the third memory, and according to the image data of the search window, the block with the least encoding cost of the block to be encoded is determined from the search window;
  • the block with the smallest coding cost of the block to be coded is used as the matching block.
  • the image data of the search window is read from the third memory, and the image data of the search window is determined from the search window according to the image data of the search window.
  • the block with the least coding cost of the block to be coded including:
  • the reduced block with the smallest encoding cost of the block to be encoded is determined from the reduced search window;
  • a block with the smallest encoding cost to the block to be encoded is determined from the search window.
  • the relative relationship is a block vector and a residual
  • the block vector is the relative displacement between the matching block and the to-be-coded block
  • the block vector is the relative displacement between the matching block and the to-be-coded block.
  • the to-be-encoded block is encoded according to the block vector sum residual of the matched block and the to-be-encoded block.
  • the encoding the block to be encoded according to the block vector and the residual of the matching block and the block to be encoded includes:
  • Entropy encoding is performed on the matching block, the block vector of the block to be encoded, and the forward transformed and quantized first residual data to obtain encoded video stream data;
  • the to-be-coded block is reconstructed according to the second residual data.
  • the first memory includes a system cache or a system buffer memory disposed outside the video encoding device
  • the second memory includes a dynamic random access memory disposed outside the video encoding device
  • the third memory includes a buffer or buffer provided inside the video encoding apparatus.
  • An embodiment of the present application further provides an apparatus for performing image processing in a video encoding apparatus, wherein the apparatus includes:
  • a first determining module configured to determine the block to be encoded from the current frame image
  • the second determination module is configured to determine, from the reconstructed frame image of the current frame image, a first area that needs to be read repeatedly for multiple times, and store the image data of the first area in a preset memory, so that The power consumption of the preset memory is less than the preset power consumption threshold;
  • a reading module for reading the image data of the first area from the preset memory
  • a third determining module configured to determine a matching block matching the block to be encoded from the first region according to the image data of the first region
  • An encoding module configured to encode the to-be-encoded block according to the relative relationship between the matched block and the to-be-encoded block.
  • the first determining module is further configured to:
  • the block to be encoded is determined from the row of blocks to be encoded.
  • the preset memory includes a first memory and a second memory, and the power consumption of the second memory is greater than a first preset multiple of the power consumption of the first memory;
  • the second determining module is used to: determine the first area that needs to be read repeatedly for multiple times from the reconstructed frame image of the current frame image; store the image data of the first area in the first memory respectively and in the second memory;
  • the first area includes a plurality of block rows
  • the second determining module is used for:
  • the image data of the moved block line is stored in the first memory and the second memory respectively; the block lines that are not used when the next block line to be encoded is encoded removing from the first memory;
  • the first area includes a plurality of block lines, each of the block lines includes a plurality of blocks, and the second determining module 402 is configured to:
  • the image data of the newly added block is stored in the first memory and the second memory respectively;
  • the received block is removed from the first memory.
  • the third determining module is configured to: determine image data of a search window from the read image data of the first area, and the search window is located in the first area. In the area, the left side of the block to be coded is adjacent to the right side of the lowermost block row of the search window; the image data of the search window is stored in the third memory, and the read data of the third memory The writing speed is greater than the second preset multiple of the reading and writing speed of the first memory; the image data of the search window is read from the third memory, and according to the image data of the search window, from the search window A block with the smallest coding cost relative to the block to be encoded is determined in the window; the block with the smallest encoding cost relative to the to-be-encoded block is used as the matching block.
  • the third determining module 404 is used for:
  • Embodiments of the present application further provide a computer-readable storage medium on which a computer program is stored, wherein, when the computer program is executed on a computer, the computer is caused to execute the video in any of the above embodiments.
  • a method of image processing in an encoding device is provided.
  • An embodiment of the present application further provides an electronic device, including a memory, a processor, and a video encoding apparatus, wherein the processor executes the video encoding described in any one of the above embodiments by invoking a computer program stored in the memory.
  • an electronic device including a memory, a processor, and a video encoding apparatus, wherein the processor executes the video encoding described in any one of the above embodiments by invoking a computer program stored in the memory.
  • Embodiments of the present application further provide an image processing system, which includes a video encoding device, a first memory, and a second memory, wherein the power consumption of the second memory is greater than a first preset multiple of the power consumption of the first memory , the video encoding device includes a third memory, the read speed of the third memory is greater than the second preset multiple of the read speed of the first memory, the first memory and the second memory respectively store the current frame In the reconstructed frame image of the image, the image data that needs to be read repeatedly for many times, when the video encoding device encodes, from the first memory according to the first preset number of times and from the second memory according to the second preset number of times.
  • the memory reads the image data that needs to be read repeatedly for many times, and determines the image data in the search window from the image data that needs to be read repeatedly for many times, and stores the image data in the search window.
  • the video coding device reads the image data in the search window from the third memory, and determines a matching block that matches the block to be coded, according to the matching block and the block to be coded.
  • the block vector and residual of the block are encoded.
  • the first memory and the second memory respectively store images of the first region that are determined from the reconstructed frame image of the current frame image and need to be read repeatedly for multiple times. If the first area is moved down by one block line, the image data of the moved block line will be stored in the first memory and the second memory respectively, and the next block line to be encoded will not be used when encoding the next block line.
  • the block lines of the first region are removed from the first memory, and the video encoding device reads the image data of the first region from the first memory block line by block line for a first preset number of times when reading, The image data of the unread block lines in the first region is read from the second memory block line by block line a second preset number of times.
  • FIG. 1 is a schematic flowchart of a method for performing image processing in a video encoding apparatus provided by an embodiment of the present application.
  • the method for image processing in a video encoding device can be applied to a video encoding device.
  • the flow of the method for image processing in a video encoding device may include:
  • the video encoding apparatus may encode video images.
  • it is usually necessary to read the data amount of multiple frames of encoded video images.
  • the power consumption of the video encoding device is relatively large.
  • FIG. 2 is a schematic structural diagram of a video compression system in the related art.
  • a central processing unit Central Processing Unit/Processor, CPU
  • a video encoding device a display processor (Display Processing Unit, DISP) and a neural network processor (Neural Network Processing Unit, NPU) are connected through the bus and dynamic Random access memory controller (Dynamic Random Access Memory Controller, DRAMC) reads and writes data from DRAM, central processing unit, video encoding device, display processor and neural network processor time-sharing bandwidth, central processing unit, display processor and
  • DRAMC Dynamic Random Access Memory Controller
  • the priority of the neural network processor is higher than that of the video encoding device.
  • the video encoding apparatus needs to perform a search operation when encoding, which will occupy a large bandwidth.
  • Video encoding devices attach great importance to the level of cost.
  • DRAM is usually used as the main storage space.
  • FIG. 3 is a schematic diagram of data storage in a video encoding apparatus in the related art.
  • the current frame (Current Frame) image, the reference frame (Reference Frame) image, the reconstructed frame (Reconstructed Frame) image, the bit stream (Bitstreams) and the temporary data (Temporary data) are all stored in the DRAM in the video encoding device.
  • the video encoding device has a screen content coding (Screen Content Coding, SCC) function.
  • the reconstructed frame image of the current frame image can be used as the reference frame image of the next frame image. It becomes the reconstructed image of the previous frame image, and the reconstructed image of the previous frame image can be used as the reference frame image of the current frame image.
  • the temporary data may be Temporal Motion Vector (TMV), scaled frames, or other data.
  • TMV Temporal Motion Vector
  • DRAM offers less bandwidth.
  • High Efficiency Video Coding H.265/HEVC
  • Versatile Video Coding H.266/VVC
  • the first generation of video coding standards of the Open Media Alliance Alliance for Open Media Video 1, AV1
  • essential video coding Essential Video Coding, MPEG-5/EVC
  • MPEG-5/EVC essential video coding
  • it is usually used to increase the bandwidth of DRAM or increase the frequency of dynamic random access memory to accelerate the throughput of data.
  • the intra-picture block copy search of hierarchical search alleviates the problem of multiple reconstructed frame image reading, in the case of large size and high frame rate, for example, in 4096 ⁇ 2160 resolution size (ie 4K resolution) or 7680x4320 resolution (ie 8K resolution) size, requires higher throughput of DRAM.
  • the increase in throughput is usually achieved by increasing the number of DRAM channels, which can cause excessive power consumption.
  • FIG. 4 is a schematic diagram of increasing the number of DRAM channels for data access in the related art.
  • the bandwidth and frequency can be increased to increase the data throughput speed of the DRAM, but it will cause greater power consumption.
  • the bandwidth of the system DRAM consumes a large amount of energy. But whether the video encoding device performs immediate or non-real-time operations, it is very important to maintain the highest efficiency.
  • the video encoding device when the video encoding device is completed at an expected time, it will cause a huge power consumption of the DRAM.
  • Video coding apparatuses generally use a block (ie, a square) as a basic unit, and the block may be a rectangle, a square, or a trapezoid, and a triangle is pieced together. Under such conditions, a block-based comparison algorithm appears.
  • FIG. 5 is a schematic diagram of the alignment of square blocks in the related art.
  • the reference frame image is the reconstructed image of the previous frame image, where the block to be compressed and the block of the reference frame image are N ⁇ N N is an integer greater than or equal to 4.
  • Figure 5 is an example of an alignment based on square blocks, but the same alignment method can be used for blocks made up of rectangles, trapezoids or triangles.
  • each block is an N ⁇ N block, for example, it can be 4 ⁇ A block of 4, a block of 32 ⁇ 32, a block of 128 ⁇ 128, etc., where 4 ⁇ 4, 32 ⁇ 32, 128 ⁇ 128 refers to the number of pixels.
  • For each coded block go to the reconstructed frame image of the current frame image and find the block that best matches the block to be coded.
  • the relative displacement of the best matching block relative to the block to be coded can be considered as a block vector .
  • the block to be encoded is determined from the current frame image, where the block to be encoded is a block to be compressed in the current frame image, that is, a block to be encoded in the current frame image.
  • the block to be encoded may be a block of N ⁇ N size.
  • the block can be compared with the block to be encoded.
  • search range search range, SRng
  • the search range search range, SRng
  • the image data of the first area is stored in the preset memory, which is convenient for subsequent encoding.
  • Image data is read.
  • the power consumption of the preset memory is less than the preset power consumption threshold.
  • the image data of the first area is stored in the preset memory
  • the search for the first area is realized.
  • each block in the first area is compared with the block to be encoded in the current frame image. Yes, find the block that best matches the block to be coded from the first area, and the best matching block is the matching block.
  • a common encoding option in screen content encoding is In-Screen Block Copy.
  • This technology uses a common block matching algorithm.
  • the implementation method can be the same as Motion Estimation (ME). It can use a hierarchical search or a non-hierarchical search. After searching A block vector is obtained for subsequent further compression coding.
  • FIG. 6 is a schematic diagram of a hierarchical search provided by an embodiment of the present application.
  • Hierarchical search is to reduce the block to be searched and the area to be searched (such as the first area) by the same ratio, such as 1/2, 1/4 or 1/8, etc. On the reduced image, first determine the After the rough extent of the block, go back to the unreduced image for a finer block search.
  • the reduction ratio of each layer may be the same or different, for example, the reduction ratio of each layer may be 1/2, 1/4, 1/8 and 1/16.
  • Fig. 6 is an example of searching in three levels.
  • the 1/4 reduced image is searched, and then the block vector is obtained from the 1/4 reduced image range, and the finer and smaller range is performed in the 1/2 reduced image range.
  • the block vector obtained from the 1/2 reduced range image is searched again for the range of the original size image to obtain the final block vector.
  • FIG. 7 is a schematic diagram of a non-hierarchical search provided by an embodiment of the present application.
  • Non-hierarchical search refers to performing block comparison tasks directly in unreduced images. Common methods such as full search and n-step search can be seen on public documents.
  • Figure 7 directly performs intra-frame motion search on the original size image, that is, intra-frame motion estimation is performed block by block in the reconstructed frame image of the current frame image, and in the intra-frame search region (search region), use full search The method finds which block has the smallest residual between the current block and the reconstructed frame pixels.
  • a block is a reference block.
  • FIG. 8 is a schematic diagram illustrating a comparison between inter-frame motion estimation and intra-frame block copy search objects provided by an embodiment of the present application.
  • Motion estimation is a mechanism for estimating motion parameters between a coding picture and a reference picture. Motion estimation is generally only performed on the coding side, and the basic unit of estimation is a block. The residual between the obtained block and the block to be coded is estimated from the reference image, and then the conversion coding is performed.
  • the search methods for intra-frame block copying and inter-frame motion estimation can be the same, but the objects are different.
  • the object of the intra-screen block copying search is the reconstructed pixels that have been encoded in the current frame image, and the block vector is obtained after searching.
  • inter-frame motion estimation searches for reconstructed pixels encoded at different previous time points, that is, encoded reconstructed pixels in historical frame images, or pixels in reconstructed frame images of historical frame images.
  • the intra-frame motion vector is referred to as a block vector.
  • the block vector is the object of encoding.
  • the reconstructed frame image of the current frame image can be searched. After the matching block is found, the matching block and the block to be encoded
  • the relative displacement of is the block vector, that is, the relative displacement between the predictor and the current prediction unit in FIG. 8 .
  • the position of the matching block in the reconstructed frame image of the current frame image can be determined according to the block vector.
  • the to-be-coded block can be encoded according to the relative displacement relationship and the relative error relationship between the matching block and the to-be-coded block.
  • the video encoding apparatus may determine the block to be encoded from the current frame image, and determine the first area that needs to be read repeatedly from the reconstructed frame image of the current frame image,
  • the image data of the first area is stored in a preset memory, and the energy consumption of the preset memory is less than the preset energy consumption threshold.
  • the image data of the first area is read from the preset memory, and a matching block matching the block to be encoded is determined from the first area according to the image data of the first area.
  • the block to be encoded is encoded.
  • the image data of the first region is stored in a preset memory with low energy consumption, so as to achieve the purpose of reducing the power consumption of the video encoding apparatus. Therefore, the embodiments of the present application can reduce the power consumption of the video encoding apparatus.
  • FIG. 9 is a schematic flowchart of another method for performing image processing in a video encoding apparatus provided by an embodiment of the present application.
  • the method for image processing in a video encoding device can be applied to a video encoding device.
  • the flow of the method for image processing in a video encoding device may include:
  • each frame of image can be divided into multiple block lines, and each block line can be divided into multiple blocks.
  • the block row to be coded refers to the block row where the block to be coded is located.
  • FIG. 10 is a schematic diagram of a scene of searching in a reconstructed frame image of a current frame image provided by an embodiment of the present application.
  • the block located on the left side of the block to be encoded in the row of blocks to be encoded is an encoded block, and a plurality of block rows located above the row of blocks to be encoded are rows of encoded blocks.
  • each block row includes a plurality of blocks, and the plurality of blocks are arranged in a row.
  • the preset memory includes a first memory and a second memory. After determining the first area that needs to be read repeatedly from the reconstructed frame image of the current frame image, the image data of the determined first area are respectively stored in the first memory and the second memory, wherein the power consumption of the second memory is greater than a first preset multiple of the power consumption of the first memory, and the sum of the power consumption of the first memory and the power consumption of the second memory is less than Preset the power consumption threshold, which can reduce the power consumption when reading and writing data.
  • the preset power consumption threshold may be considered as the power consumption when all the image data in the first area is read and written by the second memory.
  • both the first memory and the second memory are memories outside the video encoding device.
  • the first memory may include a system cache or system buffer memory disposed outside the video encoding device, that is, the first memory may include a system cache or system buffer memory disposed outside the video encoding device.
  • the Sys$ or SysBuf outside the device the second memory may include a dynamic random access memory disposed outside the video encoding device, that is, the second memory may include a DRAM disposed outside the video encoding device.
  • the first memory can also be other low-power consumption memory outside the video encoding device, and the embodiment of the present application is described by taking Sys$ or SysBuf as an example, Sys$ or SysBuf is composed of multiple SRAMs, and the second memory can be DRAM,
  • the power consumption of the DRAM is greater than the first preset multiple of the power consumption of the Sys$ or SysBuf outside the video encoding device, and the sum of the power consumption of the Sys$ or the SysBuf and the power consumption of the DRAM is less than the preset power consumption threshold, which can reduce the read
  • the preset power consumption threshold can be considered as the power consumption when all the image data in the first area is read and written by the DRAM.
  • FIG. 11 is a schematic diagram illustrating a comparison of energy consumption when reading data between a static random access memory and a dynamic random access memory according to an embodiment of the present application.
  • the difference in energy consumption between reading data in SRAM and reading data in DRAM is about 100 times, that is, the power consumption of reading data in SRAM is much smaller than that in reading data in DRAM.
  • the video encoding device uses the method of searching for blocks in the frame, and the step of copying the blocks in the frame requires a larger bandwidth of the DRAM, because some associated areas in the reconstructed frame image of the current frame image will be read during the search process (No. a region) for block search comparison. Due to cost considerations, the block row (ie the first area) covered by the search range is usually not completely stored in the video coding device, and usually only the required size within the search range is stored, such as the size of the search window range. , to meet the high-speed data access requirements during block copying in the screen.
  • the SRAM inside the video encoding device needs to be divided into more Each unit is an area (bank), which will cause the area of a single bank to become larger. As the area of a single bank becomes larger, the area of the SRAM also becomes larger, while the storage capacity of the SRAM remains unchanged, resulting in higher cost.
  • the 8-bit (bit) luminance (luma) part requires at least 512 kilobytes (KB) of storage space, and if the chroma (chroma) part is added, requires more storage space.
  • the SRAM needs to be divided into more units to meet the data entry and exit requirements, resulting in a larger area of the SRAM.
  • the image data of the search window will be stored in the cache or buffer inside the video encoding device.
  • the cache or buffer includes a finely divided SRAM group, and the finely divided means that the area of the same storage unit becomes larger, for example, 1 bit in
  • the average area ratio in the bank is larger than the average area ratio in the SRAM, which can provide enough data bandwidth for the block vector search circuit of the block copy in the screen. This not only results in a large SRAM area, but also makes layout routing more difficult because it is divided into many banks.
  • a block search circuit will be used when performing the block copy search in the frame, which may use a hierarchical search or a non-hierarchical search.
  • the vertical search range will be multiple times the height of the block to be encoded, which results in that the bandwidth for reading the image data in the first area will be multiple times the bandwidth for writing the image data in the first area. And this situation is even more serious when the picture to be encoded reaches 4K or 8K.
  • the resolution of 4K pictures is 3840 ⁇ 2160 pixels, and the resolution of 4K pictures is 7680 ⁇ 4320 pixels.
  • the vertical search range must be larger than the 1080P resolution, otherwise the degree of picture compression will be greatly reduced.
  • the first area includes a plurality of block rows
  • the image data of the first area in 204 is respectively stored in the first memory and the second memory, which may include:
  • the image data of the moved block line is stored in the first memory and the second memory respectively;
  • FIG. 12 is a schematic structural diagram of a video compression system using a system cache provided by an embodiment of the present application.
  • FIG. 13 is another schematic structural diagram of a video compression system using a system cache provided by an embodiment of the present application.
  • FIG. 14 is a schematic structural diagram of a video compression system using a system buffer memory provided by an embodiment of the present application.
  • the image data of the first area is stored in Sys$ or SysBuf
  • the image data of the search window is stored in the cache or buffer inside the video encoding device.
  • n in Fig. 12 to Fig. 14 is a number indicating the size of the storage capacity.
  • the DRAM reads and writes data at a speed of 0.5GB/s to 2GB/s
  • the Sys$ or SysBuf reads and writes data at a speed of 3GB/s to 8GB/s
  • the cache or buffer reads and writes data at a speed of 3GB/s to 8GB/s. 10GB/s ⁇ 50GB/s.
  • the speed of reading and writing data of DRAM, the speed of reading and writing data of Sys$ or SysBuf, and the speed of reading and writing data of cache or buffer can also be other values, but the speed of reading and writing data of cache or buffer must be satisfied.
  • the speed of data is greater than the speed of Sys$ or SysBuf to read and write data and the speed of DRAM to read and write data
  • the speed of Sys$ or SysBuf to read and write data is greater than the speed of DRAM to read and write data.
  • Sys$ can read data from DRAM through DramC, and the data read by Sys$ from DRAM through DramC can be read by central processing unit, video encoding device, display processor and neural network processor.
  • the first area moves down a block line
  • both Sys$ and DRAM store the new block line
  • Sys$ simultaneously removes the block line that is not used in the next block line to be encoded, that is, the first area is the most The image data of the block row above is removed from Sys$.
  • the video encoding device needs to encode, the data in the first area stored in Sys$ can be directly read.
  • Sys$ is also read from DRAM through DramC. The image data of the first area is taken, and then read by the video encoding device.
  • the Sys$ or SysBuf outside the video encoding device will discard the block lines that will not be used in the next line to be encoded, so the cache or buffer inside the video encoding device can read the first area from the DRAM.
  • the number of times of image data is doubled from multiple times, and since DRAM access consumes 100 times more energy than SRAM, it can greatly reduce power consumption.
  • the position where the reconstructed frame image of the current frame image is read ie the first area
  • the behavior (repeated reading) are predictable
  • reading the reconstructed frame image of the current frame image will be the same as reading the current frame image.
  • the structure of the reconstructed frame image of the current frame image compressed by the video coding device it can be determined how many relevant block lines of the reconstructed frame image of the current frame image are to be stored in such a low-power-consumption storage space. Whenever the block line to be encoded moves down one line, remove the top block line stored in the first area in Sys$ or SysBuf, then re-read the new block line in the first area when moving down, and store Added image data for block lines.
  • FIG. 15 is a schematic diagram of a scene when the reconstructed frame image of the current frame image provided by the embodiment of the present application is moved down by one block line.
  • the video encoding device encodes a block line downward, it evicts the unrelated block line originally stored in the upper part of Sys$ or SysBuf, and then sends the image data of the newly needed block line into Sys$ or SysBuf. That is, when the first area covered by the search range can move down with the block to be encoded, the unused block lines are expelled from Sys$ or SysBuf, and the block lines that are to be used are stored in Sys$ or SysBuf.
  • the video encoding device every time the video encoding device encodes a block row, it removes the irrelevant block row above the first area stored in Sys$ or SysBuf, and then stores the latest encoded block row in the Sys$ outside the video encoding device. or SysBuf.
  • the first area includes a plurality of block rows, and each block row includes a plurality of blocks.
  • the image data of the first area is respectively stored in the first memory and the second memory, which may include :
  • the image data of the newly added block is stored in the first memory and the second memory respectively;
  • Blocks that are not used in encoding the next block to be encoded are removed from the first memory.
  • the image data of the encoded block is stored in the Sys$ or SysBuf and DRAM outside the video encoding device.
  • the Sys$ or SysBuf outside the video encoding device will discard the next If the block to be encoded is not used in encoding, the cache or buffer inside the video encoding device can read the image data of the first area from DRAM from multiple times to double. The energy consumed is 100 times higher, so this can drastically reduce power consumption.
  • the video encoding apparatus reads data from Sys$ or SysBuf block by block during encoding, instead of waiting for a block line to be encoded before reading, so this can improve the overall running speed of the video encoding apparatus.
  • the image data of the first region may be read block by line from the first memory, for example, the image data of the first region may be read from Sys$ or SysBuf.
  • the image data of the first region may be read from Sys$ or SysBuf.
  • it is read block by row, that is, read in order from top to bottom.
  • the size of the Coding Tree Unit (CTU) in Figure 15 is 16 ⁇ 16 pixels, that is, 16 pixels in the horizontal direction and 16 pixels in the vertical direction.
  • the size can also be 32 ⁇ 32 pixels, 64 ⁇ 64 pixels, etc.
  • the coding tree unit is the processing unit of H.265.
  • the coding tree unit is the block to be coded in the current frame image. This processing unit is similar to the macroblock in Advanced Video Coding (H.264/AVC).
  • the total number of times of reading from the second memory can be divided into reading the first preset number of times from the first memory, and reading the second preset number of times from the second memory.
  • the number of times is set, and the sum of the first preset number of times and the second preset number of times is the number of times that all the times are originally read from the second memory.
  • the first memory and the second memory are read in no particular order, as long as the number of readings from the first memory reaches the first preset number of times, and the number of readings from the second memory reaches the first number of times. Two preset times can be used.
  • the original number of times of all reading from the second memory is 7 times.
  • the second preset number of times is set to 1 time. If the minimum requirement of Sys$ or SysBuf is met, the 7 pieces of data read from DRAM will be disassembled into 1 read from DRAM and 6 reads from Sys$ or SysBuf.
  • the 7 data volumes read from DRAM can be disassembled into 2 reads from DRAM and 5 reads from Sys$ or SysBuf, etc.
  • the power consumption is demanding conditions In this case, all 7 copies of data read from DRAM can be read from Sys$ or SysBuf. At this time, the power consumption is the lowest, but the cost will increase.
  • the original number of readings from DRAM can be split into several readings from SRAM, and the other several readings from DRAM, which can reduce the power consumption of reading data as a whole. And the number of reads from the SRAM and the number of reads from the DRAM can be adjusted to meet the needs of different power consumption.
  • FIG. 16 is a schematic diagram of a power consumption curve when reading data from a multi-channel DRAM provided by an embodiment of the present application.
  • the abscissa is the position of the reconstructed frame image of the current frame image, for example, the top position of the image, the middle position of the image, and the bottom position of the image
  • the ordinate is the power consumption of reading and writing data during video encoding.
  • the limited power consumption provided by the video compression system can make the video encoding device unable to meet the speed requirements, or cause the video compression System is overheating. If the upper limit of power consumption is considered, the speed of reading data is limited, and the reading speed when the upper limit of power consumption is not considered cannot be reached.
  • FIG. 17 is a schematic diagram of a power consumption curve when reading and writing data from Sys$ or SysBuf and DRAM, respectively, according to an embodiment of the present application.
  • the video encoding device replaces a large amount of DRAM power consumption with the power consumption of Sys$ or SysBuf, which greatly reduces power consumption.
  • the search window is located in the first area, and the left side of the block to be encoded is adjacent to the right side of the lowermost block row of the search window.
  • the search window in the first area is the searched range, and matching blocks can be searched out from the search window.
  • a search window can be determined from the first area, so that the search range can be narrowed, and the power consumption can be further reduced.
  • the intra-screen block copy mode when the non-hierarchical search is used, it is the reconstructed frame image of the non-reduced current frame image, and when the hierarchical search is used, it is the reconstructed frame image of the reduced or non-reduced current frame image, as long as it is predictable
  • the vertical position of the search window can be applied.
  • FIG. 18 is a schematic diagram of a scene of a search range of a search window in a reconstructed frame image of a current frame image provided by an embodiment of the present application. It can be seen from FIG. 18 that the search window is located in the first area, the area between the adjacent dotted lines in the first area is the block row, and the left side of the block to be encoded is the same as the right side of the bottommost block row of the search window Neighbors, the searched block vector can point anywhere within the search window.
  • L, R and H are the left search range, right search range and vertical search range (Vertical SRng) of the search window, respectively, R is a positive number, L and H are negative numbers, and L is not necessarily equal to R.
  • the third memory may be a memory inside the video encoding device, and the third memory may include a memory set inside the video encoding device cache or buffer. Since the blocks within the search window range are searched when performing block copying in the screen, the demand for bandwidth is high when performing block search. Therefore, the read and write speeds of the third memory are higher than the read and write speeds of the first memory and the third memory. Second memory read and write speed. To meet the needs of search speed and high bandwidth. Wherein, the read/write speed of the third memory is greater than the second preset multiple of the read/write speed of the first memory.
  • the image data of the search window is read from the third memory, and the search can be carried out in a hierarchical or non-hierarchical manner.
  • the coding cost may include a residual
  • the coding cost may include a block vector and a residual, and so on. It can be known that the block with the smallest coding cost may be the block with the smallest residual difference with the block to be coded, or the block with the smallest coding cost after comprehensively considering the block vector and the residual difference with the block to be coded.
  • the block vector may be the relative displacement between the searched block and the block to be encoded.
  • the residual may be a difference obtained by subtracting the two-dimensional pixel of the block corresponding to the searched block from the two-dimensional pixel of the block to be encoded.
  • the image data of the search window is read from the third memory, and according to the image data of the search window, the block with the least encoding cost of the block to be encoded is determined from the search window, Can include:
  • the reduced block with the smallest encoding cost of the block to be encoded is determined from the reduced search window;
  • a block with the smallest encoding cost to the block to be encoded is determined from the search window.
  • a hierarchical search method when searching, a hierarchical search method can be used, and the search level is different according to the number of levels. For example, if a search using two layers is used, a search in two layers is performed, and if a search using three layers is used, a search in three layers is performed.
  • an appropriate number of layers can be set according to specific needs. It should be noted that the reduction ratio of each layer may be the same or different.
  • the search window is reduced according to the preset number of layers. For example, the search window is reduced according to the number of two layers to obtain a reduced search window.
  • the search window is 1/2 of the original search window size. Then, according to the image data of the reduced search window, a reduced block with the smallest coding cost of the block to be encoded is determined from the reduced search window, and the reduced block and the reduced search window are reduced.
  • the magnification is the same.
  • the reduced search window On the image of the reduced search window, first determine the approximate range of the reduced block to be searched, and then return to the image of the unreduced search window to perform a finer block search, that is, according to the reduced block, the reduced search
  • the approximate range in the window, the original search window is searched more finely, and the block with the least encoding cost of the block to be encoded can be determined from the unreduced search window.
  • the search window is reduced according to the preset number of layers, for example, the search window is reduced according to the number of three layers to obtain a reduced search window.
  • the search window is 1/4 of the original search window size. Then, according to the image data of the 1/4 reduced range search window, determine the reduced block with the smallest coding cost of the block to be coded from the 1/4 reduced range search window, and obtain the 1/4 reduced range The corresponding block vector under the search window.
  • the search window is reduced according to the preset number of layers, for example, the search window is reduced according to the number of three layers, and a reduced search window is obtained.
  • the search window is 1/6 of the original search window size.
  • the reduced block with the smallest coding cost of the block to be coded is determined from the 1/6 reduced range search window, and the 1/6 reduced range is obtained.
  • the corresponding block vector under the search window is 1/6 of the original search window size.
  • the block with the smallest coding cost with the block to be coded is taken as the matching block.
  • the relative relationship between the matching block and the block to be coded may be a block vector and a residual.
  • the block to be coded can be coded according to the block vector and the residual of the matching block and the block to be coded.
  • encoding the block to be coded according to the block vector and the residual of the matching block and the block to be coded may include:
  • Entropy coding (Entropy Coding, EC) is performed on the matching block and the block vector of the block to be coded and the first residual data after forward transformation and quantization to obtain coded video stream data; or
  • the first residual data after described forward transformation and quantization is carried out inverse quantization and transformation (De-Quantization & Inv.Transform, DQIT) to obtain the second residual data;
  • the to-be-coded block is reconstructed according to the second residual data.
  • FIG. 19 is a schematic diagram of a scene of encoding by a video encoding apparatus provided by an embodiment of the present application. It can be seen from Figure 19 that the intra-picture block copy is located in the data flow relationship between the video encoding device and other modules. The intra-picture block copy will convert the block to be encoded in the current frame image and the reconstructed frame image of the current frame image. The blocks are compared for similarity.
  • in-screen block copying searches the reconstructed frame image of the current frame image, and a matching block is found.
  • the matching block is the same as the current block (ie The relative displacement of the block to be coded) is the block vector, and the residual is obtained according to the error between the current block and the matching block.
  • the residual is forward transformed and quantized.
  • the forward transformation adopts Fast Fourier Transformation (FFT) to obtain the spectrum.
  • FFT Fast Fourier Transformation
  • the Pixels in space are converted into uncorrelated and energy-concentrated spectral coefficients, and the forward-transformed data is only converted to the frequency domain, and the amount of data does not change, which can reduce distortion.
  • Quantization can be achieved by dividing the forward transformed matrix by the value of the corresponding position in the quantization matrix.
  • the spectral coefficients are further compressed by quantization and entropy coding to obtain a compressed video stream.
  • entropy coding is performed, entropy coding is performed on the block vector and the quantized first residual data. Among them, the quantization process removes some unimportant high-frequency information, which can compress the amount of image data, so quantization is the key to compression.
  • the first residual data is obtained after forward transformation and quantization.
  • the first residual data obtained after forward transformation and quantization is subjected to inverse quantization and transformation to the spatial domain, that is, the second residual data of the matching block and the block to be encoded are obtained, and the block to be encoded of the current frame image is passed through the picture block area.
  • Reconstruction Block Reconstruction, BlkRec
  • the In-loop Filter (InF) is used to deal with the continuity problem between blocks and make the image smoother.
  • a commonly used loop filter is a linear low-pass filter that filters out high frequency components and noise.
  • the embodiments of the present application can predict data access behavior (ie, repeated reading behavior) based on screen content encoding, so as to achieve efficient selection of data storage methods and reduce power consumption of the video encoding device.
  • data to be read such as the filtered first area in the reconstructed frame image of the current frame image
  • Sys$ or SysBuf the low-power Sys$ or SysBuf first
  • the reconstructed frame images of some or all of the current frame images of Sys$ or SysBuf have the highest number of repeated readings, so as to minimize power consumption and ensure that the video encoding device can always maintain the lowest power consumption state when entering and exiting data.
  • the Sys$ or SysBuf has high-speed bandwidth at the same time, since the Sys$ or SysBuf can satisfy the bandwidth required for repeatedly reading data, the bandwidth of the DRAM can be further reduced.
  • the embodiments of the present application can ensure that the processing power consumption of the video encoding device is controllable, and the hardware or software of the video encoding device can complete the encoding work as soon as possible, and make full use of the possibility that the video encoding device will repeatedly read the image data of the first area for many times. Desired behavior to change the storage characteristics of the read data allows the video encoding device to maintain operating speed while reducing power consumption because accessing data saves power.
  • the speed at which data can be read is not limited by power consumption, so the video encoding device does not overheat and trigger downclocking.
  • the SRAM in Sys$ or SysBuf has low latency when reading and writing, which can improve the processing frame rate and reduce the response latency. Since the power consumption can be greatly reduced, the usage time of the battery in the video encoding device can be increased, and the user experience can be improved.
  • the target location or attribute of data reading can be selected according to the screen content related to video such as screen sharing and recording requirements, as well as the requirement of low heat dissipation cost and large power consumption caused by predictable behavior.
  • the data that needs to be read repeatedly are read from Sys$ or SysBuf, and DRAM respectively, instead of all read from DRAM, because the same data is read, the power consumption of SRAM is much smaller than that of DRAM , so the embodiment of the present application can greatly reduce the power consumption when reading data.
  • the embodiments of the present application describe in detail how to reduce the power consumption of reading data by taking screen content encoding as an example.
  • all modules and applications requiring high bandwidth but predictable data access behavior such as video decoders, frame rate up conversion devices, etc.
  • the behavior of these modules and applications is usually predictable, such as the number of repeated reads.
  • the corresponding storage characteristics can be pre-allocated, that is, the repeatedly read data is stored in low-power memory, such as
  • the energy consumption corresponding to different levels of storage targets is selected according to the access times of the image data of all or part of the frames. When the energy consumption is different, the number of times of reading data from Sys$ or SysBuf and DRAM can be reasonably allocated.
  • the video decoder can also determine the behavior of accessing data by analyzing the code stream in advance, and the frame rate boosting device can know which areas will be used multiple times during processing through simple analysis, and so on. Therefore the behavior of the video coder, frame rate boosting device is predictable.
  • the embodiments of the present application can also be applied to fixed artificial intelligence (Artificial Intelligence, AI) network behaviors, and the repeatedly read part of the AI network behavior is a feature map (feature map) part, and the AI network behavior is predictable.
  • AI Artificial Intelligence
  • FIG. 20 is a schematic structural diagram of an apparatus for performing image processing in a video encoding apparatus according to an embodiment of the present application.
  • the apparatus 400 for performing image processing in a video encoding apparatus may include: a first determination module 401 , a second determination module 402 , a reading module 403 , a third determination module 404 , and an encoding module 405 .
  • the first determination module 401 is used to determine the block to be encoded from the current frame image
  • the second determining module 402 is configured to determine, from the reconstructed frame image of the current frame image, a first area that needs to be read repeatedly for multiple times, and store the image data of the first area in a preset memory, and the The power consumption of the preset memory is less than the preset power consumption threshold;
  • a reading module 403 configured to read the image data of the first region from the preset memory
  • a third determining module 404 configured to determine a matching block matching the block to be encoded from the first region according to the image data of the first region;
  • the encoding module 405 is configured to encode the block to be encoded according to the relative relationship between the matching block and the block to be encoded.
  • the first determining module 401 may also be used to:
  • the block to be encoded is determined from the row of blocks to be encoded.
  • the preset memory includes a first memory and a second memory
  • the power consumption of the second memory is greater than a first preset multiple of the power consumption of the first memory
  • the second determination Module 402 can be used to:
  • the image data of the first area is stored in the first memory and the second memory, respectively.
  • the first area includes a plurality of block rows
  • the second determination module 402 may be used to:
  • the image data of the moved block line is stored in the first memory and the second memory respectively;
  • Block lines that are not used in encoding the next block line to be encoded are removed from the first memory.
  • the first area includes a plurality of block rows, and each of the block rows includes a plurality of blocks
  • the second determination module 402 may be used to:
  • the image data of the newly added block is stored in the first memory and the second memory respectively;
  • Blocks that are not used in encoding the next block to be encoded are removed from the first memory.
  • the reading module 403 can be used to:
  • the image data of the first region is read from the first memory block by line for a first preset number of times, and the image data of unread block lines in the first region is read from the second memory block by line The second preset number of times.
  • the third determining module 404 may be used to:
  • the image data of the search window is determined from the read image data of the first area.
  • the search window is located in the first area, and the left side of the block to be encoded is the lowermost one of the search window.
  • the right side of the block row is adjacent;
  • the image data of the search window is stored in a third memory, and the read-write speed of the third memory is greater than the second preset multiple of the read-write speed of the first memory;
  • the image data of the search window is read from the third memory, and according to the image data of the search window, the block with the least encoding cost of the block to be encoded is determined from the search window;
  • the block with the smallest coding cost of the block to be coded is used as the matching block.
  • the third determining module 404 may be used to:
  • the reduced block with the smallest encoding cost of the block to be encoded is determined from the reduced search window;
  • a block with the smallest encoding cost to the block to be encoded is determined from the search window.
  • the relative relationship is a block vector and a residual
  • the block vector is the relative displacement between the matching block and the block to be encoded
  • the encoding module 405 may be used to:
  • the to-be-encoded block is encoded according to the block vector sum residual of the matched block and the to-be-encoded block.
  • the encoding module 405 may be used to:
  • Entropy encoding is performed on the matching block, the block vector of the block to be encoded, and the forward transformed and quantized first residual data to obtain encoded video stream data;
  • the to-be-coded block is reconstructed according to the second residual data.
  • the first memory includes a system cache or a system buffer memory provided outside the video encoding device
  • the second memory includes a dynamic random access memory provided outside the video encoding device.
  • the third memory includes a buffer or buffer provided inside the video encoding device.
  • An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed on a computer, the computer is made to execute the image encoding in a video encoding device as provided in this embodiment. Process in the method of processing.
  • An embodiment of the present application further provides an electronic device, including a memory, a processor, and a video encoding apparatus.
  • the processor is configured to execute the video encoding apparatus provided in this embodiment by calling a computer program stored in the memory. The flow in the method of image processing.
  • the above-mentioned electronic device may be a mobile terminal such as a tablet computer or a smart phone.
  • FIG. 21 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • the electronic device 500 may include a video encoding apparatus 501, a memory 502, a processor 503 and other components.
  • a video encoding apparatus 501 may include a video encoding apparatus 501, a memory 502, a processor 503 and other components.
  • FIG. 21 does not constitute a limitation on the electronic device, and may include more or less components than the one shown, or combine some components, or arrange different components.
  • the video encoding device 501 can be used to encode the screen content to compress the screen content of the video image.
  • Memory 502 may be used to store applications and data.
  • the application program stored in the memory 502 contains executable code.
  • Applications can be composed of various functional modules.
  • the processor 503 executes various functional applications and data processing by running the application programs stored in the memory 502 .
  • the processor 503 is the control center of the electronic device, uses various interfaces and lines to connect various parts of the entire electronic device, and executes the electronic device by running or executing the application program stored in the memory 502 and calling the data stored in the memory 502.
  • the various functions and processing data of the device are used to monitor the electronic equipment as a whole.
  • the processor 503 in the electronic device loads the executable code corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 503 executes and stores it in the memory 502 in the application, which executes:
  • the to-be-encoded block is encoded according to the relative relationship between the matched block and the to-be-encoded block.
  • the electronic device 500 may include a video encoding apparatus 501, a memory 502, a processor 503, a battery 504, an input unit 505, an output unit 506 and other components.
  • the video encoding device 501 can be used to encode the screen content to compress the screen content of the video image.
  • Memory 502 may be used to store applications and data.
  • the application program stored in the memory 502 contains executable code.
  • Applications can be composed of various functional modules.
  • the processor 503 executes various functional applications and data processing by running the application programs stored in the memory 502 .
  • the processor 503 is the control center of the electronic device, uses various interfaces and lines to connect various parts of the entire electronic device, and executes the electronic device by running or executing the application program stored in the memory 502 and calling the data stored in the memory 502.
  • the various functions and processing data of the device are used to monitor the electronic equipment as a whole.
  • the battery 504 may be used to provide electrical support for various components of the electronic device, thereby ensuring the normal operation of the various components.
  • the input unit 505 can be used to receive an input video stream of video images, for example, can be used to receive a video stream that needs to be compressed.
  • the output unit 506 may be used to output the compressed video stream.
  • the processor 503 in the electronic device loads the executable code corresponding to the process of one or more application programs into the memory 502 according to the following instructions, and the processor 503 executes and stores it in the memory 502 in the application, which executes:
  • the to-be-encoded block is encoded according to the relative relationship between the matched block and the to-be-encoded block.
  • the processor 503 when the processor 503 performs the determining of the block to be coded from the current frame image, the processor 503 may further perform: determining the row of the block to be coded from the current frame image; The block to be encoded is determined in the block row.
  • the preset memory includes a first memory and a second memory
  • the power consumption of the second memory is greater than a first preset multiple of the power consumption of the first memory
  • the processor 503 When performing the determining from the reconstructed frame image of the current frame image, the first area that needs to be read repeatedly for many times, and storing the image data of the first area in the preset memory, you can also perform: A first area that needs to be read repeatedly is determined from the reconstructed frame image of the current frame image; and the image data of the first area is stored in the first memory and the second memory respectively.
  • the first area includes a plurality of block rows, and when the processor 503 executes the storing of the image data of the first area in the first memory and the second memory, respectively, It can also be performed: if the first area is moved down by one block line, the image data of the moved block line is stored in the first memory and the second memory respectively; when the next block line to be encoded is encoded, it is not used. The resulting block row is removed from the first memory.
  • the first area includes a plurality of block rows, and each of the block rows includes a plurality of blocks
  • the processor 503 executes the step of storing the image data of the first area in the respective blocks.
  • the first memory and the second memory are in the first memory and the second memory, it is also possible to execute: if a new block is added to the lowermost block row of the first area, the image data of the newly added block is stored in the first memory and the second memory, respectively. In the second memory; removing blocks that are not used when the next block to be encoded is encoded from the first memory.
  • the processor 503 when the processor 503 executes the reading of the image data of the first region from the preset memory, the processor 503 may further execute: reading from the first memory block by line The image data of the first area is read for a first preset number of times, and the image data of the unread block line in the first area is read from the second memory block line by block line for a second preset number of times.
  • the processor 503 when the processor 503 executes the determining, according to the image data of the first area, a matching block that matches the block to be encoded from the first area, the processor 503 may further execute:
  • the image data of the search window is determined from the read image data of the first area.
  • the search window is located in the first area, and the left side of the block to be encoded is the lowermost one of the search window.
  • the right side of the block row is adjacent; the image data of the search window is stored in a third memory, and the read and write speed of the third memory is greater than the second preset multiple of the read and write speed of the first memory; from The image data of the search window is read in the third memory, and according to the image data of the search window, the block with the least encoding cost of the block to be encoded is determined from the search window; The block with the least coding cost of the to-be-coded block is used as the matching block.
  • the processor 503 executes the reading of the image data of the search window from the third memory, and according to the image data of the search window, extracts the image data from the search window from the search window.
  • the following steps may be performed: reading the image data of the search window from the third memory; reducing the search window according to a preset number of layers, obtaining a reduced search window; according to the image data of the reduced search window, determine a reduced block with the smallest coding cost of the block to be encoded from the reduced search window; according to the reduced The position of the subsequent block in the reduced search window is determined from the search window to determine the block with the smallest encoding cost compared to the block to be encoded.
  • the relative relationship is a block vector and a residual
  • the block vector is the relative displacement between the matching block and the to-be-coded block
  • the processor 503 executes the method according to the matching block.
  • the relative relationship with the to-be-encoded block, when encoding the to-be-encoded block, may further perform: encoding the to-be-encoded block according to the block vector and the residual between the matching block and the to-be-encoded block .
  • the processor 503 when the processor 503 performs the encoding of the to-be-encoded block according to the block vector sum of the matching block and the to-be-encoded block, the processor 503 may also perform: Perform forward transformation and quantization on the residual of the matching block and the block to be coded; perform entropy coding on the block vector of the matching block and the block to be coded and the first residual data after forward transformation and quantization to obtain video stream encoded data; or perform inverse quantization and transformation on the first residual data after the forward transformation and quantization to obtain second residual data; perform the encoding process on the block to be encoded according to the second residual data Refactor.
  • the first memory includes a system cache or a system buffer memory provided outside the video encoding device
  • the second memory includes a dynamic random access memory provided outside the video encoding device.
  • the third memory includes a buffer or buffer provided inside the video encoding device.
  • FIG. 23 is a schematic structural diagram of the image processing system provided by the embodiment of the present application.
  • FIG. 24 is another schematic structural diagram of an image processing system provided by an embodiment of the present application.
  • the image processing system 600 includes a video encoding apparatus 601, a first memory 602 and a second memory 603, wherein the power consumption of the second memory 603 is greater than a first preset multiple of the power consumption of the first memory 602, and the video encoding apparatus 601 may Including a third memory, the read speed of the third memory is greater than the second preset multiple of the read speed of the first memory, and the first memory 602 and the second memory 603 respectively store the reconstructed frame image of the current frame image.
  • the video encoding device 601 reads the image data that needs to be read repeatedly from the first memory 602 according to the first preset number of times and from the second memory 603 according to the second preset number of times when encoding.
  • the image data in the search window can be determined from the image data that needs to be read repeatedly for many times, and the image data in the search window can be stored in the third memory.
  • the first region that needs to be read repeatedly can be determined from the reconstructed frame image of the current frame image.
  • Image data of an area is stored in the first memory 602 and the second memory 603, respectively. Therefore, the first memory 602 and the second memory 603 respectively store the image data of the first region that is determined from the reconstructed frame image of the current frame image and needs to be read repeatedly.
  • the image data of the moved block line is stored in the first memory 602 and the second memory 603 respectively, and the next block line to be encoded is encoded with the The obtained block row is removed from the first memory 602.
  • the image data of the first region can be read block by row from the first memory 602 for a first preset number of times, and the image data of the first region can be read from the second memory 603.
  • the image data of the unread block lines in the first area is read block by block line for a second preset number of times.
  • the video encoding apparatus 601 may directly read the image data from the second memory 603 , or the first memory 602 may read the image data from the second memory 603 after For storage, this part of the image data is directly read from the first memory 602 by the video encoding device 601 .
  • the video encoding device 601 can read the image data in the search window from the third memory, according to the image data in the search window read from the third memory, from the search window, determine the matching block that matches the module to be encoded, And encoding is performed according to the block vector sum residual of the matching block and the block to be encoded.
  • the apparatus for performing image processing in a video coding apparatus provided by the embodiments of the present application and the method for performing image processing in a video coding apparatus in the above embodiments belong to the same concept. Any of the methods provided in the method embodiments for performing image processing in a video encoding device can be run on the device of a video encoding device. For details of the specific implementation process, please refer to the method embodiments for performing image processing in a video encoding device. Repeat.
  • the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM, Read Only Memory), a random access memory (RAM, Random Access Memory), and the like.
  • each functional module may be integrated in a processing chip, or each module may exist physically alone, or two or more modules may be used. integrated in one module.
  • the above-mentioned integrated modules can be implemented in the form of hardware, and can also be implemented in the form of software function modules. If the integrated module is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium, such as a read-only memory, a magnetic disk or an optical disk, etc. .

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

一种在视频编码装置中进行图像处理的方法、装置、存储介质、电子设备及系统。该方法包括：从当前帧图像中确定出待编码块；从当前帧图像的重构帧图像中确定出第一区域，并将第一区域的图像数据存储在预设存储器中；从预设存储器中读取第一区域的图像数据；根据图像数据从第一区域中确定出与待编码块相匹配的匹配块；根据匹配块与待编码块的相对关系，对待编码块进行编码。

Description

在视频编码装置中进行图像处理的方法、装置、介质及系统
本申请要求于2021年04月01日提交中国专利局、申请号为202110357576.3、发明名称为“在视频编码装置中进行图像处理的方法、装置、介质及系统”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请属于电子设备技术领域，尤其涉及一种在视频编码装置中进行图像处理的方法、装置、存储介质、电子设备及系统。
背景技术
随着技术的不断发展，视频编码装置(Video Encoder，VENC)的功能越来越强大。视频编码装置可以对视频图像进行编码。在对一帧视频图像进行编码时，通常会需要多帧已编码视频图像数据量的读取。
发明内容
本申请实施例提供一种在视频编码装置中进行图像处理的方法、装置、存储介质、电子设备及系统。
第一方面，本申请实施例提供一种在视频编码装置中进行图像处理的方法，所述方法包括：
从当前帧图像中确定出待编码块(encoded block)；
从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
从所述预设存储器中读取所述第一区域的图像数据；
根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
第二方面，本申请实施例提供一种在视频编码装置中进行图像处理的装置，所述装置包括：
第一确定模块，用于从当前帧图像中确定出待编码块；
第二确定模块，用于从当前帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
读取模块，用于从所述预设存储器中读取所述第一区域的图像数据；
第三确定模块，用于根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
编码模块，用于根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
第三方面，本申请实施例提供一种存储介质，其上存储有计算机程序，当所述计算机程序在计算机上执行时，使得所述计算机执行本申请实施例提供的在视频编码装置中进行图像处理的方法。
第四方面，本申请实施例还提供一种电子设备，包括存储器，处理器以及视频编码装置，所述处理器通过调用所述存储器中存储的计算机程序，用于执行本申请实施例提供的在视频编码装置中进行图像处理的方法。
第五方面，本申请实施例还提供一种图像处理系统，包括视频编码装置、第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数，所述视频编码装置包括第三存储器， 所述第三存储器的读取速度大于所述第一存储器的读取速度的第二预设倍数，所述第一存储器和第二存储器分别存储当前帧图像的重构帧图像中需要多次重复读取的图像数据，所述视频编码装置在编码时，按照第一预设次数从所述第一存储器以及按照第二预设次数从所述第二存储器读取所述需要多次重复读取的图像数据，并从所述需要多次重复读取的图像数据中确定出搜索窗(Search Window，SWin)内的图像数据，将所述搜索窗内的图像数据存储在所述第三存储器中，所述视频编码装置从所述第三存储器中读取所述搜索窗内的图像数据，并确定出与待编码块相匹配的匹配块，根据所述匹配块与待编码块的块矢量(Block Vector)和残差进行编码。
附图说明
图1是本申请实施例提供的在视频编码装置中进行图像处理的方法的一种流程示意图。
图2是相关技术中视频压缩系统的结构示意图。
图3是相关技术中视频编码装置中数据存储的示意图。
图4是相关技术中增加动态随机存取内存(Dynamic Random Access Memory,DRAM)的通道(channel)数量进行数据存取的示意图。
图5是相关技术中正方形块的比对示意图。
图6是本申请实施例提供的阶层式搜索的示意图。
图7是本申请实施例提供的非阶层式搜索的示意图。
图8是本申请实施例提供的帧间运动估计(Inter-frame ME)与画面内区块复制(Intra block copy，IntraBC)搜索对象的对比示意图。
图9是本申请实施例提供的在视频编码装置中进行图像处理的方法的另一种流程示意图。
图10是本申请实施例提供的在当前帧图像的重构帧图像中进行搜索的场景示意图。
图11是本申请实施例提供的静态随机存取存储器(Static Random-Access Memory，SRAM)与动态随机存取内存在读取数据时功耗的对比示意图。
图12是本申请实施例提供的使用系统高速缓存(system cache,Sys$)的视频压缩系统的一种架构示意图。
图13是本申请实施例提供的使用系统高速缓存(system cache,Sys$)的视频压缩系统的另一种架构示意图。
图14是本申请实施例提供的使用系统缓冲存储器(System Buffer，SysBuf)的视频压缩系统的架构示意图。
图15是本申请实施例提供的下移一个编码块行时的场景示意图。
图16是本申请实施例提供的从多通道DRAM读取数据时的功耗曲线示意图。
图17是本申请实施例提供的分别从Sys$或SysBuf以及DRAM读取数据时的功耗曲线示意图。
图18是本申请实施例提供的当前帧图像的重构帧图像中搜索窗的搜索范围的场景示意图。
图19是本申请实施例提供的视频编码装置编码的场景示意图。
图20是本申请实施例提供的在视频编码装置中进行图像处理的装置的结构示意图。
图21是本申请实施例提供的电子设备的结构示意图。
图22是本申请实施例提供的电子设备的另一结构示意图。
图23是本申请实施例提供的图像处理系统的结构示意图。
图24是本申请实施例提供的图像处理系统的另一结构示意图。
具体实施方式
本申请实施例提供一种在视频编码装置中进行图像处理的方法，其中，所述方法包括：
从当前帧图像中确定出待编码块；
从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
从所述预设存储器中读取所述第一区域的图像数据；
根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
在本申请实施例中，视频编码装置可以从当前帧图像中确定出待编码块，从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将第一区域的图像数据存储在预设存储器中，该预设存储器的功耗小于预设功耗阈值。然后，从预设存储器中读取第一区域的图像数据，根据第一区域的图像数据从第一区域中确定出与待编码块相匹配的匹配块。之后，根据匹配块与待编码块的相对关系，对待编码块进行编码。即，本申请实施例中，通过将第一区域的图像数据存储在功耗较小的预设存储器中，以达到降低视频编码装置功耗的目的。因此，本申请实施例可以降低视频编码装置的功耗。
本申请一种可选的实施例中，所述从当前帧图像中确定出待编码块，包括：
从所述当前帧图像中确定出待编码块行；
从所述待编码块行中确定出所述待编码块。
本申请一种可选的实施例中，所述预设存储器包括第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数，所述从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，包括：
从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域；
将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中。
本申请一种可选的实施例中，所述第一区域包括多个块行，所述将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中，包括：
若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；
将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除。
本申请一种可选的实施例中，所述第一区域包括多个块行，每个所述块行包括多个块，所述将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中，包括：
若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；
将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
本申请一种可选的实施例中，所述从所述预设存储器中读取所述第一区域的图像数据，包括：
从所述第一存储器中逐块行读取所述第一区域的图像数据第一预设次数，从所述第二存储器中逐块行读取第一区域中未读取块行的图像数据第二预设次数。
本申请一种可选的实施例中，所述根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块，包括：
从读取的所述第一区域的图像数据中确定出搜索窗的图像数据，所述搜索窗位于所述第一区域内，所述待编码块的左侧与所述搜索窗的最下面一个块行的右侧相邻；
将所述搜索窗的图像数据存储在第三存储器中，所述第三存储器的读写速度大于所述第一存储器的读写速度的第二预设倍数；
从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块；
将与所述待编码块的编码代价最小的块作为所述匹配块。
本申请一种可选的实施例中，所述从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块，包括：
从所述第三存储器中读取所述搜索窗的图像数据；
将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；
根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；
根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗中确定出与所述待编码块的编码代价最小的块。
本申请一种可选的实施例中，所述相对关系为块矢量和残差，所述块矢量为所述匹配块与所述待编码块的相对位移，所述根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码，包括：
根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码。
本申请一种可选的实施例中，所述根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码，包括：
将所述匹配块与所述待编码块的残差进行正向变换和量化；
将所述匹配块与所述待编码块的块矢量以及正向变换和量化后的第一残差数据进行熵编码，得到视频流编码数据；或者
将所述正向变换和量化后的第一残差数据进行反向量化与变换，得到第二残差数据；
根据所述第二残差数据对所述待编码块进行重构。
本申请一种可选的实施例中，所述第一存储器包括设置在视频编码装置外部的系统高速缓存或系统缓冲存储器，所述第二存储器包括设置在视频编码装置外部的动态随机存取内存。
本申请一种可选的实施例中，所述第三存储器包括设置在视频编码装置内部的缓存或缓冲。
本申请实施例还提供一种在视频编码装置中进行图像处理的装置，其中，所述装置包括：
第一确定模块，用于从当前帧图像中确定出待编码块；
第二确定模块，用于从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
读取模块，用于从所述预设存储器中读取所述第一区域的图像数据；
第三确定模块，用于根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
编码模块，用于根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
本申请一种可选的实施例中，所述第一确定模块还用于：
从所述当前帧图像中确定出待编码块行；
从所述待编码块行中确定出所述待编码块。
本申请一种可选的实施例中，所述预设存储器包括第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数；所述第二确定模块用于：从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域；将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中；
或者所述第一区域包括多个块行，所述第二确定模块用于：
若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除；
或者所述第一区域包括多个块行，每个所述块行包括多个块，所述第二确定模块402用于：
若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
本申请一种可选的实施例中，所述第三确定模块用于：从读取的所述第一区域的图像数据中确定出搜索窗的图像数据，所述搜索窗位于所述第一区域内，所述待编码块的左侧与所述搜索窗的最下面一个块行的右侧相邻；将所述搜索窗的图像数据存储在第三存储器中，所述第三存储器的读写速度大于所述第一存储器的读写速度的第二预设倍数；从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块；将与所述待编码块的编码代价最小的块作为所述匹配块。
或者所述第三确定模块404用于：
从所述第三存储器中读取所述搜索窗的图像数据；将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗中确定出与所述待编码块的编码代价最小的块。
本申请实施例还提供一种计算机可读的存储介质，其上存储有计算机程序，其中，当所述计算机程序在计算机上执行时，使得所述计算机执行如上任一个实施例所述的在视频编码装置中进行图像处理的方法。
本申请实施例还提供一种电子设备，包括存储器，处理器以及视频编码装置，其中，所述处理器通过调用所述存储器中存储的计算机程序，以执行如上任一个实施例所述的在视频编码装置中进行图像处 理的方法。
本申请实施例还提供一种图像处理系统，其中，包括视频编码装置、第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数，所述视频编码装置包括第三存储器，所述第三存储器的读取速度大于所述第一存储器的读取速度的第二预设倍数，所述第一存储器和第二存储器分别存储当前帧图像的重构帧图像中需要多次重复读取的图像数据，所述视频编码装置在编码时，按照第一预设次数从所述第一存储器以及按照第二预设次数从所述第二存储器读取所述需要多次重复读取的图像数据，并从所述需要多次重复读取的图像数据中确定出搜索窗内的图像数据，将所述搜索窗内的图像数据存储在所述第三存储器中，所述视频编码装置从所述第三存储器中读取所述搜索窗内的图像数据，并确定出与待编码块相匹配的匹配块，根据所述匹配块与待编码块的块矢量和残差进行编码。
本申请一种可选的实施例中，所述第一存储器和第二存储器中分别存储从所述当前帧图像的重构帧图像中确定出的需要多次重复读取的第一区域的图像数据，若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中，并将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除，所述视频编码装置在读取时，从所述第一存储器中逐块行读取所述第一区域的图像数据第一预设次数，从所述第二存储器中逐块行读取所述第一区域中未读取块行的图像数据第二预设次数。
请参照图示，其中相同的组件符号代表相同的组件，本申请的原理是以实施在一适当的运算环境中来举例说明。以下的说明是基于所例示的本申请具体实施例，其不应被视为限制本申请未在此详述的其它具体实施例。
请参阅图1，图1是本申请实施例提供的在视频编码装置中进行图像处理的方法的一种流程示意图。该在视频编码装置中进行图像处理的方法可以应用于视频编码装置中。该在视频编码装置中进行图像处理的方法的流程可以包括：
101、从当前帧图像中确定出待编码块。
随着技术的不断发展，视频编码装置的功能越来越强大。视频编码装置可以对视频图像进行编码。在对一帧视频图像进行编码时，通常会需要多帧已编码视频图像数据量的读取。然而，相关技术中，在对已编码视频图像的数据进行读取时，视频编码装置的功耗较大。
请参阅图2，图2为相关技术中视频压缩系统的结构示意图。该视频压缩系统中，中央处理器(Central Processing Unit/Processor,CPU)、视频编码装置、显示处理器(Display Processing Unit,DISP)和神经网络处理器(Neural Network Processing Unit,NPU)通过总线和动态随机存取内存控制器(Dynamic Random Access Memory Controller,DRAMC)从DRAM读写数据，中央处理器、视频编码装置、显示处理器和神经网络处理器分时共用带宽，中央处理器、显示处理器和神经网络处理器的优先级高于视频编码装置的优先级。视频编码装置在进行编码时需要进行搜索动作，会占用较大的带宽。
视频编码装置非常重视成本的高低，在帧图像缓冲时，为了达到最低成本与最高生产良率，通常都是以DRAM作为主要的存放空间。请参阅图3，图3是相关技术中视频编码装置中数据存储的示意图。其中，当前帧(Current Frame)图像、参考帧(Reference Frame)图像、重构帧(Reconstructed Frame)图像、比特流(Bitstreams)以及临时数据(Temporary data)都存储在视频编码装置中的DRAM中。 该视频编码装置具有屏幕内容编码(Screen Content Coding，SCC)功能。
需要说明的是，对当前帧图像进行编码后变成重构帧图像，该当前帧图像的重构帧图像可以作为下一帧图像的参考帧图像，同理，对前一帧图像进行编码后变成前一帧图像的重构图像，前一帧图像的重构图像可以作为当前帧图像的参考帧图像。临时数据可以是时域运动矢量(Temporal Motion Vector,TMV)、缩放帧(scaled frames)或其它数据。然而，DRAM提供的带宽较小。
随着新型视频标准的出现，如高效率视讯编码(High Efficiency Video Coding,H.265/HEVC)、多功能影像编码(Versatile Video Coding,H.266/VVC)，开放媒体联盟影像编码1代标准(Alliance for Open Media Video 1,AV1)，必要影像编码(Essential Video Coding,MPEG-5/EVC)等，其针对越来越大画面尺寸且越来越高帧率。基于此，通常使用增加DRAM的带宽或提高动态随机存取内存频率的方式以达到加速吞吐数据量。
即使是阶层式搜索(hierarchical search)的画面内区块复制搜索减轻了多倍重构帧图像读取的问题，在大尺寸高帧率的情况下，如，在4096×2160分辨率尺寸(即4K分辨率)或7680x4320分辨率(即8K分辨率)尺寸的情况下，需要DRAM较高的吞吐量。通常会通过增加DRAM的通道数量来实现吞吐量的上升，这样会造成能耗过高的问题。
请参阅图4，图4是相关技术中增加DRAM的通道数量进行数据存取的示意图。通过增加DRAM的通道数量，可以增大带宽，提高频率，以增加DRAM吞吐数据的速度，但会造成较大的功耗。如，为了满足视频编码装置达到读取速度的需求，系统DRAM的带宽消耗较大的能量。但不论视频编码装置是执行即时操作还是非即时操作，维持最高效率是非常重要的。相关技术中的方法，当视频编码装置在预期时间完成的情况下，会造成DRAM极大的功耗。
视频编码装置普遍使用块(即方块)为基本单位，该块可以是长方形，正方形，或梯形，三角形拼凑出来，这样的条件下就出现了以块为单位的比较算法。请参阅图5，图5是相关技术中正方形块的比对示意图。使用正方形块来比对当前帧图像将要压缩的块与参考帧图像的块，该参考帧图像为前一帧图像的重构图像，其中，将要压缩的块与参考帧图像的块为N×N的块，N为大于或等于4的整数，通过块的比对，可以最大化减少时域上的信息冗余，达到压缩数据的效果。图5是以正方形块为基础的比对示例，但长方形、梯形或三角形拼凑出的块也可以使用同样的比对方法。
本申请实施例中，在画面内区块复制时，将图像划分为多个不互相重叠的块，这些块构成矩形阵列，其中每个块是N×N大小的块，比如，可以是4×4的区块，32×32的块，128×128的块等等，其中，4×4、32×32、128×128指的是像素数量。对于每个编码的块，再到当前帧图像的重构帧图像中与待编码块的周围寻找与其最匹配的块，该最匹配的块相对于待编码块的相对位移，可以认为是块矢量。
本申请实施例中，从当前帧图像中确定出待编码块，该待编码块为当前帧图像中将要压缩的块，即当前帧图像中将要编码的块。该待编码块可以是N×N大小的块。在对待编码块进行编码时，通常需要将其与当前帧图像的重构帧图像中的块进行比对，因此需要对当前帧图像的重构帧图像中需要比对的块进行搜索。
102、从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将第一区域的图像数据存储在预设存储器中，预设存储器的功耗小于预设功耗阈值。
比如，从当前帧图像的重构帧图像中搜索需要比对的块后，才能将该块与待编码块进行比对。需要说明的是，当对当前帧图像的重构帧图像中的块进行搜索时，需要知道当前帧图像的重构帧图像中的搜索范围(search range,SRng)，即第一区域，当在该第一区域中搜索块时，需要多次重复读取该第一区域的图像数据。因此，本申请实施例中从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域后，将该第一区域的图像数据存放在预设存储器中，便于后续编码时进行图像数据的读取。另外，该预设存储器的功耗小于预设功耗阈值。通过采用功耗小的预设存储器存取图像数据，可以降低视频编码装置的功耗。
103、从预设存储器中读取第一区域的图像数据。
比如，在将第一区域的图像数据存储在预设存储器后，当对第一区域中的块进行搜索时，需要从预设存储器中读取第一区域的图像数据，以从中找到与待编码块最匹配的块。
104、根据第一区域的图像数据从第一区域中确定出与待编码块相匹配的匹配块。
比如，通过读取预设存储器中存储的第一区域的图像数据，实现对第一区域的搜索，在搜索过程中，将第一区域中每个块分别与当前帧图像中待编码块进行比对，从第一区域中找到与待编码块最匹配的块，该最匹配的块就是匹配块。
随着屏幕内容编码越来越重要，新型视频标准如H.265/HEVC，AV1，通用视频编码(Versatile Video Coding,VVC)等都纳入了屏幕内容编码工具，以实现对屏幕图形常发生的状况进行优化压缩。屏幕内容编码中有一个常见的编码选项是画面内区块复制。该技术以常见的块搜索比对(block matching)算法，实现方式可以跟运动估计(Motion Estimation,ME)一样，可以采用阶层式搜索，也可以是非阶层式搜索(not hierarchical search)，经由搜索以后得到块矢量，以进行后续进一步的压缩编码。
请参阅图6，图6是本申请实施例提供的阶层式搜索的示意图。阶层式搜索就是将要搜索的块与被搜索的区域(如第一区域)都同样缩小同样倍率，例如1/2，1/4或1/8等，在缩小的图像上，先决定将要搜索的块的大致范围后，再回到未缩小的图像进行更精细的块搜索。在阶层式搜索中，每个阶层的缩小倍率可以相同，也可以不同，例如每个阶层的缩小倍率可以是1/2、1/4、1/8和1/16。图6是以3个阶层的搜索作为示例，先搜索1/4缩小的图像，然后，由1/4缩小范围的图像得到块矢量，在1/2缩小的图像范围进行更精细范围更小的搜索，之后，由1/2缩小范围的图像得到的块矢量再搜索原始大小图像的范围，得到最终的块矢量。
请参阅图7，图7是本申请实施例提供的非阶层式搜索的示意图。非阶层式搜索指的是直接在未缩小的图像中进行块比对任务，常见的有全搜索(full search)，n步搜索(n-step search)等方法能在公开的文件上看到。图7是在原始大小的图像上直接进行帧内运动搜索，即在当前帧图像的重构帧图像中逐区块进行帧内运动估计，在帧内搜索区域(search region)中，使用全搜索方式找寻当前块(current block)与重构帧像素(reconstructed frame pixels)中哪一个块有最小的残差。即在对当前帧图像的重构帧图像进行搜索后，对当前帧图像中的当前块进行编码时，将当前帧图像的重构帧图像中与当前块残差最小的块作为匹配块，匹配块即为参考块。
请参阅图8，图8是本申请实施例提供的帧间运动估计与画面内区块复制搜索对象的对比示意图。动作估计是估算编码图像(coding picture)与参考图像(reference picture)间运动参数的一种机制。动作估计 一般只在编码方进行，其估算的基本单位为块，由参考图像估算所得块与待编码块之间的残差，然后再进行转换编码处理。画面内区块复制与帧间运动估计的搜索方法可以相同，但对象不同，画面内区块复制搜索的对象是当前帧图像已经编码完成的重建像素，搜寻完之后得到块矢量。但帧间运动估计搜索的是前面不同时间点编码过后的重建像素，即历史帧图像中编码过后的重建像素，或者历史帧图像的重构帧图像中的像素。
需要说明的是，本申请实施例中，将帧内运动矢量称为块矢量。块矢量是编码的对象，在对屏幕内容进行编码时，在画面内区块复制模式下，可以对当前帧图像的重构帧图像进行搜索，搜索出匹配块后，该匹配块与待编码块的相对位移即为块矢量，即图8中预测变量(predictor)与当前帧预测单元(current prediction unit)的相对位移。
可以理解的是，根据块矢量可以确定出匹配块在当前帧图像的重构帧图像中的位置。
105、根据匹配块与待编码块的相对关系，对待编码块进行编码。
比如，根据匹配块与待编码块之间的相对位移和相对误差关系，如将待编码块的二维像素减去匹配块对应位置的二维像素，得到匹配块与待编码块的相对误差关系，可以根据匹配块与待编码块的相对位移关系和相对误差关系对待编码块进行编码。
可以理解的是，在本申请实施例中，视频编码装置可以从当前帧图像中确定出待编码块，从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将第一区域的图像数据存放在预设存储器中，该预设存储器的能耗小于预设能耗阈值。然后，从预设存储器中读取第一区域的图像数据，根据第一区域的图像数据从第一区域中确定出与待编码块相匹配的匹配块。之后，根据匹配块与待编码块的相对关系，对待编码块进行编码。即，本申请实施例中，通过将第一区域的图像数据存储在能耗较小的预设存储器中，以达到降低视频编码装置功耗的目的。因此，本申请实施例可以降低视频编码装置的功耗。
请参阅图9，图9为本申请实施例提供的在视频编码装置中进行图像处理的方法的另一种流程示意图。该在视频编码装置中进行图像处理的方法可以应用于视频编码装置中。该在视频编码装置中进行图像处理的方法的流程可以包括：
201、从当前帧图像中确定出待编码块行(block line)。
比如，每一帧图像都可以划分为多个块行，每个块行可以划分为多个块。在确定当前帧图像的待编码块之前，需要从当前帧图像中确定出待编码块行。待编码块行指的是待编码块所在的块行。
202、从待编码块行中确定出待编码块。
比如，在确定出待编码块行后，需要从待编码块行中的多个块中确定出待编码块。在该待编码块行中，位于该待编码块之前的块均是已编码块。请参阅图10，图10是本申请实施例提供的在当前帧图像的重构帧图像中进行搜索的场景示意图。图10中，待编码块行中位于待编码块左侧的块为已编码块，位于待编码块行上方的多个块行为已编码块行。
203、从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域。
比如，当确定出待编码块后，需要从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，第一区域可以包括多个块行。其中，每个块行都包括多个块，该多个块排成一行。
204、将第一区域的图像数据分别存储在第一存储器和第二存储器中。
比如，预设存储器包括第一存储器和第二存储器，当从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域后，将确定出的第一区域的图像数据分别存储在第一存储器和第二存储器中，其中，第二存储器的功耗大于第一存储器的功耗的第一预设倍数，且第一存储器的功耗和第二存储器的功耗的总和小于预设功耗阈值，这样可以降低读写数据时的功耗。其中，预设功耗阈值可以认为是将第一区域的图像数据全部由第二存储器进行读写时的功耗。
比如，第一存储器和第二存储器均为视频编码装置外部的存储器，例如，第一存储器可以包括设置在视频编码装置外部的系统高速缓存或系统缓冲存储器，即第一存储器可以包括设置在视频编码装置外部的Sys$或SysBuf，第二存储器可以包括设置在视频编码装置外部的动态随机存取内存，即第二存储器可以包括设置在设置在视频编码装置外部的DRAM。当然第一存储器还可以是视频编码装置外部的其它低功耗存储器等，本申请实施例以Sys$或SysBuf为例进行说明，Sys$或SysBuf由多个SRAM组成，第二存储器可以为DRAM，DRAM的功耗大于视频编码装置外部的Sys$或SysBuf的功耗的第一预设倍数，且Sys$或SysBuf的功耗与DRAM的功耗的总和小于预设功耗阈值，这样可以降低读写数据时的功耗，该预设功耗阈值可以认为是将第一区域的图像数据全部由DRAM进行读写时的功耗。
请参阅图11，图11是本申请实施例提供的静态随机存取存储器与动态随机存取内存在读取数据时能耗的对比示意图。读取SRAM的数据与读取DRAM的数据所消耗的能量相差约为100倍，即读取SRAM中的数据的功耗远远小于读取DRAM中数据的功耗。通过将多个块行的图像数据分别存放在Sys$或SysBuf，以及DRAM，当读取Sys$或SysBuf中的图像数据时，可以降低读取数据时的功耗。
视频编码装置使用画面内区块搜索的方式，画面内区块复制搜索步骤需要DRAM较大的带宽，因为在搜索过程中会读取当前帧图像的重构帧图像中某些关联的区域(第一区域)来做块搜索比对。因成本考虑，通常不会将搜索范围涵盖到的块行(即第一区域)都完整的存于视频编码装置的内部，通常只会储存搜索范围内所需要的大小，如搜索窗范围内的，来满足画面内区块复制时的高速数据存取要求。
若将第一区域的图像数据都存放在视频编码装置的硬件内部，即缓存(cache)或缓冲(buffer)，cache或buffer包括多个SRAM，则需要将视频编码装置内部的SRAM切分成更多的单元，每个单元就是一个区域(bank)，这样会导致单个bank的面积变大。由于单个bank的面积变大，则SRAM的面积也随之变大，而SRAM的存储容量保持不变，这样造成成本较高。比如，以宽度为8192个像素且垂直搜索范围为-64为例，8位(bit)亮度(luma)部分至少需要512千字节(KB)存储空间，若加上色度(chroma)部分，则需要更大的存储空间。另外，由于使用画面内区块复制算法，需要SRAM切分成更多的单元来满足数据进出需求，造成SRAM的面积变大。
需要说明的是，用搜索窗的形式进行画面内区块复制搜索，处理一个块行的压缩，会需要当前帧图像的重构帧图像中多条块行的数据，这意味着处理一帧数据会需要多帧数据量的读取。
比如，视频编码装置在进行块矢量搜索时，请参见图10，通常因搜索窗内的需求带宽很大，在垂直方向的搜索范围越大，占用的带宽越大。而会让搜索窗的图像数据存储于视频编码装置内部的cache或buffer，该cache或buffer包括切分较细的SRAM群，切分的细代表同存储单位的面积变大，例如1个bit在bank中的平均面积占比就比SRAM中的平均面积占比大，这样可以提供足够的数据带宽给画面内区块复制的块矢量搜索电路。这样不仅造成SRAM面积大，且因为切分成较多bank使得版图绕线 较困难，因此不会将整个第一区域的块行都使用这种存储单位(例如1bit)面积高的方法实现。需要说明的是，在进行画面内区块复制搜索时会用到块搜索电路，其可以采用阶层式或非阶层式搜索。
也就是说第一区域涵盖到的块行在每次编码过程中下移一条块行时，第一区域会重新又被抓取一次。通常垂直搜索范围会是待编码块高度的多倍，也就造成了读取第一区域的图像数据的带宽会是多倍于写第一区域的图像数据的带宽。且该情况在要编码的画面到达4K或是8K时更加的严重。4K画面的分辨率为3840×2160像素，4K画面的分辨率为7680×4320像素，4K与8K画面编码时，垂直搜索范围必须要比1080P分辨率大一定程度，否则画面压缩程度会大打折扣。
本申请实施例中，第一区域包括多个块行，204中的将第一区域的图像数据分别存储在第一存储器和第二存储器中，可以包括：
若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；
将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除。比如，若把需要重复读取的当前帧图像的重构帧图像中的块行区域(即第一区域)事先存放在视频编码装置外部的Sys$或SysBuf中，请一并参阅图12至图14，图12是本申请实施例提供的使用系统高速缓存的视频压缩系统的一种架构示意图。图13是本申请实施例提供的使用系统高速缓存的视频压缩系统的另一种架构示意图。图14是本申请实施例提供的使用系统缓冲存储器的视频压缩系统的架构示意图。在Sys$或SysBuf中存储的是第一区域的图像数据，视频编码装置内部的cache或buffer存储的是搜索窗的图像数据。图12至图14中的n为数字，表示存储容量的大小。比如，在一个实施例中，DRAM读写数据的速度为0.5GB/s～2GB/s，Sys$或SysBuf读写数据的速度为3GB/s～8GB/s，cache或buffer读写数据的速度为10GB/s～50GB/s。
需要说明的是，在其他实施例中，DRAM读写数据的速度、Sys$或SysBuf读写数据的速度、cache或buffer读写数据的速度也可以为其他值，但要满足cache或buffer读写数据的速度大于Sys$或SysBuf读写数据的速度以及DRAM读写数据的速度，且Sys$或SysBuf读写数据的速度大于DRAM读写数据的速度。
以图12为例，Sys$可以通过DramC从DRAM中读取数据，且Sys$通过DramC从DRAM读取的数据可以被中央处理器、视频编码装置、显示处理器和神经网络处理器读取。当第一区域下移一个块行时，Sys$和DRAM均存入新的块行，Sys$同时将下一个待编码块行编码时用不到的块行移除出去，即将第一区域最上面一个块行的图像数据从Sys$中移除，当视频编码装置需要进行编码时，可以直接读取Sys$中存储的第一区域的数据，另外，Sys$中还通过DramC从DRAM中读取第一区域的图像数据，之后被视频编码装置读取。
在存储时，视频编码装置外部的Sys$或SysBuf会丢弃下一行待编码块行编码时不会用到的块行，则可以让视频编码装置内部的cache或buffer从DRAM读取第一区域的图像数据的次数从多倍变成1倍，同时由于DRAM比SRAM的存取消耗的能量高100倍，因此这样可以大幅降低功耗。
比如，由于当前帧图像的重构帧图像被读取的位置(即第一区域)与行为(重复读取)是可预测的，且读取当前帧图像的重构帧图像会是读取当前帧图像所需带宽的多倍。若将被读取多次的第一区域的图 像数据存于Sys$或SysBuf等低功耗的存储空间，在有效维持视频编码装置运算的同时，还能大大降低整个系统的功耗，从而可以改善使用者体验。可根据视频编码装置压缩的当前帧图像的重构帧图像的结构，决定要存多少当前帧图像的重构帧图像的相关块行到这类低功耗的存储空间。每当要编码的块行下移一行，就移除存储在Sys$或SysBuf中第一区域最上面的一个块行，然后重新读取第一区域中下移时新增的块行，并存储新增块行的图像数据。
请参阅图15，图15是本申请实施例提供的当前帧图像的重构帧图像下移一个块行时的场景示意图。视频编码装置每往下编码一个块行，就驱逐原本存储在Sys$或SysBuf的上方无关的块行，然后把新需要的块行的图像数据送进Sys$或SysBuf。即当搜索范围涵盖到的第一区域能够跟着待编码块下移的时候，将用不到的块行驱逐出Sys$或SysBuf，且将即将用到的块行存放在Sys$或SysBuf中。即，视频编码装置每往下编码一个块行，就将Sys$或SysBuf中存储的第一区域上方无关的块行移除，然后把最新已编码的块行存放在视频编码装置外部的Sys$或SysBuf中。
在另一种实施方式中，第一区域包括多个块行，每个块行包括多个块，204中的将第一区域的图像数据分别存储在第一存储器和第二存储器中，可以包括：
若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；
将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
在存储时，当编码完一个块，则将该编码完的块的图像数据存储到视频编码装置外部的Sys$或SysBuf以及DRAM中，同时，视频编码装置外部的Sys$或SysBuf会丢弃下一个待编码块编码时用不到的块，则可以让视频编码装置内部的cache或buffer从DRAM读取第一区域的图像数据的次数从多倍变成1倍，同时由于DRAM比SRAM的存取消耗的能量高100倍，因此这样可以大幅降低功耗。且该情况下，视频编码装置在编码时，是逐块从Sys$或SysBuf中读取数据，而不用等待编码完一个块行再进行读取，因此这样可以提高视频编码装置的整体运行速度。
205、从第一存储器中逐块行读取第一区域的图像数据第一预设次数，从第二存储器中逐块行读取第一区域中未读取块行的图像数据第二预设次数。
比如，当视频编码装置需要进行编码时，可以从第一存储器中逐块行读取第一区域的图像数据，如从Sys$或SysBuf读取第一区域的图像数据。在进行读取时，是逐块行进行读取的，即按照从上向下的顺序进行读取。
以H.265/HEVC为例，假设图15中的编码树单元(Coding Tree Unit,CTU)的大小为16×16像素，即横向16个像素乘以纵向16个像素，当然，编码树单元的大小还可以是32×32像素，64×64像素等，编码树单元是H.265的处理单元。该编码树单元为当前帧图像中的待编码块。此处理单元类似进阶视讯编码(Advanced Video Coding,H.264/AVC)中的宏区块(Macroblock)。垂直搜索范围为-96，当前帧图像编码后的重构帧图像被读取的次数是当前帧图像被读取的次数的7倍，即(96+16)/16＝7。
比如，在对第一区域的图像数据进行读取时，可以将原来全部从第二存储器读取的次数分为从第一存储器读取第一预设次数，从第二存储器读取第二预设次数，第一预设次数与第二预设次数的总和为原来全部从第二存储器读取的次数。需要说明的是，第一存储器和第二存储器中在读取时不分先后顺序， 只要从第一存储器中读取的次数达到第一预设次数，从第二存储器中读取的次数达到第二预设次数即可。
比如，原来全部从第二存储器读取的次数为7次，当将第一预设次数设置为6次时，则将第二预设次数设置为1次。如在满足Sys$或SysBuf最低需求量的条件下，将对DRAM读取的7份数据量拆解成1次从DRAM读取与6次从Sys$或SysBuf读取。在Sys$或SysBuf的辅助下读取数据的功耗降低到没有辅助时的14.96％，即(1×640+6×5)/(7×640)＝14.96％，该情况下的功耗较低。比如，还可以根据具体的需求，将对DRAM读取的7份数据量拆解成2次从DRAM读取与5次从Sys$或SysBuf读取，等等，当对功耗要求苛刻的条件下，还可以将对DRAM读取的7份数据量全部从Sys$或SysBuf读取，此时功耗最低，但成本会上升。
由于SRAM成本较高，DRAM成本较低，在考虑成本的情况下，SRAM一般不会做的太大，而DRAM可以做的比较大，因此本申请实施例为了降低读取数据时的功耗，可以将原来从DRAM读取的次数，拆分成几次从SRAM读取，另外几次从DRAM读取，从整体上可以降低读取数据的功耗。而且从SRAM读取的次数与从DRAM读取的次数是可以调整的，以适应对不同功耗的需求。
比如，在读取第一区域的图像数据时，可以先从Sys$或SysBuf中读取，当读取的次数达到第一预设次数时，则切换到从DRAM中读取第一区域中未读取块行的图像数据。再比如，在读取第一区域的图像数据时，可以先从DRAM中读取，当读取的次数达到第二预设次数时，则切换到从Sys$或SysBuf中读取第一区域中未读取块行的图像数据。当读取同样的图像数据时，DRAM消耗的能量大于SRAM消耗的能量的100倍。因此，通过将第一区域中图像数据的一部分从Sys$或SysBuf中读取，另一部分数据从DRAM中读取，可以降低读取数据的功耗。
请参阅图16，图16是本申请实施例提供的从多通道DRAM读取数据时的功耗曲线示意图。图16中，横坐标是当前帧图像的重构帧图像的位置，比如，图像的顶端位置，图像的中间位置，图像的底端位置，纵坐标是视频编码时读写数据的功耗。在视频编码装置过度依赖DRAM或其它便宜但耗电的存储以及高带宽的情况下，因视频压缩系统提供的功耗上限是有限的，会使视频编码装置无法满足速度要求，或者会使视频压缩系统过热。如果考虑功耗上限，则读取数据的速度受限，不能达到未考虑功耗上限时的读取速度。
请参阅图17，图17是本申请实施例提供的分别从Sys$或SysBuf以及DRAM读写数据时的功耗曲线示意图。视频编码装置将大量的DRAM功耗改由Sys$或SysBuf的功耗来取代，大大降低功耗。
206、从读取的第一区域的图像数据中确定出搜索窗的图像数据，搜索窗位于第一区域内，待编码块的左侧与搜索窗的最下面一个块行的右侧相邻。
比如，第一区域中的搜索窗是被搜索的范围，从该搜索窗中可以搜索出匹配块。为了进一步缩小搜索范围，可以从第一区域中确定出搜索窗，这样就可以将搜索范围缩小，从而可以进一步降低功耗。对于画面内区块复制模式，采用非阶层式搜索时为非缩小当前帧图像的重构帧图像，采用阶层式搜索时为缩小或非缩小的当前帧图像的重构帧图像，只要是可以预测垂直方向位置的搜索窗都可以适用。
请参阅图18，图18是本申请实施例提供的当前帧图像的重构帧图像中搜索窗的搜索范围的场景示意图。从图18中可以看出搜索窗位于第一区域内，第一区域中相邻虚线之间的区域即为块行，待编码块的左侧与搜索窗的最下面一个块行的右侧相邻，搜索的块矢量可以指向搜索窗内的任何地方。其中， L、R和H分别为搜索窗的左侧搜索范围、右侧搜索范围和垂直搜索范围(Vertical SRng)，R为正数，L和H为负数，且L不一定等于R。
207、将搜索窗的图像数据存储在第三存储器中，第三存储器的读写速度大于第一存储器的读写速度的第二预设倍数。
比如，当确定出搜索窗的图像数据后，将搜索窗的图像数据存储在第三存储器中，该第三存储器可以是视频编码装置内部的存储器，第三存储器可以包括设置在视频编码装置内部的缓存或缓冲。由于进行画面内区块复制时，是对搜索窗范围内的块进行搜索，进行块搜索时对带宽的需求较高，因此第三存储器的读写速度均大于第一存储器的读写速度以及第二存储器的读写速度。以满足搜索速度和高带宽的需求。其中，第三存储器的读写速度大于第一存储器的读写速度的第二预设倍数。
208、从第三存储器中读取搜索窗的图像数据，并根据搜索窗的图像数据，从搜索窗中确定出与待编码块的编码代价最小的块。
比如，在进行搜索时，从第三存储器中读取搜索窗的图像数据，可以采用阶层式或非阶层式方式进行搜索，根据搜索窗的图像数据，将读取的搜索窗中的块与待编码块进行比对，可以确定出读取的搜索窗中的块与待编码块的编码代价最小的块。比如，在一种实施方式中，编码代价可以包括残差，比如，在另外一种实施方式中，编码代价可以包括块矢量和残差，等等。可知，编码代价最小的块可以是与待编码块的残差最小的块，还可以是综合考虑与待编码块的块矢量和残差后编码代价最小的块。
例如，对于画面内区块复制，在搜索窗中逐块行扫描，将搜索的块与待编码块进行比对，从而可以从搜索窗中找到与待编码块的残差最小的块。其中，块矢量可以是搜索的块与待编码块的相对位移。残差可以是待编码块的二维像素减去搜索的块对应位置的二维像素后得到的差值。
比如，在一种实施方式中，208中的从第三存储器中读取搜索窗的图像数据，并根据搜索窗的图像数据，从搜索窗中确定出与待编码块的编码代价最小的块，可以包括：
从所述第三存储器中读取所述搜索窗的图像数据；
将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；
根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；
根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗中确定出与所述待编码块的编码代价最小的块。
比如，在进行搜索时，可以采用阶层式搜索方式，根据阶层数的不同，搜索的层级也不同。比如，若采用2个阶层的搜索，则进行2个层级的搜索，若采用采用3个阶层的搜索，则进行3个层级的搜索。当然，阶层数越多，则搜索的结果越准确，但同时也会增加系统计算资源的消耗。在实际应用中，可以根据具体需求设置合适的阶层数。需要说明的是，每个阶层的缩小倍率可以相同，也可以不同。
例如，从第三存储器中读取搜索窗的图像数据后，将搜索窗按照预设阶层数进行缩小，如按照2个阶层数将搜索窗进行缩小，得到缩小后的搜索窗，该缩小后的搜索窗为原来搜索窗大小的1/2。然后，根据该缩小后的搜索窗的图像数据，从该缩小后的搜索窗中确定出与待编码块的编码代价最小的缩小后的块，该缩小后的块与缩小后的搜索窗的缩小倍率是相同的。在缩小的搜索窗的图像上，先决定将要搜 索的缩小的块的大致范围后，再回到未缩小的搜索窗的图像进行更精细的块搜索，即根据该缩小的块在缩小后的搜索窗中的大致范围，对原始搜索窗进行更精细的搜索，可以从未缩小的搜索窗中确定出与待编码块的编码代价最小的块。
再例如，从第三存储器中读取搜索窗的图像数据后，将搜索窗按照预设阶层数进行缩小，如按照3个阶层数将搜索窗进行缩小，得到缩小后的搜索窗，该缩小后的搜索窗为原来搜索窗大小的1/4。然后，根据该1/4缩小范围的搜索窗的图像数据，从该1/4缩小范围的搜索窗中确定出与待编码块的编码代价最小的缩小后的块，得到该1/4缩小范围的搜索窗下对应的块矢量。之后，在1/2缩小范围的搜索窗进行更精细范围更小的搜索，最后，根据1/2缩小范围的搜索窗得到的块矢量再搜索原始大小的搜索窗的范围，得到最终的块矢量，从而可以确定出与待编码块的编码代价最小的块。
又如，从第三存储器中读取搜索窗的图像数据后，将搜索窗按照预设阶层数进行缩小，如按照3个阶层数将搜索窗进行缩小，得到缩小后的搜索窗，该缩小后的搜索窗为原来搜索窗大小的1/6。然后，根据该1/6缩小范围的搜索窗的图像数据，从该1/6缩小范围的搜索窗中确定出与待编码块的编码代价最小的缩小后的块，得到该1/6缩小范围的搜索窗下对应的块矢量。之后，在1/3缩小范围的搜索窗进行更精细范围更小的搜索，最后，根据1/3缩小范围的搜索窗得到的块矢量再搜索原始大小的搜索窗的范围，得到最终的块矢量，从而可以确定出与待编码块的编码代价最小的块。
由此可知，在缩小的搜索窗的图像上，先决定将要搜索的缩小的块的大致范围后，再回到未缩小的搜索窗的图像进行更精细的块搜索，即根据该缩小的块在缩小后的搜索窗中的大致范围，对原始搜索窗进行更精细的搜索，可以从未缩小的搜索窗中确定出与待编码块的编码代价最小的块。
209、将与待编码块的编码代价最小的块作为匹配块。
比如，当从搜索窗中找到与待编码块的编码代价最小的块后，将该与待编码块之间编码代价最小的块作为匹配块。
210、根据匹配块与待编码块的块矢量和残差，对待编码块进行编码。
比如，匹配块与待编码块的相对关系可以是块矢量和残差，在找到匹配块后，可以根据匹配块与待编码块的块矢量和残差，对待编码块进行编码。
在一种实施方式中，210中的根据匹配块与待编码块的块矢量和残差，对待编码块进行编码，可以包括：
将所述匹配块与所述待编码块的残差进行正向变换和量化(Forward Transform&Quantization,FTQ)；
将所述匹配块与所述待编码块的块矢量以及正向变换和量化后的第一残差数据进行熵编码(Entropy Coding,EC)得到视频流编码数据；或者
将所述正向变换和量化后的第一残差数据进行反向量化与变换(De-Quantization&Inv.Transform,DQIT)，得到第二残差数据；
根据所述第二残差数据对所述待编码块进行重构。
请参阅图19，图19是本申请实施例提供的视频编码装置编码的场景示意图。从图19中可以看出画面内区块复制位于视频编码装置中与其它模块之间的数据流关系，画面内区块复制会将当前帧图像中待编码块与当前帧图像的重构帧图像的块做相似度比对。
比如，请参阅图19，画面内区块复制(可以采用阶层式搜索或非阶层式搜索)对当前帧图像的重构帧图像进行搜索，会搜到到匹配块，匹配块与当前块(即待编码块)的相对位移即是块矢量，根据当前块与匹配块的误差得到残差。将残差进行正向变换与量化，其中，正向变换采用快速傅氏变换(Fast Fourier Transformation,FFT)变换得到频谱，频谱曲线上横坐标为频率，纵坐标为能量，经过正向变换，将空间中的像素转换成不相关而且能量集中的频谱系数，正向变换后的数据只是转换到频域，数据量并没有变化，其可以减少失真。正向变换后的矩阵除以量化矩阵中对应位置的值，即可实现量化。频谱系数再用量化与熵编码进一步压缩，得到压缩的视频流。在进行熵编码时，是对块矢量和量化后的第一残差数据进行熵编码。其中，量化过程去掉了一些不重要的高频信息，这样可以压缩图像数据量，所以量化才是压缩的关键。经过正向变换和量化后得到第一残差数据。
将经过正向变换和量化后得到第一残差数据经过反向量化与变换到空域，即得到匹配块与待编码块的第二残差数据，将当前帧图像的待编码块经过画面块区域重构(Block Reconstruction，BlkRec)，作为下一个待编码块的邻居。环路内滤波器(In-loop Filter，InF)用于处理块之间的连续性问题，使图像更加平滑。常用的环路滤波器是一个线性低通滤波器，可以滤除高频分量和噪声。用正向变换与量化可以消除视频图像空间上的冗余，用熵编码可以消除编码冗余。
可以理解的是，本申请实施例基于屏幕内容编码时可预测数据存取行为(即多次重复读取的行为)，从而实现有效率的选择数据存储方式，可以降低视频编码装置的功耗。可以根据编码时帧参考读取策略改变将要读取的数据(如当前帧图像的重构帧图像中已滤波的第一区域)是否该先存储到低功耗的Sys$或SysBuf，使存入Sys$或SysBuf的部分或全部当前帧图像的重构帧图像重复读取的次数最高，以最大程度降低功耗，保证视频编码装置进出数据时能一直维持在最低功耗状态。若该Sys$或SysBuf同时具有高速带宽，由于该Sys$或SysBuf可以满足重复读取数据时所需带宽，这样可以进一步降低DRAM的带宽。
本申请实施例可以保证视频编码装置处理功耗可控，且能让视频编码装置的硬件或软件尽快完成编码工作，充分利用视频编码装置会有多次重复读取第一区域的图像数据的可预期行为来改变所读取数据的存储特性，因为存取数据省电，而使视频编码装置可以维持运行速度，同时又降低功耗。读取数据的速度不会受功耗的限制，因此视频编码装置不会过热，不会触发降频。另外，Sys$或SysBuf中SRAM在读写时本身的时延就低，这样可以提高处理帧率，降低反应时延。由于可以大幅降低功耗，则可以提高视频编码装置中电池的使用时间，提升用户体验。
可以理解的是，本申请实施例可以根据屏幕分享与录制需求等视频相关的屏幕内容，以及低散热成本需求和可预测行为造成的较大功耗，可以选择数据读取的目标位置或属性。比如，将需要重复读取的数据分别从Sys$或SysBuf，以及DRAM进行读取，而不是全部都是从DRAM读取，由于读取相同的数据，SRAM的功耗远远小于DRAM的功耗，因此本申请实施例可以大大降低读取数据时的功耗。
本申请实施例以屏幕内容编码为示例详细说明了如何降低读取数据的功耗。在其它实施方式中，还可以适用所有需要高带宽但存取数据行为可预测的模块与应用，如视频译码器，帧频提升(frame rate up conversion)装置等。这些模块与应用的行为通常是可以预测的，如重复读取的次数，通过这些可以预测的行为，可以预先分配相应的存储特性，即将重复读取的数据存放在低功耗的存储器中，例如根据全 部帧或部分帧的图像数据的存取次数需求，来对应不同等级存储目标的能量消耗，即根据全部帧或部分帧图像数据的存取次数需求，来选择对应不同等级的能量消耗，当能量消耗不同时，就可以合理分配从Sys$或SysBuf以及DRAM读取数据的次数。如，视频译码器事先解析码流也可以确定存取数据的行为，帧频提升装置可以通过简单分析得知哪些区域在处理时会被用到多次，等等。因此视频译码器、帧频提升装置的行为是可预期的。
本申请实施例还可以适用于固定的人工智能(Artificial Intelligence，AI)网络行为，AI网络行为重复读取的部分是特征图(feature map)部分，该AI网络行是可预期的。
请参阅图20，图20为本申请实施例提供的在视频编码装置中进行图像处理的装置的结构示意图。该在视频编码装置中进行图像处理的装置400可以包括：第一确定模块401，第二确定模块402，读取模块403，第三确定模块404，编码模块405。
第一确定模块401，用于从当前帧图像中确定出待编码块；
第二确定模块402，用于从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存放在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
读取模块403，用于从所述预设存储器中读取所述第一区域的图像数据；
第三确定模块404，用于根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
编码模块405，用于根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
在一种实施方式中，所述第一确定模块401还可以用于：
从所述当前帧图像中确定出待编码块行；
从所述待编码块行中确定出所述待编码块。
在一种实施方式中，所述预设存储器包括第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数，所述第二确定模块402可以用于：
从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域；
将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中。
在一种实施方式中，所述第一区域包括多个块行，所述第二确定模块402可以用于：
若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；
将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除。
在一种实施方式中，所述第一区域包括多个块行，每个所述块行包括多个块，所述第二确定模块402可以用于：
若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；
将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
在一种实施方式中，所述读取模块403可以用于：
从所述第一存储器中逐块行读取所述第一区域的图像数据第一预设次数，从所述第二存储器中逐块 行读取第一区域中未读取块行的图像数据第二预设次数。
在一种实施方式中，所述第三确定模块404可以用于：
从读取的所述第一区域的图像数据中确定出搜索窗的图像数据，所述搜索窗位于所述第一区域内，所述待编码块的左侧与所述搜索窗的最下面一个块行的右侧相邻；
将所述搜索窗的图像数据存储在第三存储器中，所述第三存储器的读写速度大于所述第一存储器的读写速度的第二预设倍数；
从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块；
将与所述待编码块的编码代价最小的块作为所述匹配块。
在一种实施方式中，所述第三确定模块404可以用于：
从所述第三存储器中读取所述搜索窗的图像数据；
将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；
根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；
根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗中确定出与所述待编码块的编码代价最小的块。
在一种实施方式中，所述相对关系为块矢量和残差，所述块矢量为所述匹配块与所述待编码块的相对位移，所述编码模块405可以用于：
根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码。
在一种实施方式中，所述编码模块405可以用于：
将所述匹配块与所述待编码块之间的残差进行正向变换和量化；
将所述匹配块与所述待编码块的块矢量以及正向变换和量化后的第一残差数据进行熵编码，得到视频流编码数据；或者
将所述正向变换和量化后的第一残差数据进行反向量化与变换，得到第二残差数据；
根据所述第二残差数据对所述待编码块进行重构。
在一种实施方式中，所述第一存储器包括设置在视频编码装置外部的系统高速缓存或系统缓冲存储器，所述第二存储器包括设置在视频编码装置外部的动态随机存取内存。
在一种实施方式中，所述第三存储器包括设置在视频编码装置内部的缓存或缓冲。
本申请实施例提供一种计算机可读的存储介质，其上存储有计算机程序，当所述计算机程序在计算机上执行时，使得所述计算机执行如本实施例提供的在视频编码装置中进行图像处理的方法中的流程。
本申请实施例还提供一种电子设备，包括存储器，处理器以及视频编码装置，所述处理器通过调用所述存储器中存储的计算机程序，用于执行本实施例提供的在视频编码装置中进行图像处理的方法中的流程。
例如，上述电子设备可以是诸如平板电脑或者智能手机等移动终端。请参阅图21，图21为本申请实施例提供的电子设备的结构示意图。
该电子设备500可以包括视频编码装置501、存储器502、处理器503等部件。本领域技术人员可以理解，图21中示出的电子设备结构并不构成对电子设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
视频编码装置501可以用于对屏幕内容进行编码，以对视频图像的屏幕内容进行压缩。
存储器502可用于存储应用程序和数据。存储器502存储的应用程序中包含有可执行代码。应用程序可以组成各种功能模块。处理器503通过运行存储在存储器502的应用程序，从而执行各种功能应用以及数据处理。
处理器503是电子设备的控制中心，利用各种接口和线路连接整个电子设备的各个部分，通过运行或执行存储在存储器502内的应用程序，以及调用存储在存储器502内的数据，执行电子设备的各种功能和处理数据，从而对电子设备进行整体监控。
在本实施例中，电子设备中的处理器503会按照如下的指令，将一个或一个以上的应用程序的进程对应的可执行代码加载到存储器502中，并由处理器503来运行存储在存储器502中的应用程序，从而执行：
从当前帧图像中确定出待编码块；
从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存放在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
从所述预设存储器中读取所述第一区域的图像数据；
根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
请参阅图22，电子设备500可以包括视频编码装置501、存储器502、处理器503、电池504、输入单元505、输出单元506等部件。
视频编码装置501可以用于对屏幕内容进行编码，以对视频图像的屏幕内容进行压缩。
存储器502可用于存储应用程序和数据。存储器502存储的应用程序中包含有可执行代码。应用程序可以组成各种功能模块。处理器503通过运行存储在存储器502的应用程序，从而执行各种功能应用以及数据处理。
处理器503是电子设备的控制中心，利用各种接口和线路连接整个电子设备的各个部分，通过运行或执行存储在存储器502内的应用程序，以及调用存储在存储器502内的数据，执行电子设备的各种功能和处理数据，从而对电子设备进行整体监控。
电池504可用于为电子设备的各个部件提供电力支持，从而保障各个部件的正常运行。
输入单元505可用于接收视频图像的输入视频流，例如可以用于接收需要进行视频压缩的视频流。
输出单元506可以用于用于输出已压缩的视频流。
在本实施例中，电子设备中的处理器503会按照如下的指令，将一个或一个以上的应用程序的进程对应的可执行代码加载到存储器502中，并由处理器503来运行存储在存储器502中的应用程序，从而执行：
从当前帧图像中确定出待编码块；
从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存放在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
从所述预设存储器中读取所述第一区域的图像数据；
根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
在一种实施方式中，所述处理器503执行所述从当前帧图像中确定出待编码块时，还可以执行：从所述当前帧图像中确定出待编码块行；从所述待编码块行中确定出所述待编码块。
在一种实施方式中，所述预设存储器包括第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数，所述处理器503执行所述从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中时，还可以执行：从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域；将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中。
在一种实施方式中，所述第一区域包括多个块行，所述处理器503执行所述将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中时，还可以执行：若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除。
在一种实施方式中，所述第一区域包括多个块行，每个所述块行包括多个块，所述处理器503执行所述将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中时，还可以执行：若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
在一种实施方式中，所述处理器503执行所述从所述预设存储器中读取所述第一区域的图像数据时，还可以执行：从所述第一存储器中逐块行读取所述第一区域的图像数据第一预设次数，从所述第二存储器中逐块行读取第一区域中未读取块行的图像数据第二预设次数。
在一种实施方式中，所述处理器503执行所述根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块时，还可以执行：从读取的所述第一区域的图像数据中确定出搜索窗的图像数据，所述搜索窗位于所述第一区域内，所述待编码块的左侧与所述搜索窗的最下面一个块行的右侧相邻；将所述搜索窗的图像数据存储在第三存储器中，所述第三存储器的读写速度大于所述第一存储器的读写速度的第二预设倍数；从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块；将与所述待编码块的编码代价最小的块作为所述匹配块。
在一种实施方式中，所述所述处理器503执行所述从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块时，还可以执行：从所述第三存储器中读取所述搜索窗的图像数据；将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗 中确定出与所述待编码块的编码代价最小的块。
在一种实施方式中，所述相对关系为块矢量和残差，所述块矢量为所述匹配块与所述待编码块的相对位移，所述处理器503执行所述根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码时，还可以执行：根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码。
在一种实施方式中，所述处理器503执行所述根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码时，还可以执行：将所述匹配块与所述待编码块的残差进行正向变换和量化；将所述匹配块与所述待编码块的块矢量以及正向变换和量化后的第一残差数据进行熵编码，得到视频流编码数据；或者将所述正向变换和量化后的第一残差数据进行反向量化与变换，得到第二残差数据；根据所述第二残差数据对所述待编码块进行重构。
在一种实施方式中，所述第一存储器包括设置在视频编码装置外部的系统高速缓存或系统缓冲存储器，所述第二存储器包括设置在视频编码装置外部的动态随机存取内存。
在一种实施方式中，所述第三存储器包括设置在视频编码装置内部的缓存或缓冲。
本申请实施例还提供一种图像处理系统，请参阅图23和图24，图23是本申请实施例提供的图像处理系统的结构示意图。图24是本申请实施例提供的图像处理系统的另一结构示意图。该图像处理系统600包括视频编码装置601、第一存储器602和第二存储器603，其中，第二存储器603的功耗大于第一存储器602的功耗的第一预设倍数，视频编码装置601可以包括第三存储器，第三存储器的读取速度大于第一存储器的读取速度的第二预设倍数，第一存储器602和第二存储器603分别存储当前帧图像的重构帧图像中需要多次重复读取的图像数据，视频编码装置601在进行编码时，按照第一预设次数从第一存储器602以及按照第二预设次数从第二存储器603读取需要多次重复读取的图像数据，即在读取需要多次重复读取的图像数据时，可以分成从第一存储器602中读取第一预设次数，从第二存储器603中读取第二预设次数。然后，可以从需要多次重复读取的图像数据中确定出搜索窗内的图像数据，将搜索窗内的图像数据存储在第三存储器中。
比如，在从当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域后，可以将从当前帧图像的重构帧图像中确定出的需要多次重复读取的第一区域的图像数据分别存储在第一存储器602和第二存储器603。因此，第一存储器602和第二存储器603中分别存储从当前帧图像的重构帧图像中确定出的需要多次重复读取的第一区域的图像数据。
在编码过程中，若第一区域下移一个块行，则将下移块行的图像数据分别存储在第一存储器602和第二存储器603中，并将下一个待编码块行编码时用不到的块行从第一存储器602中进行移除，在读取数据时，可以从第一存储器602中逐块行读取第一区域的图像数据第一预设次数，从第二存储器603中逐块行读取第一区域中未读取块行的图像数据第二预设次数。
需要说明的是，当从第二存储器603读取图像数据时，视频编码装置601可以直接从第二存储器603读取图像数据，或者是由第一存储器602从第二存储器603读取图像数据后进行存储，该部分图像数据由视频编码装置601直接从第一存储器602中读取。
视频编码装置601可以从第三存储器读取搜索窗内的图像数据，根据从第三存储器中读取的搜索窗内的图像数据，从搜索窗中确定出与待编码模块相匹配的匹配块，并根据匹配块与待编码块的块矢量和 残差进行编码。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述的部分，可以参见上文针对在视频编码装置中进行图像处理的方法的详细描述，此处不再赘述。
本申请实施例提供的所述在视频编码装置中进行图像处理的装置与上文实施例中的在视频编码装置中进行图像处理的方法属于同一构思，在所述在视频编码装置中进行图像处理的装置上可以运行所述在视频编码装置中进行图像处理的方法实施例中提供的任一方法，其具体实现过程详见所述在视频编码装置中进行图像处理的方法实施例，此处不再赘述。
需要说明的是，对本申请实施例所述在视频编码装置中进行图像处理的方法而言，本领域普通技术人员可以理解实现本申请实施例所述在视频编码装置中进行图像处理的方法的全部或部分流程，是可以通过计算机程序来控制相关的硬件来完成，所述计算机程序可存储于一计算机可读取存储介质中，如存储在存储器中，并被至少一个处理器执行，在执行过程中可包括如所述在视频编码装置中进行图像处理的方法的实施例的流程。其中，所述的存储介质可为磁碟、光盘、只读存储器(ROM，Read Only Memory)、随机存取记忆体(RAM，Random Access Memory)等。
对本申请实施例的所述在视频编码装置中进行图像处理的装置而言，其各功能模块可以集成在一个处理芯片中，也可以是各个模块单独物理存在，也可以两个或两个以上模块集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销售或使用时，也可以存储在一个计算机可读取存储介质中，所述存储介质譬如为只读存储器，磁盘或光盘等。
以上对本申请实施例所提供的一种在视频编码装置中进行图像处理的方法、装置、存储介质、电子设备及系统进行了详细介绍，本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (20)

  1. 一种在视频编码装置中进行图像处理的方法，其中，所述方法包括：
    从当前帧图像中确定出待编码块；
    从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
    从所述预设存储器中读取所述第一区域的图像数据；
    根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
    根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
  2. 根据权利要求1所述的在视频编码装置中进行图像处理的方法，其中，所述从当前帧图像中确定出待编码块，包括：
    从所述当前帧图像中确定出待编码块行；
    从所述待编码块行中确定出所述待编码块。
  3. 根据权利要求2所述的在视频编码装置中进行图像处理的方法，其中，所述预设存储器包括第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数，所述从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将所述第一区域的图像数据存储在预设存储器中，包括：
    从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域；
    将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中。
  4. 根据权利要求3所述的在视频编码装置中进行图像处理的方法，其中，所述第一区域包括多个块行，所述将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中，包括：
    若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；
    将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除。
  5. 根据权利要求3所述的在视频编码装置中进行图像处理的方法，其中，所述第一区域包括多个块行，每个所述块行包括多个块，所述将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中，包括：
    若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；
    将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
  6. 根据权利要求4或5所述的在视频编码装置中进行图像处理的方法，其中，所述从所述预设存储器中读取所述第一区域的图像数据，包括：
    从所述第一存储器中逐块行读取所述第一区域的图像数据第一预设次数，从所述第二存储器中逐块行读取第一区域中未读取块行的图像数据第二预设次数。
  7. 根据权利要求6所述的在视频编码装置中进行图像处理的方法，其中，所述根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块，包括：
    从读取的所述第一区域的图像数据中确定出搜索窗的图像数据，所述搜索窗位于所述第一区域内，所述待编码块的左侧与所述搜索窗的最下面一个块行的右侧相邻；
    将所述搜索窗的图像数据存储在第三存储器中，所述第三存储器的读写速度大于所述第一存储器的读写速度的第二预设倍数；
    从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块；
    将与所述待编码块的编码代价最小的块作为所述匹配块。
  8. 根据权利要求7所述的在视频编码装置中进行图像处理的方法，其中，所述从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块，包括：
    从所述第三存储器中读取所述搜索窗的图像数据；
    将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；
    根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；
    根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗中确定出与所述待编码块的编码代价最小的块。
  9. 根据权利要求7所述的在视频编码装置中进行图像处理的方法，其中，所述相对关系为块矢量和残差，所述块矢量为所述匹配块与所述待编码块的相对位移，所述根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码，包括：
    根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码。
  10. 根据权利要求9所述的在视频编码装置中进行图像处理的方法，其中，所述根据所述匹配块与所述待编码块的块矢量和残差，对所述待编码块进行编码，包括：
    将所述匹配块与所述待编码块的残差进行正向变换和量化；
    将所述匹配块与所述待编码块的块矢量以及正向变换和量化后的第一残差数据进行熵编码，得到视频流编码数据；或者
    将所述正向变换和量化后的第一残差数据进行反向量化与变换，得到第二残差数据；
    根据所述第二残差数据对所述待编码块进行重构。
  11. 根据权利要求3所述的在视频编码装置中进行图像处理的方法，其中，所述第一存储器包括设置在视频编码装置外部的系统高速缓存或系统缓冲存储器，所述第二存储器包括设置在视频编码装置外部的动态随机存取内存。
  12. 根据权利要求7所述的在视频编码装置中进行图像处理的方法，其中，所述第三存储器包括设置在视频编码装置内部的缓存或缓冲。
  13. 一种在视频编码装置中进行图像处理的装置，其中，所述装置包括：
    第一确定模块，用于从当前帧图像中确定出待编码块；
    第二确定模块，用于从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域，并将 所述第一区域的图像数据存储在预设存储器中，所述预设存储器的功耗小于预设功耗阈值；
    读取模块，用于从所述预设存储器中读取所述第一区域的图像数据；
    第三确定模块，用于根据所述第一区域的图像数据从所述第一区域中确定出与所述待编码块相匹配的匹配块；
    编码模块，用于根据所述匹配块与所述待编码块的相对关系，对所述待编码块进行编码。
  14. 根据权利要求13所述的在视频编码装置中进行图像处理的装置，其中，所述第一确定模块还用于：
    从所述当前帧图像中确定出待编码块行；
    从所述待编码块行中确定出所述待编码块。
  15. 根据权利要求13所述的在视频编码装置中进行图像处理的装置，其中，所述预设存储器包括第一存储器和第二存储器，所述第二存储器的功耗大于所述第一存储器的功耗的第一预设倍数；所述第二确定模块用于：从所述当前帧图像的重构帧图像中确定出需要多次重复读取的第一区域；将所述第一区域的图像数据分别存储在所述第一存储器和第二存储器中；
    或者所述第一区域包括多个块行，所述第二确定模块用于：
    若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中；将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除；
    或者所述第一区域包括多个块行，每个所述块行包括多个块，所述第二确定模块402用于：
    若所述第一区域的最下面一个块行中新增一个块，则将新增块的图像数据分别存储在所述第一存储器和第二存储器中；将下一个待编码块编码时用不到的块从所述第一存储器中进行移除。
  16. 根据权利要求13所述的在视频编码装置中进行图像处理的装置，其中，所述第三确定模块用于：从读取的所述第一区域的图像数据中确定出搜索窗的图像数据，所述搜索窗位于所述第一区域内，所述待编码块的左侧与所述搜索窗的最下面一个块行的右侧相邻；将所述搜索窗的图像数据存储在第三存储器中，所述第三存储器的读写速度大于所述第一存储器的读写速度的第二预设倍数；从所述第三存储器中读取所述搜索窗的图像数据，并根据所述搜索窗的图像数据，从所述搜索窗中确定出与所述待编码块的编码代价最小的块；将与所述待编码块的编码代价最小的块作为所述匹配块。
    或者所述第三确定模块404用于：
    从所述第三存储器中读取所述搜索窗的图像数据；将所述搜索窗按照预设阶层数进行缩小，得到缩小后的搜索窗；根据所述缩小后的搜索窗的图像数据，从所述缩小后的搜索窗中确定出与所述待编码块的编码代价最小的缩小后的块；根据所述缩小后的块在所述缩小后的搜索窗中的位置，从所述搜索窗中确定出与所述待编码块的编码代价最小的块。
  17. 一种计算机可读的存储介质，其上存储有计算机程序，其中，当所述计算机程序在计算机上执行时，使得所述计算机执行如权利要求1至12中任一项所述的方法。
  18. 一种电子设备，包括存储器，处理器以及视频编码装置，其中，所述处理器通过调用所述存储器中存储的计算机程序，以执行如权利要求1至12中任一项所述的方法。
  19. 一种图像处理系统，其中，包括视频编码装置、第一存储器和第二存储器，所述第二存储器的 功耗大于所述第一存储器的功耗的第一预设倍数，所述视频编码装置包括第三存储器，所述第三存储器的读取速度大于所述第一存储器的读取速度的第二预设倍数，所述第一存储器和第二存储器分别存储当前帧图像的重构帧图像中需要多次重复读取的图像数据，所述视频编码装置在编码时，按照第一预设次数从所述第一存储器以及按照第二预设次数从所述第二存储器读取所述需要多次重复读取的图像数据，并从所述需要多次重复读取的图像数据中确定出搜索窗内的图像数据，将所述搜索窗内的图像数据存储在所述第三存储器中，所述视频编码装置从所述第三存储器中读取所述搜索窗内的图像数据，并确定出与待编码块相匹配的匹配块，根据所述匹配块与待编码块的块矢量和残差进行编码。
  20. 根据权利要求19所述的图像处理系统，其中，所述第一存储器和第二存储器中分别存储从所述当前帧图像的重构帧图像中确定出的需要多次重复读取的第一区域的图像数据，若所述第一区域下移一个块行，则将下移块行的图像数据分别存储在所述第一存储器和第二存储器中，并将下一个待编码块行编码时用不到的块行从所述第一存储器中进行移除，所述视频编码装置在读取时，从所述第一存储器中逐块行读取所述第一区域的图像数据第一预设次数，从所述第二存储器中逐块行读取所述第一区域中未读取块行的图像数据第二预设次数。
PCT/CN2022/077311 2021-04-01 2022-02-22 在视频编码装置中进行图像处理的方法、装置、介质及系统 WO2022206217A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110357576.3A CN115190305A (zh) 2021-04-01 2021-04-01 在视频编码装置中进行图像处理的方法、装置、介质及系统
CN202110357576.3 2021-04-01

Publications (1)

Publication Number Publication Date
WO2022206217A1 true WO2022206217A1 (zh) 2022-10-06

Family

ID=83455582

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/077311 WO2022206217A1 (zh) 2021-04-01 2022-02-22 在视频编码装置中进行图像处理的方法、装置、介质及系统

Country Status (2)

Country Link
CN (1) CN115190305A (zh)
WO (1) WO2022206217A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117291953B (zh) * 2023-11-27 2024-03-08 长春理工大学 一种用于无人机在复杂背景下跟踪地面动态目标的方法

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1997156A (zh) * 2006-12-01 2007-07-11 清华大学 视频编码器中一种快速的搜索块读取模块
CN101360237A (zh) * 2008-08-13 2009-02-04 北京中星微电子有限公司 一种参考帧处理方法、视频解码方法及装置
CN102238385A (zh) * 2010-05-07 2011-11-09 美信集成产品公司 编码器和/或解码器中垂直和/或水平高速缓存设备和方法
CN102592259A (zh) * 2011-01-05 2012-07-18 索尼公司 图像处理设备以及图像处理方法
CN103916674A (zh) * 2012-12-31 2014-07-09 富士通株式会社 存储装置及方法、映射方法、编解码装置和图像处理设备
US20150365693A1 (en) * 2014-06-17 2015-12-17 Stmicroelectronics International N.V. Video encoders/decoders and video encoding/decoding methods for video surveillance applications
CN209593615U (zh) * 2019-03-11 2019-11-05 深圳信息职业技术学院 一种用于视频去隔行的帧场图像缓存及访问电路
CN110495179A (zh) * 2017-04-06 2019-11-22 松下电器（美国）知识产权公司 编码装置、解码装置、编码方法及解码方法
WO2020024152A1 (zh) * 2018-08-01 2020-02-06 深圳市大疆创新科技有限公司 视频处理设备和方法
CN111757116A (zh) * 2018-03-29 2020-10-09 联发科技股份有限公司 具有限制的重建缓冲器的视频编码设备以及关联的视频编码方法
CN112352424A (zh) * 2018-07-02 2021-02-09 华为技术有限公司 用于运动矢量预测的方法和装置

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1997156A (zh) * 2006-12-01 2007-07-11 清华大学 视频编码器中一种快速的搜索块读取模块
CN101360237A (zh) * 2008-08-13 2009-02-04 北京中星微电子有限公司 一种参考帧处理方法、视频解码方法及装置
CN102238385A (zh) * 2010-05-07 2011-11-09 美信集成产品公司 编码器和/或解码器中垂直和/或水平高速缓存设备和方法
CN102592259A (zh) * 2011-01-05 2012-07-18 索尼公司 图像处理设备以及图像处理方法
CN103916674A (zh) * 2012-12-31 2014-07-09 富士通株式会社 存储装置及方法、映射方法、编解码装置和图像处理设备
US20150365693A1 (en) * 2014-06-17 2015-12-17 Stmicroelectronics International N.V. Video encoders/decoders and video encoding/decoding methods for video surveillance applications
CN110495179A (zh) * 2017-04-06 2019-11-22 松下电器（美国）知识产权公司 编码装置、解码装置、编码方法及解码方法
CN111757116A (zh) * 2018-03-29 2020-10-09 联发科技股份有限公司 具有限制的重建缓冲器的视频编码设备以及关联的视频编码方法
CN112352424A (zh) * 2018-07-02 2021-02-09 华为技术有限公司 用于运动矢量预测的方法和装置
WO2020024152A1 (zh) * 2018-08-01 2020-02-06 深圳市大疆创新科技有限公司 视频处理设备和方法
CN209593615U (zh) * 2019-03-11 2019-11-05 深圳信息职业技术学院 一种用于视频去隔行的帧场图像缓存及访问电路

Also Published As

Publication number Publication date
CN115190305A (zh) 2022-10-14

Similar Documents

Publication Publication Date Title
US10057590B2 (en) Method and apparatus using software engine and hardware engine collaborated with each other to achieve hybrid video encoding
US9380312B2 (en) Encoding blocks in video frames containing text using histograms of gradients
US8395634B2 (en) Method and apparatus for processing information
WO2018150024A1 (en) Method for image processing and video compression
JP7492051B2 (ja) クロマブロック予測方法及び装置
US20130128957A1 (en) Apparatus and methodology for a video codec system with noise reduction capability
CN112333446B (zh) 一种帧内块复制参考块压缩方法
WO2022206217A1 (zh) 在视频编码装置中进行图像处理的方法、装置、介质及系统
WO2020143585A1 (zh) 视频编码器、视频解码器及相应方法
KR20230162989A (ko) 멀티미디어 데이터 프로세싱 방법, 장치, 디바이스, 컴퓨터-판독가능 저장 매체, 및 컴퓨터 프로그램 제품
TWI816684B (zh) 視訊編碼器件以及編碼器
US20110249959A1 (en) Video storing method and device based on variable bit allocation and related video encoding and decoding apparatuses
CN116233453B (zh) 一种视频编码方法及装置
US20060171454A1 (en) Method of video coding for handheld apparatus
US20230022526A1 (en) Video processing method and apparatus, device, and storage medium
WO2022206166A1 (zh) 在视频编码装置中进行图像处理的方法、装置及系统
WO2018205781A1 (zh) 一种实现运动估计的方法及电子设备
JP5053774B2 (ja) 動画像符号化装置
KR100891116B1 (ko) 대역폭 인식 움직임 보상 장치 및 그 방법
WO2022206199A1 (zh) 在视频解码装置中进行图像处理的方法、装置及系统
CN114697650A (zh) 基于下采样的帧内划分方法、相关装置设备及介质
WO2018205780A1 (zh) 一种运动估计实现方法及电子设备
JP6610115B2 (ja) 動画像符号化装置、動画像符号化方法、及び動画像符号化プログラム
WO2023065890A1 (zh) 多媒体数据处理方法、装置、计算机设备、计算机可读存储介质及计算机程序产品
RU2783385C2 (ru) Кодер, декодер и соответствующие способы с использованием компактного mv хранилища

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22778402

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22778402

Country of ref document: EP

Kind code of ref document: A1