TW201839714A - Method and system for storing an image - Google Patents

Abstract

Aspects of the disclosure provide a method and device for storing an input image into a memory. The disclosure describes allocating one or more frame buffers in the memory. The disclosure further describes dividing the input image into access units corresponding to subsets of the input image and allocating a main portion and a secondary portion in the frame buffer for each of the access units, wherein at least one of the secondary portions is not sequentially located after its respective main portion within the frame buffer. The disclosure also describes compressing the access units into compressed access units and storing each of the compressed access units into its respective main portion, and if a size of the compressed access unit exceeds a size of the main portion, then storing a remainder of the compressed access unit into its respective secondary portion.

Description

   Method and system for storing input image    [Cross-reference to related applications]

This application claims the priority of the following applications: US Provisional Patent Application No. 62 / 489,588 entitled "Memory Access Efficiency Optimization for Frame Buffer Compression" filed on April 25, 2017, and filed on October 18, 2017 US Patent Application No. 15 / 786,908 entitled "Distributed Access Unit for Frame Buffer Compression", which is incorporated herein by reference in its entirety.

The disclosed embodiments of the present invention relate to storage technology, and more specifically, to a method and system for storing input images.

The prior art description provided herein is used for the purpose of generally presenting the content of the invention. The contents of the current inventor's work include both the work described in this prior art section and aspects not considered prior art at the time of application. These are neither explicitly nor implicitly considered to be the invention current technology.

An electronic device, such as a computer system, may include one or more memories. In one example, the electronic device includes a component, such as a central processing unit (CPU) located on an integrated circuit chip different from the memory, which accesses the memory through a memory controller. The memory accessed by the CPU generates a heavy amount of data transfer between the CPU and the memory.

In view of this, the present invention provides a method and a system for storing an input image to efficiently complete a compressed access unit stored in a memory.

An embodiment of the present invention provides a method for storing an input image, and the input image is stored in a memory. A method for storing an input image includes: allocating one or more frame registers in a memory; dividing the input image into a plurality of access units corresponding to a plurality of subsets of the input image, and storing in the frame register Each access unit of a plurality of access units to assign a main portion and a sub portion, at least one of which is not sequentially located behind its respective main portion in a frame register; compressing the plurality of access units into Multiple compressed access units; and each compressed access unit is stored in its own main section, and if the size of the compressed access unit exceeds the size of the main section, the remainder of the compressed access unit is stored in its own In the vice section.

Another embodiment of the present invention is a system for storing an input image. A system for storing an input image includes: a memory having one or more frame registers; a memory allocation device for receiving an input image; and allocating a frame register in the memory to store the input image, The input image is divided into a plurality of access units corresponding to a plurality of subsets of the input image, and each access unit is assigned a main part and a sub part in a frame register, at least one of which is in a frame. The registers are not sequentially behind their respective main sections; and a memory controller for storing each compressed access unit in the respective main section in response to a plurality of instructions of the memory allocation device, and If the size of the compressed access unit exceeds the size of the main part, the remaining parts of the compressed access unit are stored in the respective secondary parts.

Yet another embodiment of the present invention provides a non-transitory computer-readable medium storing computer-readable instructions. When the processing circuit executes the computer-readable instructions, the processing circuit executes a method. The method includes: allocating one or more frame registers in the memory; dividing the input image into multiple access units corresponding to multiple subsets of the input image, and giving multiple accesses in the frame register Each access unit in the unit is assigned a main part and a sub-part, at least one of which is not sequentially behind its respective main part in the frame register; compressing multiple access units into multiple compressed accesses Units; and each compressed access unit is stored in a respective main portion, and if the size of the compressed access unit exceeds the size of the main portion, the remaining portions of the compressed access unit are stored in respective secondary portions.

The present invention allocates a main part and a sub part in the memory for each access unit. After the access unit is compressed, the compressed access unit is stored in the main part. When the size of the main part is smaller than the compressed access unit, The remainder of the compressed access unit is also stored to the secondary portion so that the compressed access unit can be efficiently accessed.

100‧‧‧Memory System

110‧‧‧Memory allocation device

120‧‧‧Memory Controller

130‧‧‧Memory

131‧‧‧Frame Register

141‧‧‧CPU

142‧‧‧GPU

143‧‧‧Multimedia Engine

144‧‧‧display circuit

145‧‧‧Image Processor

146‧‧‧Video Codec

200‧‧‧ Data Structure

210‧‧‧ input image

221 (1) -221 (3) ‧‧‧Main Section

231A, 231B‧‧‧Frame Register

241 (1) -241 (3) ‧‧‧Vice part

250 (1) -250 (4) ‧‧‧Memory boundary

261-263‧‧‧compressed access unit

221 (1) -221 (3) ‧‧‧Main Section

331A-331C‧‧‧Frame Register

341A-341C‧‧‧Super Block

431A-431C‧‧‧Frame Register

441A-441C‧‧‧Super Block

531A-531B‧‧‧Frame Register

541A-541B‧‧‧Super Block

631A-631B‧‧‧Frame Register

700‧‧‧flow chart

S701-S799‧‧‧step

The present invention can be more fully understood by reading the following detailed description and the examples in combination with the following drawings, wherein the same symbols represent the same elements: FIG. 1 is an exemplary block diagram of a memory system according to an embodiment of the present invention.

FIG. 2 is an exemplary data structure according to an embodiment of the present invention.

FIG. 3 shows three exemplary superblocks in three frame registers according to an embodiment of the present invention.

FIG. 4 illustrates three exemplary superblocks in three frame registers according to an embodiment of the present invention.

FIG. 5 is two exemplary superblocks in two frame registers according to an embodiment of the present invention.

FIG. 6 is an example of an optional frame register according to an embodiment of the present invention.

FIG. 7 is a flowchart describing an exemplary process according to an embodiment of the present invention.

FIG. 1 shows an exemplary block diagram of a memory system 100 according to an embodiment of the present invention. As shown, the memory system 100 may include a memory allocation device 110, a memory controller 120, and a memory 130. The memory 130 may include a frame register 131. The memory system 100 is configured to divide an input image into one or more access units, and store each compressed access unit (compressed access unit) in the frame register 131 and is used for the respective access unit. Main portion and secondary portion.

The memory system 100 may be any suitable system for storing information. In one embodiment, the memory system 100 is an electronic device, such as a desktop computer, a tablet computer, a smart phone, a wearable device, a smart TV, a video camera, a camcorder, a media player, and the like. In one example embodiment, the memory system 100 may further include other components that access data stored in the memory 130. For example, other components may include a CPU 141, a graphics processing unit (GPU) 142, a multimedia engine 143, a display circuit 144, an image processor 145, a video codec 146, and the like.

In one embodiment, the memory 130 may have a sequence of memory blocks separated by a memory boundary based on page size or channel division, for example, at every 32 bytes, 64-bit, 128-bit, 256-bit, 512-bit 1K-byte, 2K-byte, or 4K-byte. Accessing a certain amount of data stored in a block of memory between two adjacent boundaries is more efficient than accessing the same amount of data stored separately in two blocks of memory that cross the memory boundary. Therefore, when the starting address of the data is aligned with the memory boundary, the data in the memory 130 can be effectively accessed by another element of the memory system 100. The memory boundary is formed at an address that is a multiple of the memory block size. In one embodiment, the memory block size can accommodate a certain amount of data, which can be used in burst read / write commands with single or several precharge commands and between the memory 130 and other elements of other memory systems 100 and The serial form of the start command is transmitted quickly. The memory block size may be selected based on characteristics of the memory 130 and other components of the memory system 100 that access the memory 130, such as page size and channel division of the memory 130, and the memory of the memory 130 and the access memory 130 The architecture and operating modes of other elements of the body system 100.

The memory allocation device 110 is configured to receive an input image and divide it into one or more access units. The memory allocation device 110 is further configured to allocate a portion of the memory 130 to the input image, such as the frame register 131, and allocate two memory portions to each access unit in the frame register 131, that is, the main portion And vice section. In one example, the start address of the frame register 131 may be aligned with the memory boundary, for example, at 0 bytes. The memory allocation device 110 is configured to compress each access unit and store each compressed access unit to a respective main part. If the size of the compressed access unit exceeds the size of the main part, the remaining part of the compressed access unit is stored Stored into their respective side sections. In one embodiment, the memory allocation device 100 may be integrated in any element that accesses data stored in the memory 130, such as one or more elements of the memory system 100, including a CPU 141, a GPU 142, a multimedia The engine 143, the display circuit 144, the image processor 145, the video codec 146, and the like.

In one embodiment, the main portion may have a starting address aligned with the memory boundary and be one or more times the size of the memory block. Therefore, the materials stored in the main part can be efficiently accessed. Optionally, when the size of the main part is smaller than the size of the memory block, each main part may be located in a respective memory block, and the start address of one or more main parts may be divided from one or more memories. The lines are aligned.

The size of the auxiliary part may be a part of the size of the memory block. Therefore, two or more secondary parts can be combined together and stored independently of their respective main parts. When the size of the compressed access unit is less than or equal to the size of the main part, the compressed access unit can be completely stored inside the main part without using the auxiliary part. In this way, because the main part can be efficiently accessed, access to the compressed access unit can be efficiently completed.

In another embodiment, at least one of the sub-sections is not consecutively located behind its respective main section in the frame register, which includes the reverse order of the main section having an address greater than its respective sub-section.

The memory controller 120 is configured to manage memory accesses from the memory allocation device 110 to the memory 130. The memory controller 120 may be configured to receive a request from the memory allocation device 110 to store the compressed access unit in the main and sub-parts of the frame register 131 of the memory 130. Based on these requests, the memory controller 120 may send commands to the memory 130 with instructions to store the compressed access units to the respective main and auxiliary portions in the frame register 131. The memory controller 120 may also be used to schedule and cache these requests and the like.

The memory 130 may be any suitable device for storing information. In one embodiment, the memory 130 includes a dynamic random access memory (DRAM) type memory module, such as a double data rate synchronous DRAM (DDR SDRAM), Double data rate two synchronous DRAM (DDR2 SDRAM), double data rate three synchronous DRAM (DDR3 SDRAM), double data rate four synchronous DRAM (DDR4) SDRAM), low power DDR SDRAM (low power DDR SDRAM, LPDDR SDRAM), etc.

In one embodiment, the memory system 100 may be a system-on-chip (SOC), where all components are located on a single-chip integrated circuit (IC) chip. In addition, other components such as the CPU 141, the GPU 142, the multimedia engine 143, the display circuit 144, the image processor 145, and the video transcoder 146 may also be included on the same monolithic IC chip. Alternatively, the elements in the memory system 100 may be distributed across several ICs. For example, the memory distribution device 110, the memory controller 120, the memory 130, and other components of the memory system 100 may be located on multiple IC chips. In addition, the memory allocation device 110 may be integrated into any element that accesses data stored in the memory 130, such as one or more elements of the memory system 100, which includes a CPU 141, a GPU 142, a multimedia engine 143, a display A circuit 144, an image processor 145, a video codec 146, and the like.

During operation, the input image may be received by the memory allocation device 110. The memory allocation device 110 may divide an input image into one or more access units. In addition, the memory allocation device 110 may allocate a portion of the memory 130, such as the frame register 131, to the input image. Two memory sections, a main section and a sub section, are allocated to each access unit in the frame register 131. Under the instruction of the memory allocation device 110, the memory controller 120 may store the compressed access unit to its respective main part and a sub-part according to the size situation. The main portion may have a starting address aligned with the memory boundary and a size that is one or more times the size of the memory block. The size of the auxiliary portion may be a part of the size of the memory block. Therefore, two or more secondary parts can be combined together and stored independently of their respective main parts. When the size of the compressed access unit is less than or equal to the size of the main part, the compressed access unit can be completely stored inside the main part without using the auxiliary part. In this way, access to the compressed access unit can be efficiently completed.

FIG. 2 is an exemplary data structure 200 according to an embodiment of the present invention, which shows an input image 210, a frame register 231A, and a frame register 231B that are divided into access units. As shown, the input image 210 may be divided into an array of N × M access units. Inside the array, the size of the access unit depends on the number of pixels and pixel bit-depth in the access unit. The pixel bit depth is the number of bits of a pixel for a specified color, for example, 10 bits or 12 bits, which correspond to 1024 colors or 4049 colors, respectively. In one example, the number of pixels in the access unit may depend on the compression method used by the memory allocation device 110, for example, the size of the compression unit depends on which compression method is used to operate. For example, the size of the compression unit may be 4 × 4 pixels, 8 × 8 pixels, 16 × 4 pixels, 16 × 8 pixels, 16 × 16 pixels, and the like. An access unit may have one or more compression units.

The frame register 231A shows an exemplary frame register structure for storing an input image. The frame register may be a memory having an accessible location for storing data. The accessible locations within the memory can be combined into a memory block having a memory block size. As described above, the memory block size may be selected based on the characteristics of the memory 130 and other components of the memory system 100 that access the memory 130, such as page size and channel division of the memory 130, and the memory 130 and access memory The architecture and operation modes of other elements of the memory system 100 of the bank 130. In one example, the memory block size can be selected as 32-bit, 64-bit, 128-bit, 256-bit, 512-bit, 1K-byte, 2K-byte, 4K-bit Tuples, etc. For example, the memory 130 may be a DDR3 SDRAM device, and data is retrieved from the memory 130. The memory block size is the amount of data, which can be retrieved from the memory 130 in a single read cycle. Specifically, when the data bus width and burst length are 64 bits (that is, 8 bytes) and 4 respectively, the memory block size may be 32 bytes. In another example, the memory block size may be determined by a CPU 141 or a GPU 142 cache line that accesses the memory as a 64-bit byte, a 128-bit byte, or the like. In the example in FIG. 2, the memory block size is 64 bytes, and therefore, in the frame register 231A and the frame register 231B, the memory dividing line 250 (1) -250 (n) is located at the 0 bit Tuples, 64-bits, 128-bits, 192-bits, etc.

A main part and a sub part can be assigned to each access unit. In one embodiment, the size of the main part may depend on the compressibility of the input image, the compression method, the memory block size, and the like. Furthermore, in one embodiment, the sum of the size of the main part and the size of the sub-part may be equal to the size of the access unit. Likewise, the ratio of the size of the main part to the size of the sub part may depend on the compressibility of the input image, the compression method, the size of the memory block, and the like. For example, when the access unit can be compressed to a smaller size, the smaller main part is sufficient to store the compressed access unit, and the respective sub-parts can be left empty, so that the size of the main part is proportional to the size of the sub-part smaller. For example, the ratio of the size of the main portion to the size of the sub portion may be 2, 4, or 8, and so on.

In addition, the main part may have a starting address aligned with the memory boundary and a size that is a multiple of the memory block size, so that the data stored in the main part can be efficiently accessed. In the example of Figure 2, the size of the main part 221 (1) -221 (3) can be selected to have a memory block size of 64 bytes, and the start of the main part 221 (1) -221 (3) The addresses are aligned with the memory boundary 250 (1) -250 (3), respectively. The size of the respective sub-portions 241 (1) -241 (3) may be selected to be smaller than 64 bytes, such as 32 bytes.

The frame register 231B shows an example when the compressed access units 261-263 having various sizes are stored. The compressed access units 261-263 can be stored in the respective main sections 221 (1) -221 (3) and the sub-sections 241 (1) -241 (3). In the example of FIG. 2, the size of the compressed access unit 261 is smaller than the memory block size of a 64-bit byte. Therefore, the compressed access unit 261 can be completely stored in the main section 221 (1), while the sub-section 241 (1) remains empty. The size of the compressed access unit 262 is equal to the memory block size of a 64-bit byte. Therefore, the compressed access unit 262 can be completely stored in the main section 221 (2), while the sub-section 241 (2) remains empty. However, the size of the compressed access unit 263 is larger than the memory block size of a 64-bit byte. Therefore, the first part of the compressed access unit 263 may be filled in the main part 221 (3), and the second part or the remaining part of the compressed access unit 263 may be stored in the sub-part 241 (3).

In various embodiments, the size of the access unit, the size of the main part, and the size of the sub-part may be selected and kept constant for the input image. On the other hand, multiple input images, such as video frames, can be stored by the memory system 100. The size of the access unit, the size of the main part, and the size of the sub-part can be selected for each individual input image, so it can dynamically change from one input image to another.

The main portion and the sub portion may be set in the frame register 131 according to various layouts. Figures 3-5 show example layouts that include a repeating model of the primary and secondary sections in a periodic manner, with the smallest repeating unit being a superblock. Therefore, the main part and the sub-part in the frame register 131 may be arranged, for example, by sequentially placing super blocks adjacent to each other. In one embodiment, the size of the superblock may be a multiple of the size of the memory block.

FIG. 3 is three exemplary superblocks 341A-341C in the three frame registers 331A-331C according to an embodiment of the present invention. The superblocks 341A-341C share the same characteristics, in which all the subparts in the superblock are combined into a subpart group, which is located in the middle of the respective superblock. The starting address of the main section is aligned with the memory boundary. The size of the sub-section group is one or more times the size of the memory block, and the first sub-section of the sub-section group is aligned with the memory boundary.

Referring to super block 341A, the size of the access unit is set to 160 bytes, the size of the memory block is set to 128 bytes, and the size of the main portion and the sub portion are set to 128 bytes and 32 bytes, respectively. . As shown in the figure, the superblock model has a sub-section group including four sub-sections (ie, S0-S3), which are inserted in the first main section group (ie, M0-M1) and the second main section group (ie, M2- M3). The size of the super block 341A is 5 times the size of the memory block (ie, 640 bytes). The memory boundary of super block 341A is located at 0 bytes, 128 bytes, 256 bytes, 384 bytes, 512 bytes, and 640 bytes. The starting bits of all main parts M0-M3 The addresses are aligned with the memory boundaries at 0, 128, 384, and 512 bytes, respectively. The sub-section has a size of 128 bytes, and the first sub-section S0 is aligned with the memory boundary line located at 256 bytes.

Referring to super block 341B, the size of the access unit is set to 192 bytes, the size of the memory block is set to 128 bytes, and the size of the main part and the subsidiary part are set to 128 bytes and 64 bytes . As shown in the figure, the superblock model has a sub-section group including four sub-sections (ie, S0-S3), which are inserted in the first main section group (ie, M0-M1) and the second main section group (ie, M2- M3). The size of the super block 341B is 6 times the size of the memory block (that is, 768 bytes). The memory boundary of super block 341B is located at 0 bytes, 128 bytes, 256 bytes, 384 bytes, 512 bytes, 640 bytes, and 768 bytes. All main parts M0- The starting address of M3 is aligned with the memory boundaries at 0 bytes, 128 bytes, 512 bytes, and 640 bytes, respectively. The sub-section has a size of 256 bytes, and the first sub-section S0 is aligned with a memory boundary line located at 256-bytes.

Referring to super block 341C, the size of the access unit is set to 384 bytes, the size of the memory block is set to 256 bytes, and the size of the main part and the subsidiary part are set to 256 bytes and 128 bytes, respectively. . As shown, the super block model has a sub-section group including two sub-sections (ie, S0-S1), which is inserted between the first main section M0 and the second main section M1. The size of the super block 341C is three times the size of the memory block (that is, 768 bytes). The memory demarcation line of super block 341C is located at 0 bytes, 256 bytes, 512 bytes, and 768 bytes. The starting address of the main part M0-M1 and the memory located at 0 byte are respectively The demarcation line is aligned with the 512-byte memory demarcation line. The sub-section has a size of 256 bytes, and the first sub-section S0 is aligned with a memory boundary line located at 256-bytes.

FIG. 4 shows three exemplary superblocks 441A-441C in the three frame registers 431A-431C according to an embodiment of the present invention. Superblocks 441A-441C share the same characteristics, in which all the sub-parts in the super-block are combined into a sub-part group, which follows the main part group containing the main part. The starting address of the main section is aligned with the memory boundary. The size of the sub-section group is one or more times the size of the memory block, and the first sub-section of the sub-section group is aligned with the memory boundary.

Referring to super block 441A, the size of the access unit is set to 192 bytes, the size of the memory block is set to 128 bytes, and the size of the main part and the subsidiary part are set to 128 bytes and 64 bytes, respectively. . As shown, the superblock model 441A has a sub-section group including two sub-sections (ie, S0-S1), which follows the main section group (ie, M0-M1). The size of the super block 441A is three times the size of the memory block (ie, 384 bytes). The memory boundary of super block 441A is located at 0 bytes, 128 bytes, 256 bytes, and 384 bytes. The starting addresses of all main parts M0-M1 and the memory located at 0 bytes are respectively The volume boundary is aligned with the memory boundary at 128 bytes. The sub-section has a size of 128 bytes, and the first sub-section S0 is aligned with the memory boundary line located at 256 bytes.

Referring to super block 441B, the size of the access unit is set to 160 bytes, the size of the memory block is set to 128 bytes, and the size of the main part and the subsidiary part are set to 128 bytes and 32 bytes, respectively. . As shown, the superblock model has a sub-part group including four sub-parts (ie, S0-S3), which follows the main part group (ie, M0-M3). The size of the super block 441B is five times the size of the memory block (ie, 640 bytes). The memory boundary of super block 441B is located at 0 bytes, 128 bytes, 256 bytes, 384 bytes, 512 bytes, and 640 bytes. The starting bits of all main parts M0-M3 The addresses are aligned with the memory boundaries at 0, 128, 256, and 384 bytes, respectively. The sub-section has a size of 256 bytes, and the first sub-section S0 is aligned with the memory boundary at 512 bytes.

Referring to super block 441C, the size of the access unit is set to 320 bytes, the size of the memory block is set to 128 bytes, and the size of the main portion and the sub portion are set to 256 bytes and 64 bytes, respectively. . As shown, the superblock model has a sub-section group including two sub-sections (i.e., S0-S1), which follows the main section group (i.e., M0-M1). The size of the super block 441C is 5 times the size of the memory block (ie, 640 bytes). The memory boundary of super block 441C is located at 0 bytes, 128 bytes, 256 bytes, 512 bytes, and 640 bytes. The starting addresses of all main parts M0-M1 are located at 0 and 0 respectively. Bytes are aligned with the memory boundary at 256 bytes. The sub-section has a size of 256 bytes, and the first sub-section S0 is aligned with the memory boundary line located at 512 bytes.

FIG. 5 is two exemplary superblocks 541A-541B in two frame registers 531A-531B according to an embodiment of the present invention. Superblocks 541A-541C share the same characteristics, where the size of the main part (that is, 128 bytes) is smaller than the size of the memory block (that is, 256 bytes). In addition, some compressed access units may need to be stored in the main part and the secondary part, and some compressed access units may be stored in the main part. In order to allow efficient access to the compressed access units stored in the main part and the sub-part, as many sub-parts corresponding to the main part may be contained in the same memory block, and preferably, they follow each other The main part. For example, in the super block 541A, in its respective memory block, the main part M0 is followed by its sub-part S0, and the main part M3 is followed by its sub-part S3.

Referring to super block 541A, the size of the access unit is set to 192 bytes, the size of the memory block is set to 256 bytes, and the size of the main part and the sub part are set to 128 bytes and 64 bytes, respectively. . As shown, the superblock model has a sub-section S0, which follows the respective main section M0, and a sub-section S3, which follows the respective main section M3. The size of the super block 541A is three times the size of the memory block (that is, 768 bytes). The memory boundary of super block 541A is located at 0 bytes, 256 bytes, 512 bytes, and 768 bytes. The starting addresses of the three main parts M0, M1, and M3 are respectively located at 0 bits. Memory boundaries are aligned at groups, 256 bytes, and 512 bytes. The main portion M2 is not aligned with the memory boundary, but the main portion M2 is located inside a single memory block between 256 bytes and 512 bytes.

Referring to super block 541B, the size of the access unit is set to 192 bytes, the size of the memory block is set to 256 bytes, and the size of the main part and the sub part are set to 128 bytes and 64 bytes, respectively. . As shown, the superblock model has a sub-section S0 following its main section M0, and a sub-section S1 following its main section M1. The size of the super block 541B is three times the size of the memory block (that is, 768 bytes). The memory boundary of super block 541A is located at 0 bytes, 256 bytes, 512 bytes, and 768 bytes. The starting addresses of the three main parts M0, M1, and M2 are located at 0 bits, respectively. Memory boundaries are aligned at groups, 256 bytes, and 512 bytes. The main part M3 is not aligned with the memory boundary, but the main part M3 is located inside a single memory block between 512 bytes and 768 bytes.

Figure 6 shows an example of an optional frame register according to an embodiment of the invention. By having two groups, that is, a main part group and a sub part group, the main part and the sub part of the frame register 631A and the frame register 631B can be arranged, wherein the main part group includes all the main parts placed next to each other sequentially And the sub-section group includes all the sub-sections placed next to each other sequentially. In one embodiment, the size of the main part is one or more times the size of the memory block, and the starting address of the main part may be aligned with the memory boundary. The main part group may be placed adjacent to the sub part group or may be separated from the sub part.

As shown in the frame register 631A, the size of the access unit is set to 80 bytes, the size of the memory block is set to 64 bytes, and the size of the main part and the subsidiary part are set to 64 bytes and 16 bytes. The main part group includes all main parts. As shown in the figure, the main part is placed next to each other and has a start address, which is located at 0 bytes, 64 bytes, 128 bytes, 192 bytes, 256 bytes, etc. Of continuous memory boundaries. The sub-section group includes all sub-sections. The first sub-portion S0 may have a starting address aligned with the memory boundary, for example, aligned with the memory boundary at 512 bytes.

As shown in the frame register 631B, the size of the access unit is set to 160 bytes, the size of the memory block is set to 128 bytes, and the size of the main portion and the sub portion are set to 128 bytes and 32 bytes. The main part group includes all main parts. As shown in the figure, the main part is placed next to each other and has a start address, which is located at 0 bytes, 128 bytes, 256 bytes, 384 bytes, 512 bytes, etc. Of continuous memory boundaries. The sub-section group includes all sub-sections. The first sub-portion S0 may have a starting address aligned with the memory boundary line and aligned with the memory boundary line at 4096 bytes.

In the superblocks and frame registers shown in FIGS. 3-6, at least one of the sub-parts is not sequentially behind their respective main parts.

In one embodiment, the starting addresses of the super block and the frame register may be aligned with the memory boundary of the memory 130, for example, at the zero byte shown in FIG. 3 to FIG. 6.

Although exemplary superblocks and frame registers are shown in Figures 3-6, it should be understood that in order to meet different memory usage scenarios, such as the deformation of the superblock model, the location of the superblock in the frame memory Other deformations are possible.

During operation, when the main part and the sub-part are placed in the frame register 131 according to the layout, such as the layout shown in Figs. 3-6, the compressed access units can be stored in the respective main parts in. For example, the compressed access units may be stored in respective main sections and, if necessary, in respective sub-sections. When the size of the compressed access unit is equal to or smaller than the size of the respective main portion, the compressed access unit may be completely stored in the respective main portion, and the corresponding sub-portion may remain empty.

FIG. 7 is a flowchart describing an exemplary process 700 according to an embodiment of the present invention. In one example, the process 700 is executed by the memory system 100 in FIG. 1. The flow starts at step S701 and continues to step S710.

In step S710, the input image is divided into one or more access units, for example, an access unit of an N × M array shown in FIG. 2. In one example, the memory allocation device 110 is configured to divide an input image into an array of access units. The input image may be a video frame, a photographed image, a graphic image, an animated image, and the like. For example, the video frame may be a reference frame used by the video codec 146. The flow then proceeds to step S720.

In step S720, a frame register is allocated in the memory. In one example, the memory allocation device 110 is used to allocate the frame register 131 in the memory 130. The size of the frame register is equal to or larger than the size of the input image. In one embodiment, the starting address of the frame register may be aligned with the memory boundary, such as the memory boundary at 0 bytes.

In step S730, two memory sections are allocated to each access unit in the frame register, that is, a main section and a sub-section. In one example, the memory allocation device is used to allocate a main part and a sub part to each access unit in the frame register 131. In one embodiment, the sum of the size of the main part and the size of the sub-part may be equal to the size of the access unit, for example, may be the size of the uncompressed access unit. In one embodiment, the size of the main part may depend on the compressibility, compression method, memory block size, etc. of the input image. In addition, in one embodiment, the ratio of the size of the main portion to the size of the sub portion may depend on the compressibility, compression method, and the like of the input image. For example, when the access unit can be compressed to a smaller size, the smaller main part is sufficient to store the compressed access unit, and the respective sub-parts can be left empty, so that the ratio of the size of the main part to the size of the sub-part is more small.

Further, in an embodiment, the main part may have a starting address aligned with the memory boundary and a size that is one or more times the size of the memory block, so that the data stored in the main part can be effectively used. To visit.

Optionally, when the size of the main part is smaller than the size of the memory block, each main part may be located within a respective memory block, and one or more main parts may have a line aligned with one or more memory boundaries. The starting address.

In one embodiment, the size of the secondary portion may be a portion of the size of the memory block. Therefore, two or more sub-sections can be combined together as one or more sub-section groups and stored separately from their respective main sections. Further, in one embodiment, the first sub-portion in each respective sub-portion group may have a starting address aligned with a memory boundary.

In another embodiment, in the frame register, at least one sub-portion is not sequentially behind its respective main portion.

The main part and the sub-part may be arranged in a frame register, such as the frame register 131 according to different layouts. In one embodiment, the layout may include a repeating model of a superblock, where the superblock is the smallest repeating unit in the frame register. Therefore, the main part and the sub-part in the frame register can be arranged, for example, by sequentially placing super blocks adjacent to each other. The size of the super block can be set to a multiple of the memory block size.

In one embodiment, the starting address of the main part in the superblock is aligned with the memory boundary. The sub-sections in a superblock can be combined into one or more sub-section groups whose size is a multiple of the memory block size. The first sub-section in each sub-section group may be aligned with the memory boundary. Some exemplary superblocks with the above characteristics are shown in Figures 3 and 4.

In another embodiment, the superblock may have one or more main sections that are smaller than the memory block size. Some exemplary superblocks are shown in Figure 5. For example, as many secondary sections as possible follow the respective main sections (for example, S0 follows M0 and S3 follows M3 in superblock 541A of FIG. 5). As another example, each main part is completely located in the same memory block.

In one embodiment, the layout does not include a repeating model of superblocks. In contrast, by having a main part group and a sub part group, the main part and the sub part in the frame register can be arranged, for example, as shown in the example in FIG. 6. For example, the main section group includes a main section having a start address aligned with consecutive memory boundaries. The sub-section group includes sub-sections adjacent to each other. The first sub-section of the sub-section group may be aligned with the memory boundary.

In step S740, the access unit may be compressed into a compressed access unit to reduce the bandwidth requirement for data transmission between the memory and another device accessing the memory. For example, in the memory system 100, the memory 130 may be located on a different chip from the memory allocation device 110. The memory allocation device 110 is used to compress the access unit to reduce data transmission between the memory 130 and the memory allocation device 110. Bandwidth requirements. Both the distortionless compression method and the distortion compression method can be used to compress the access unit. The lossless compression method can protect the quality of the original data, while the distortion compression method can achieve more compression. The compression method may be a general compression method, an image compression method, or a video compression method. For example, the compression method may include run-length encoding, dictionary-based algorithms, Hoffman encoding, deflation, chroma subsampling, discrete cosine transform, and the like.

In step S750, the size of the compressed access unit is compared with the size of the main portion. In one example, the memory allocation device 110 is used to compare the size of the compressed access unit with the size of the main portion. If the size of the compressed access unit is larger than the size of the main part, the flow proceeds to step S770. Otherwise, the flow proceeds to step S760.

In step S760, the compressed access units can be completely stored in the respective main parts, because the size of the compressed access units is less than or equal to the size of the main part. In one example, the memory controller 120 is configured to store the compressed access units to their respective main sections in response to instructions from the memory allocation device 110.

When the size of the compressed access unit is larger than the size of the main part, the flow proceeds to step S770. In step S770, the first part of the compressed access unit may be stored to the respective main parts. The first part of the compressed access unit can be the same size as the main part and filled in the respective main part. In one example, the memory controller 120 is configured to store the first portion of the compressed access unit to its respective main portion in response to an instruction from the memory allocation device 110.

In step S780, the second part or the remaining part of the compressed access unit may be stored in the respective sub-parts. Therefore, when the size of the compressed access unit is larger than the size of the main portion, the compressed access unit can be stored separately to the respective main and secondary portions. In one example, the memory controller 120 is configured to store the remaining portions of the compressed access unit to respective secondary portions in response to instructions from the memory allocation device 110.

Before the process continues to step S799 where the process ends, steps S740 to S780 may be repeatedly performed for all the access units. In one example, the memory allocation device 110 and the memory controller 120 are configured to repeatedly execute steps S740 to S780 for inputting all the access units in the image.

In various embodiments, the size of the access unit, the size of the main part, and the size of the sub-part may be selected and kept constant for the input image. On the other hand, multiple input images, such as sequence frames of a video, can be stored in memory. The size of the access unit, the size of the main part, and the size of the sub-part can be selected for each individual input image, and therefore, it can dynamically change from one input image to another.

In various examples, the memory allocation device 110 or the functions of the memory allocation device 110 may be implemented in hardware, software, and a combination thereof. In one example, the memory allocation device 110 is implemented in hardware, such as a processing circuit. The hardware may include one of discrete components, integrated circuits, application-specific integrated circuits (ASICs), or the like. Multiple. In another example, the memory distribution function may be implemented by software or firmware including instructions stored in a computer-readable non-transitory storage medium, and when these instructions are executed by a processing circuit, the processing circuits are caused to perform respective functions.

Since the various aspects of the present invention have been described in connection with specific embodiments of the present invention that have been proposed as examples, substitutions, modifications, and variations of these examples can be made. Therefore, the embodiments described herein are used for illustrative purposes, but not for limitation. Changes may be made without departing from the scope of the claims.

Claims (20)

    A method for storing an input image, said input image being stored in a memory, comprising: allocating one or more frame registers in said memory; and dividing said input image into corresponding to said input Multiple access units of multiple subsets of the image, and each access unit of the multiple access units in the frame register is assigned a main part and a sub part, wherein at least one of the sub parts is in the The frame registers are not sequentially behind their respective main sections; compressing the plurality of access units into a plurality of compressed access units; and storing each compressed access unit in a respective main section, and If the size of the compressed access unit exceeds the size of the main portion, the remaining portions of the compressed access unit are stored in respective secondary portions.         The method for storing an input image according to item 1 of the patent application scope, wherein the memory has a series of memory blocks, and the memory blocks are located at multiple addresses that are multiples of the memory block size. A plurality of memory boundaries are separated, and the size of the memory block is determined based on characteristics of the memory and characteristics of a plurality of devices accessing the memory.         The method for storing an input image according to item 2 of the scope of the patent application, wherein the size of each main part is one or more times the size of the memory block, and the starting address of each main part is Memory boundaries are aligned.         The method for storing an input image according to item 2 of the scope of the patent application, wherein the size of each main part is a part of the size of the memory block, and each main part is located inside the respective memory block.         The method for storing an input image according to item 2 of the scope of patent application, wherein the size of each sub-portion is a part of the size of the memory block, and a plurality of sub-portions are combined into one or more sub-portion groups.         The method for storing an input image according to item 5 of the scope of patent application, wherein the size of the one or more sub-groups is one or more times the size of the memory block, and each of the sub-groups is The starting address of the first subsection of aligns with the memory boundary.         The method for storing an input image according to item 2 of the scope of patent application, wherein a plurality of main parts and sub-parts are arranged in a preset model to form a super size that is one or more times the size of the memory block. Blocks; and the plurality of main sections and the plurality of sub sections in the frame register are arranged by sequentially placing a plurality of super blocks adjacent to each other.         The method for storing an input image according to item 2 of the scope of patent application, wherein the memory block size is selected as 32-bit, 64-bit, 128-bit, 256-bit, 512-bit Group, 1K byte, 2K byte, or 4K byte.         The method for storing an input image according to item 1 of the scope of patent application, wherein the input image is a still image or a video frame.         A system for storing input images includes: a memory having one or more frame registers; a memory allocation device for receiving the input images; and allocating frame registers in the memory to store The input image, the input image is divided into a plurality of access units corresponding to a plurality of subsets of the input image, and each access unit is assigned a master in the frame register A section and a sub-section, at least one of the sub-sections is not sequentially behind their respective main sections in the frame register; and a memory controller for responding to a plurality of the memory allocation devices Instructions to store each compressed access unit in a respective main portion, and if the size of the compressed access unit exceeds the size of the main portion, store the remaining portion of the compressed access unit in a respective In the subsection.         The system for storing an input image according to item 10 of the scope of patent application, wherein the memory has a series of memory blocks, and the memory blocks are located at multiple addresses that are multiples of the memory block size. A plurality of memory boundaries are separated, and the size of the memory block is determined based on characteristics of the memory and characteristics of multiple devices accessing the memory.         The system for storing input images according to item 11 of the scope of patent application, wherein the memory allocation device is configured to select a size of each main part to be one or more times the size of the memory block, and The starting address of each main section is aligned with the memory boundary.         The system for storing an input image according to item 11 of the patent application scope, wherein the memory allocation device is configured to select a size of each main part as a part of the size of the memory block, and Placed inside the respective memory block.         The system for storing an input image according to item 11 of the scope of patent application, wherein the memory allocation device is configured to select a size of each sub-portion as a part of the size of the memory block, and a plurality of sub-portions Group one or more sub-groups.         The system for storing input images according to item 14 of the scope of patent application, wherein the memory allocation device is configured to select one or more sub-groups whose size is one or more times the size of the memory block, The starting address of the first sub-section of each of the sub-section groups is aligned with the memory boundary.         The system for storing input images according to item 11 of the scope of patent application, wherein the memory allocation device is used to arrange a plurality of main parts and sub-parts in a preset model to form a memory block having a size equal to the size of the memory block. One or more times of super blocks, and multiple super blocks adjacent to each other are placed in the frame register.         The system for storing input images according to item 11 of the scope of patent application, wherein the memory allocation device is configured to determine the size of the memory block as 32-bit, 64-bit, 128-bit, 256 Bytes, 512 bytes, 1K bytes, 2K bytes, or 4K bytes.         The system for storing input images according to item 10 of the scope of patent application, wherein the memory is located on an integrated circuit chip different from the memory distribution device.         The system for storing input images as described in claim 10, wherein the memory allocation device is integrated in a video codec.         A non-transitory computer-readable medium storing computer-readable instructions. When the processing circuit executes the computer-readable instructions, the processing circuit executes a method. The method includes: allocating one or Multiple frame registers; dividing the input image into multiple access units corresponding to multiple subsets of the input image, and giving each of the multiple access units in the frame register Access units are assigned a main part and a sub-part, at least one of which is not sequentially behind its respective main part in the frame register; compressing the plurality of access units into a plurality of compressed accesses Unit; each compressed access unit is stored in a respective main section, and if the size of the compressed access unit exceeds the size of the main section, the remaining portion of the compressed access unit is stored in a respective In the subsection.