JP2010073210A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus that allows enhancement of throughput by executing processors in parallel. <P>SOLUTION: In an image processing method, image data developed in a memory are divided longways, each piece of the divided image data is performed with JPEG (Joint Photographic Experts Group) compression processing in parallel; a prescribed amount of code data are asynchronously written every time a prescribed amount of the compression data are stored; and information allowing identification of writing of own processor is recorded in the memory. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image processing apparatus capable of increasing throughput by executing processors in parallel.

  An image processing apparatus called an MFP (Multi-Functional peripheral) is widely used.

  In MFPs, there is an increasing demand for faster image processing such as printing and higher resolution. In addition, many MFPs with functions such as facsimiles and image scanners have been supplied to the market, and the need for high-speed processing of large-capacity digital image data is increasing.

  Various image processing performed in the MFP generally requires a large number of calculations to be performed as fast as possible. As a method for increasing the processing speed, generally, individual calculation processes are performed in parallel. A method for improving the throughput is effective, and many apparatuses and methods have already been proposed.

  Patent Document 1 (prior art document) discloses a technique in which an image is generated as a packet (a data string including an image subdivision and an attribute), and each packet is processed in parallel using a plurality of data flow image processing units. Has been proposed.

  Patent Document 2 (prior art document) proposes an image processing apparatus that causes a plurality of processing units to execute processing in parallel for each drawing object (graphic, text, image, etc.).

  In addition to this, several methods for executing processing units in parallel have been proposed as effective methods for high-speed processing of a large amount of image data. According to these methods, since a plurality of image packets are compressed for each image packet, each generated compressed image packet (compressed packet) has a small amount of data and can be processed independently. It can be handled as one processing unit (image unit). Therefore, when the data amount of the compressed image packet is small, it is highly convenient for use and storage in the MFP or transfer through the communication network.

  There are various algorithms for compressing image packets, but in recent years, the JPEG (Joint Photographic Experts Group) method has become widespread.

  The major advantage of adopting JPEG image compression for MFP is the standard and widely used method in the market, and it is improved in the process of design development accumulated so far when using hardware circuits etc. Can be adopted, and the reliability and stability are high, and the format is fixed in the data transfer inside the device. The reusability is high.

  Thus, the compressed image packet obtained by dividing the image data of the original image into a plurality of image packets and performing image compression by the JPEG method for each image packet (hereinafter abbreviated as “JPEG compression”). By using an image processing apparatus that performs parallel processing, it is possible to perform high-speed data transfer / processing and handle it with a highly efficient capacity.

  In Patent Document 3 (prior art document), regarding the compression of JPEG format, which is an image data compression process, Mcu (Minimum code unit) which is a compression unit is grouped for compression of one-page image, and the compression code data of the group is stored. A method of processing data in parallel while adding a restart marker for resetting the dependency between Mcu to the compressed packet has been proposed.

  The JPEG compression shown in Document 1 or Document 2 has the above-mentioned advantages, but the compressed data length is variable, and there is a dependency between the compressed data, so that both compression and decoding are image data. Due to the necessity of processing from the beginning, it is very difficult to speed up the parallel processing by a plurality of processors.

  On the other hand, in the method disclosed in Document 3, since the code data length is made uniform every time the encoded data of the Mcu group is written, the data redundancy increases and the compression rate decreases when the Mcu group increases. There is a problem of doing. Further, if the end of data processing must be confirmed for each Mcu group, and the processing time for each processor varies depending on the data to be processed (processing time for each processor differs), the processor that has completed processing It is known that the processing of another processor must be waited for, and the utilization rate of the processor decreases, and as a result, the processing time of compression / decompression processing increases.

  SUMMARY OF THE INVENTION An object of the present invention is to provide means and an image format for performing high-speed image compression / decompression with a high compression ratio in a system having a plurality of processors, based on JPEG. Another object of the present invention is to provide an MFP capable of performing high-speed compression / decompression of large image data by parallel processing in a many-core or multiprocessor system.

  The present invention has been made on the basis of the above problems. In the JPEG method, image processing of a plurality of regions divided in a matrix in two directions orthogonal to each other is independently compressed in parallel by two or more processors. In the method, the image processing method is characterized in that each piece of image data includes information for specifying a processor when the image data of the area is compressed.

  According to one embodiment of the present invention, large-size image data can be compressed and decompressed at high speed by parallel processing in a many-core or multiprocessor system.

  That is, by applying the image compression / decompression method of the present invention, in a system equipped with a multi-core CPU, when performing compression / decompression in parallel, the execution rate of processing executed by each core is high, and synchronous processing related to memory access is performed. The overhead can be kept low.

  Even when compression / decompression processing is performed on another system, the execution rate of the system with the higher degree of parallelism can be increased in each system that performs compression / decompression. Total throughput can be improved.

  In addition, the processing speed is reduced by suppressing the number of swap operations between the secondary cache memory that is commonly accessed from a plurality of cores, the primary cache memory in the core, and the shared secondary cache memory. Can be suppressed.

  In addition, it is possible to prevent the processing speed from being lowered by performing data transfer of DMA transfer between the secondary cache memory and the shared memory with an appropriate size.

  Therefore, the compression / decompression process by the multi-core CPU can be processed at high speed without impairing the image quality.

Schematic which shows an example of the variable-length image compression to which this invention is applied. Schematic explaining the compression system in which compression and expansion | extension processing by multiple PE of this invention are possible. 1 is a block diagram of an entire system according to an embodiment of the present invention. The flowchart of the whole process in connection with embodiment of this invention. The flowchart of the compression process in connection with embodiment of this invention. The flowchart of the expansion | extension process in connection with embodiment of this invention. The block diagram of the whole system in connection with another (2nd) embodiment of this invention. Schematic explaining the memory operation between the primary and secondary cache in the system according to the (second) embodiment shown in FIG. Schematic explaining the memory operation between the primary and secondary cache in the system according to the (second) embodiment shown in FIG. The block diagram of the whole system in connection with another (3rd) embodiment of this invention. FIG. 10 is a schematic diagram for explaining a parallel division number determination method and an example of arrangement of compressed data in the (third) embodiment shown in FIG. 9. FIG. 10 is a schematic diagram illustrating an example of processing performed by a CPU having a smaller number of cores than the number of divisions in the (third) system illustrated in FIG. 9. FIG. 10 is a schematic diagram for explaining another example (including the characteristic improvement result of the present application) of processing by a CPU having a smaller number of cores than the number of divisions in the (third) system shown in FIG. 9. 10 is a flowchart of processing in the client PC in the (third) system shown in FIG. 9. 14 is a flowchart for explaining “parallel division number determination processing” in the processing in the client PC shown in FIG. 13. FIG. 10 is a flowchart of processing in the printer controller in the (third) system shown in FIG. 9. FIG. The memory utilization image of the process regarding another (4th) embodiment of this invention. FIG. 17 is a schematic diagram for explaining data (image thereof) arranged on a shared memory by one of the cores by the method shown in FIG. 16. 17 is a flowchart for explaining “division number determination processing” executed in the (fourth) system shown in FIG. 16.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram of the most basic system according to an embodiment of an image processing apparatus (MFP, Multi-Functional Peripheral) to which the present invention can be applied.

  An image processing apparatus (hereinafter referred to as a system) 1 shown in FIG. 1 includes a plurality of devices connected to each other via a system bus 11, such as a CPU (main control device) 21, a shared memory 23, an HDD (Hard Disc Drive, Large-capacity storage device) 25, and n (n is a positive integer) PEs (arithmetic units) PE1 to PEn.

  The system bus 11 is used for data input / output with the outside and internal data communication.

  The shared memory 23 can be written and read from each processor PE1 to PEn coupled in the system.

  Next, a widely used JPEG (Joint Photographic Experts Group) compression method applicable to the system 1 shown in FIG. 1 and the compression method of the present proposal (this application) capable of increasing the speed thereof will be described.

  FIG. 2 briefly explains the encoding method of JPEG compression.

  In JPEG compression, image data is compressed for each group of pixels, which is a minimum unit of compression called Mcu (Minimum code unit). In the example of FIG. 2, it is assumed that image compression of 16 pixels in height (direction) and 16 pixels in width (direction) is compressed in units of Mcu having a height of 8 pixels and a width of 8 pixels.

  The compressed image data is processed in units of Mcu lines.

  The processing is performed by color-converting 64 pixels in the Mcu block from RGB color input image data to YCC luminance color difference color data, DCT processing for each YCC component, and converting the frequency component into frequency components. Are quantized using a quantization table, and the last quantized value is encoded by Huffman coding.

  In this process, when storing the code of the Huffman coding, regarding the storage of the DC component signal in the Mcu block generated after the DCT conversion, the value is obtained as a difference value from the DC component when the previous Mcu is compressed. Is stored. In this case, since the DC component of the image is relatively similar between adjacent Mcus, there is an effect that the value of the difference value is reduced, the stored data length is shortened, and the compression rate is increased.

  However, a dependency relationship is generated between the Mcu compression codes by this method, and the Mcu code data of the 16 × 16 image is adjacent to the unique dependency relationship (“Mcu (0, 0)”) as illustrated in FIG. Mcu (0, 1) ”is located, and“ Mcu (1, 0) ”is located on the opposite side of“ Mcu (0, 0) ”of“ Mcu (0, 1) ”.

  When decompressing image data compressed in this way, Huffman decoding, DC component calculation from adjacent Mcu DC components, inverse quantization, inverse processing are performed in order from the code data corresponding to the first Mcu block of the compressed image. DCT conversion and color conversion from YCC color data to RGB are sequentially performed.

  FIG. 3 shows a hardware circuit that has a proven record in the processing of the above-mentioned JPEG compression method that is widely used, or a part or many elements of the data transfer format inside the apparatus, and compression that enables higher speed. The system and its decompression processing of compressed data are realized. Among the n processors PE1 to PEn shown in FIG. 1, compression and decompression are processed in parallel by two processors PE1 and PE2. An example of processing that can further improve the processing speed will be described.

  The processors PE1 and PE2 divide the image data at regular intervals in the vertical direction when the image direction in which the addresses are continuous when expanded in the image data memory is the horizontal direction and the vertical direction is the vertical direction. Thus, the divided area is allocated to one of the processors and compressed.

  Each of the processors PE1 and PE2 is different from the above-described JPEG during compression, and when the difference of the Mcu DC component is closed to the allocated area and reaches the end of the area, the next Mcu row in the area is Operate as the base for the calculation of the first element.

  Thereby, there is no data dependency between the divided areas.

  When each of the processors PE1 and PE2 compresses the allocated area as described above, the compressed data length varies depending on the image data to be compressed. Therefore, the processors operating at the same speed also have local memory (referred to as a local buffer). However, the speed at which data is stored differs.

  In the present invention, each processor in charge of the area performs writing when a certain amount of compressed data is stored in the local memory, so that it can be implemented even if the local memory to be used is small. Although this is a fixed amount for writing, a value optimized for the media written by the system is preferable.

  As apparent from FIG. 3, when writing compressed data to the HDD 25 of the system 1, the size when the system 1 writes data to the HDD 25, for example, 2 kbytes ([kb]) when the system 1 writes data to the HDD 25. If data is to be written one by one, the amount of compressed data stored in the local buffer is 2 kbytes ([kb]).

  When the compressed data stored in the local buffer is written, data that can determine which processor performed the writing is similarly written. This data is stored in a different area from the code data, and held until the entire image is compressed.

  After all the images in the allocated area are compressed by the processor, the remaining compressed data is written as described above to complete the processing.

  When the compression processing for all areas is completed, the accumulated writing PE data, the compressed image parameters (width, height, color / monochrome), and the offset at which the writing PE data is written are written out.

  As a result, a compressed file containing data compressed by dividing the image vertically into strips is completed.

  Next, the decompression process of the compressed image file will be described.

  In the decompression process, each processor reads the offset at which the write PE data is written from the end of the file, and then reads the write PE data. Each processor refers to the write PE data, reads the encoded data written by the processor, sequentially performs Huffman decoding, calculates the DC component of the own Mcu from the DC component of the adjacent Mcu, inverse quantization, inverse DCT, RGB from the YCC Perform color conversion processing.

  In the process, when the compressed data is insufficient, the write PE data is referred to and the next compressed data written by the self PE is read.

  The decoded image data is written in a memory area (shared memory) 23 accessible from each processor.

More specifically, as shown,
Compression <1> → Each PE compresses the image vertically
PE1 CodeMcu (0,0) CodeMcu (1,0)
PE2 CodeMcu (0,1) CodeMcu (1,1)
Compression <2> → Each PE writes code data each time it compresses by A [kb]
Thus, compression is performed.

On the other hand,
Decompress <1> Load data in A [kb] according to the PE data written
Decompression <2> Decoding processing and decoding image writing
PE1 Write PE data
PE2 Write PE data
It becomes.

  FIG. 4 is a flowchart of overall processing according to the embodiment of the present invention.

  The system receives an image input process or an output process from the outside [ACT01].

  When the input process is accepted [ACT01-YES], information regarding the input image file is written in the header portion of the compressed file [ACT02].

  Next, the image area is divided in the vertical direction, assigned to each processor, and the area information and the start of the compression process are instructed [ACT03].

  Thereafter, each processor performs the processing shown in the flowchart described below with reference to FIG. 5, and compresses the allocated area [ACT04].

  The processor that performs compression writes the writing PE data together with the compression code [ACT05].

  After the processing of all the processors is completed, this data is written following the compression code. Finally, the offset information in which the writing PE data is written is written, and the compression processing is completed [ACT06].

  When the output process is accepted [ACT01-NO], the decoding process is instructed to each processor [ACT07]. Thereafter, the processor instructed to perform the decoding process performs the process of the flowchart described later with reference to FIG. 6 to decompress the image (data).

That is,
Export image header information to a file
Assign vertical division area to each PE and instruct compression start
Wait for compression to finish on each processor
Write PE data
Write PE data offset
Instruct each PE to start decompression
Are executed in order.

  FIG. 5 is a flowchart regarding processing in each processor at the time of compression described with reference to FIG.

  Each processor acquires information indicating an area of an image to be compressed by the processor. The area is specified in the form of the x-th column to the y-th column [ACT11].

  The image data to be processed is copied from the image data on the shared memory to the local memory and compressed. The compression process starts from the 8 × 8 region at the left end of the region (Mcu (0, 0) in FIG. 2), and is processed in the horizontal direction of the image, and the end of the region (Mcu (0, 1) in FIG. 2). Is reached in the order of proceeding again from the position where the height is shifted downward by the height 8 of Mcu (Mcu (1, 0) in FIG. 2) at the left end of the image. In the process, the compressed code data is temporarily stored in the local memory.

  Each time a certain amount (A [kb]) of Mcu data is compressed, the code amount written to the local memory is checked, and if it exceeds a certain amount, the code data is written. At this time, the data is written exclusively by using a semaphore for writing management so that the data is not written in the same area as other processors [ACT12] to [ACT13-YES] to [ACT16-YES]. [ACT17] to [ACT18-NO].

  When the compression processing of the image data to be compressed is completed [ACT14-YES], the code amount written in the code buffer is checked, and when the predetermined amount is not enough, invalid data is written to the shortage. Then, the data is written [ACT15].

That is,
Get image sequence information to process
Compression processing
Compressed data is A [kb] or more?
Compression complete
Pack invalid data up to A [kb]
Is data writable?
A [kb] compressed data, writing PE data writing
Compression complete
This routine is executed.

  FIG. 6 shows a flowchart relating to processing in each processor when the code data compressed by the processing in each processor described with reference to FIG. 4 is decompressed.

  In the decompression process, each processor reads image information written in the compressed data. This data includes information related to the image (width, height, color / monochrome) and information related to compression (which processor compresses which area) [ACT21].

  Next, the offset information of the writing PE data written at the end of the data and the writing PE data loaded based on the offset information are loaded [ACT 22]. The processor reads from the image information and the written PE data which area of the image is to be decompressed, and decodes the image while loading the compressed data to be processed based on the written PE data [ACT23], [ACT24].

  In the decoding process, the image is restored in units of Mcu, and each processor restores the image to the local memory for a certain number of lines [ACT25], [ACT26], and then writes the image data to the shared memory [ACT27] ].

  When the processing of all the processors is completed, the image is developed on the shared memory, and the processing is completed [ACT28-YES].

In other words,
Get image information
Write PE data read
A [kb] read compressed data
Signal processing
No compressed data?
Export N lines
Decryption complete?
Processing is performed in this order.

  As described above, with respect to compression of the input image, the image developed on the image memory is divided in the sub-scanning direction, and each area is assigned to the processor and encoded in parallel. Compared to the conventional method, high-speed compression processing can be realized. In addition, the compressed data can be written out immediately after the code data compressed by each processor has accumulated to a size suitable for storage, and there is little overhead and high speed can be realized.

  FIG. 7 is a block diagram of a system for realizing another image compression method applicable to an image processing apparatus (MFP) to which the present invention is applicable. In the system shown in FIG. 7, in the image processing described with reference to FIGS. 1 to 6, the data size when each processor writes the compressed image data is n times the cache memory line length (n ≧ 1). It is characterized by being.

  7, a system 101 includes a plurality of (n (n is a positive integer)) processor cores PE1, PE2,..., PEn-1, PEn each having a primary cache memory, and processor cores PE1 to PE1. Secondary cache memory 111a,..., 111n (n is a positive integer) that is directly connected to PEn and performs data reading and writing, a memory controller 103 that controls data writing and reading to each memory, and system 101, shared memory 105 accessed from all processor cores PE1 to PEn existing in 101 via primary and secondary cache memories 111a to 111n, image data, encoded data that is compressed image data, and parameters relating to compression processing HDD that stores programs executed by the processor core 07, and the secondary cache memory, consisting of a system bus 109 that realizes transmission and reception of data between HDD, and shared memory. Note that the secondary cache memories 111a to 111n are connected to the system bus 109 in units of two processor cores.

  The image data processed in the present embodiment and the encoded data obtained by compressing the image data are stored in the shared memory 105.

  In the primary cache memory, the secondary cache memory, and the shared memory, the access speed from the CPU decreases in this order, and the data capacity that can be stored is large.

  The cache memory stores data using fixed-length data as a line, and performs reading and writing using this line data as a minimum unit. When rewriting the data for 1 byte, it reads together with the data existing on the same line as the 1-byte data to be accessed, and the necessary byte data on this line is rewritten on the register in the processor core. Then, writing is performed again with the data on the same line.

  In a system composed of a plurality of processor cores, the memory controller 103 performs a process called coherent processing in order to maintain consistency so that data held in the memory is held as described in the program.

  FIG. 8A shows an example of a memory operation between the primary and secondary cache memories using the system shown in FIG.

  For example, assuming that the processor core 1 and the processor core 2 are the first and second processor cores PE1 and PE2 from the left in FIG. 7, the addresses of the processor core 1 (PE1) and the processor core 2 (PE2) are Assume that adjacent data has been read. At this time, it is assumed that two adjacent data exist on the same line on the secondary cache memory in terms of address.

  The data on the same line is copied as the same data on the primary cache memory in the processor core 1 and the processor core 2.

  Thereafter, when the processor core 1 rewrites this data, a difference occurs between the data held by the processor core 1 and the processor core 2 as the primary cache memory. There will be old data that does not reflect the changes made by.

  Therefore, the memory controller 103 checks the access to the memory from each processor core and rewrites the data area inconsistently when the data on the memory is rewritten as described above. I do. That is, when the processor core 1 (PE1) rewrites the primary cache memory, the memory controller 103 detects that the data copied in the processor core 2 (PE2) is not consistent, and the processor core The data of the corresponding cache line in the primary cache memory in 1 (PE1) is overwritten on the data in the primary cache of the processor core 2 (PE2).

  As described above, coherent processing for maintaining the consistency of the cache memory is performed in units of data called lines.

  However, in practice, when the processor cores 1 and 2 perform processing that does not share data, if these data exist in the same cache line, the above-described coherent processing is performed (that is, ). Although there is no logical problem, in the coherent process, a data copy process between different memories is performed, and during this coherent process, the processor core cannot access this area.

  For this reason, execution of coherent processing for processing that does not share data as described above has a large overhead, and is logically a copy of unnecessary data, and is called false sharing.

  FIG. 8B shows another example of the memory operation between the primary and secondary cache memories using the system shown in FIG. That is, FIG. 8B shows the size when writing the compressed data in the secondary cache memory in order to avoid a decrease in the execution efficiency of each processor core in relation to the false sharing that occurs in the embodiment described with reference to FIG. 8A. It is characterized by N times the cache line size.

  In FIG. 8B, by taking the size of the encoded data writing described with reference to FIG. 8A as an integral multiple of the line size of the cache memory, the areas used by the plurality of cores sharing the memory on the shared memory are respectively , No more crossing the cash line. This eliminates the need to read data used by other processors when each other accesses the memory. Therefore, the coherent operation performed by the memory controller 103 does not occur, no unnecessary overhead occurs, and the processing can be realized at high speed.

  More specifically, in FIG. 8B, when the processor core 1 (PE1) and the processor core 2 (PE2) execute the compression processing of the allocated area in parallel, the processor core 1 When the compression process according to the above advances and the encoded data reaches the cache line size, the encoded data is written into the memory by the cache line size. As a result, the encoded data is written to the secondary cache memory.

  If this write size does not match (does not match) the cache line size, as described above, coherent processing occurs for other CPU cores that describe encoded data, and performance, particularly processing speed, is greatly increased. 8B, in this method shown in FIG. 8B, the memory write size is limited to an integral multiple of the cache line size, so that coherent operation does not occur, and overhead associated with synchronization processing between processors can be reduced. it can.

  FIG. 9 is a block diagram of a system for realizing still another image compression method applicable to an image processing apparatus (MFP) to which the present invention is applicable.

  The embodiment shown in FIG. 9 switches the number of parallel divisions of image data in accordance with the number of processors when either or both of the apparatuses that perform compression processing and decompression processing have multi-core processors. Thus, a high throughput is realized.

  The system 201 shown in FIG. 9 includes a client PC (data supply source) 221 and a printer engine (including an MFP) 231 connected to each other via a network (for example, a LAN) to the client PC 221 and the printer engine 231. Each of the printer controllers 233 for inputting image data for image output includes a multi-core CPU responsible for image compression and expansion processing.

  In this system, image data in a document printed by a user is compressed by a multi-core on the client PC side.

  Data in a state where the image in the document is compressed is transmitted to the printer controller 233 connected to the printer engine 231 via the network.

  After receiving the print data, the printer controller 233 performs image forming processing on this data. At this time, the compressed image data in the received print data is decompressed by its own multi-core.

  In this way, by compressing and decompressing image data contained in a document in a multi-core manner, parallel processing is performed, and the data size of the print data amount is reduced to shorten the communication time between the client PC and the controller. The printing throughput can be improved.

  A method for determining a division size for parallel processing of image data compression according to the present embodiment will be described.

  FIG. 10 shows an example of a method for determining parallel partitioning when there is a difference in the number of CPU (processor) cores of the client PC and the printer controller.

  Here, it is assumed that the CPU core number of the client PC is 2 and the CPU core number of the printer controller is 4. As the number of parallel divisions in the present embodiment, the number of CPU cores that perform compression processing and decompression processing is compared and a large number is used.

  In the case of FIG. 10, since the compression side has 2 cores and the decompression side has 4 cores, the number of image area divisions for parallelization is 4. When the number of divisions is determined in this way, if the number of cores is different between the compression processing side and the decompression processing side, either side divides the image more than the actual number of CPU cores.

  11 and 12 show processing examples when the number of processor cores of the client PC shown in FIG. 10 is different from that of the printer controller when the number of processor cores of the printer controller is different.

  When the number of CPU cores is small, a plurality of image division regions are assigned as processing targets to one processor core.

  FIG. 11 shows a case in which the number of CPU cores is set to 2 and the number of parallel divisions is set to 3. In this case, for CPU core 1, areas A and C, and for CPU core 2, , Area B is allocated.

  In this case, if the method of the first embodiment of the present invention is used for this compression as it is, the processing flow as shown in FIG. That is, first, the processing of the areas A and B to which the cores are allocated is performed, and the encoded data of these areas is written to the compressed data as needed when the write amount is reached. When the writing is completed, the CPU core 1 performs the compression process of the area C.

  In this way, when the compression process is performed, the encoded data is “area A code data” and “area B”, which are the compressed data of the areas to which the CPU core is allocated first, as shown in FIG. For the set of “code data”, “region C code data”, which is compressed data in the other region, is stored at a distant position in the data.

  When such encoded data is decoded by the apparatus of the present invention, since the data in the adjacent area is stored at a distant position on the file, both data cannot be stored simultaneously in the memory and the cache memory. As a result, a disk swap occurs and the performance is extremely lowered.

  On the other hand, in the method of the present embodiment, as shown in FIG. 12A, a threshold for the number of processing lines is determined in advance for the region processing, the number of lines exceeds a certain value, and the code amount is It is assumed that when the amount of writing has been reached, that is, when the amount of encoded data has exceeded the band line, writing, processing of that area is interrupted, and processing for another allocated area is started.

  As a result, code data belonging to the same line can be stored close to each other in the compressed data as schematically shown in FIG. 12B, so that memory access can be performed efficiently. , Processing can be performed at high speed.

  In the system shown in FIG. 9, the division number determination method described above with reference to FIG. 10 and the processing method described with reference to FIG. 12 are used, and the processing flows on the compression side and decompression side shown in FIGS. The processing is realized by the above.

  FIG. 13 is a flowchart for explaining the “process with parallel division” shown in FIG.

  In FIG. 13, first, “parallel division number determination processing” described later with reference to FIG. 14 is executed, and the division number for executing “processing with parallel division” is set [ACT 31].

  When a page to be printed (image output) is described, it is checked whether or not the description target includes a compressed image [ACT33].

  When the description target includes a compressed image [ACT33-YES], the processing target area is indicated [ACT34], and the encoded data is written out after the encoding process [ACT35].

  Subsequently, the writing PE data is written out [ACT36], and until there is no remaining page (print target) to be described [ACT32-YES], whether or not the description target includes a compressed image ([ACT33]), When the description target includes a compressed image, the process target area instruction ([ACT 34]) and the encoded data writing ([ACT 35]) are repeated after the encoding process.

  When the description object does not include a compressed image [ACT33-NO], the description for the next description object is repeated [ACT37]. On the other hand, when there is no remaining page (print target) to be described, the process ends [ACT32-YES].

That is,
Parallel separation number determination processing
Print all pages complete <end of description>?
Is the description target compressed image?
Processing area instruction
Encoding processing, encoded data writing
Write PE data export
Description processing
These steps are performed.

  FIG. 14 is a flowchart for explaining the “parallel division number determination process”.

  In the “parallel division number determination process” shown in FIG. 14, the number of printer controller CPU cores is first checked (printer controller CPU core number acquisition) [ACT41].

  Then, the number of client (PC) CPU cores is checked (client CPU core number acquisition) [ACT42].

  Thereafter, the acquired number of printer controller CPU cores and the number of client CPU cores are compared [ACT 43], and the division number is set according to the larger number of CPU cores. That is, when the number of printer controller CPU cores is large [ACT43-YES], the division number is set to the number of printer controller CPU cores [ACT44]. On the other hand, when the number of client (PC) CPU cores is large [ACT43-NO], the number of divisions is set to the number of client (PC) CPU cores [ACT45].

That is,
Get the number of CPU cores of the printer controller
Client <PC> Get CPU core count
<CPU core number comparison>
<Number of controller CPU cores [Large]>
Number of divisions = number of controller CPU cores
<PC <client> Number of CPU cores [Large]>
Number of divisions = number of PCCPU cores
Are executed in order.

  FIG. 15 shows an example of processing on the printer controller side corresponding to the processing in the client PC (CPU core) described with reference to FIG.

  As is apparent from FIG. 15, it is checked whether or not the drawing target includes a compressed image [ACT52]. If the target includes a compressed image [ACT52-YES], the number of parallel divisions is acquired [ACT53], and processing is performed. The target area is indicated [ACT 54].

  Thereafter, the decoding process is performed [ACT 55], and until there is no remaining page (print target) to be rendered [ACT 51-YES], whether or not the rendering target includes a compressed image ([ACT 53]), and the rendering target is determined. The process area indication ([ACT54]) and the decoding process ([ACT55]) are repeated when a compressed image is included.

  When the drawing target does not include a compressed image [ACT53-NO], drawing for the next drawing target is repeated [ACT56]. If there is no remaining page (print target) to be drawn, the process ends [ACT51-YES].

  FIG. 16 is a block diagram of a system for realizing still another image compression method applicable to an image processing apparatus (MFP) to which the present invention is applicable. Note that the example shown in FIG. 16 is substantially equivalent to the state in which the two processor cores PE1, PE2, the secondary cache memory 111A, and the shared memory 105 in the system shown in FIG. 7 are extracted.

  In the system shown in FIG. 16, in order to optimize the utilization efficiency of the shared memory 105, the number of parallel divisions is determined and processing is performed, so that the bottleneck that occurs in the memory is eliminated and processing is performed at high speed. Is to realize. In FIG. 16, the processing parameters are not swapped out from the primary cache memory of each core, and the read / write data amount related to the transfer between the secondary cache of the processing data and the shared memory is optimal for the transfer. Since the transmission is performed with a small size, the overhead is small, and the DMA transfer can be executed under the optimum conditions, the throughput can be improved. FIG. 17 shows an example of a data structure arranged on the shared memory by one of the cores realized by the embodiment shown in FIG.

  In the example shown in FIG. 16, in the state where the memory (shared memory 105) on the system used by each of the core processors PE1 and PE2 is connected to the system bus 109, each of the memories of the core processors PE1 and PE2 ( The data stored in the primary cache memory) is JPEG data. On the memory (shared memory 105), compression / decompression parameters, specifically, quantization tables, Huffman tables, color differences (U and V components) ) Is stored. In the data on the secondary cache memory 111a located between the processors PE1 and PE2 and the shared memory 105, parameters are attached to the head (header) of each processing data. It should be noted that a parameter is attached to the head of the work area in the individual data on the primary cache memory in each of the processors PE1 and PE2.

  Assume that these parameters use predetermined values. Further, image data to be processed and encoded data obtained by compressing the image data are also stored in a memory, and these are stored in an area reserved for work.

  Since the size of the data to be compressed is determined depending on the compression parameter and the content of the data to be compressed, it cannot be determined exactly how much is necessary. Normally, if this area size is made too small, the number of times of data transfer between the shared memory and the cache memory increases, and the performance deteriorates due to overhead due to the transfer. Further, if the size of the work area is too large, the data in the primary cache overflows and a swap operation to the secondary cache occurs, which also causes a decrease in performance. Here, as shown in FIG. 17, by optimizing the DMA minimum transfer size that an arbitrary PE defines on the shared memory, the overhead of DMA transfer is optimally suppressed in the compression of an average image. High overhead can be realized with respect to the intended use.

  FIG. 17 shows an example of the configuration of the minimum DMA transfer size defined by any PE on the shared memory.

  As apparent from FIG. 17, the DMA minimum transfer size defined by any PE on the shared memory includes processing data including parameters, input data, and output data.

  As described above, as shown in FIGS. 16 and 17, the data held in the shared memory is obtained by optimally suppressing the swap processing between the memories and the overhead of the DMA transfer as described above in the average image compression. Therefore, it is possible to realize a high overhead for many usages.

  Although this embodiment will be described below with reference to FIG. 18, a sample chart or the like is compressed in advance with the same value as the parameter adopted in the method, an average compression rate is calculated, and the work memory size is calculated based on this. By predicting, the number of parallel divisions is adjusted so that memory swap and DMA overhead do not occur in an average usage scene.

  FIG. 18 shows an example for explaining a method of determining the number of parallel divisions in the system shown in FIG.

  In processing, the system acquires information on a group of processors sharing the cache memory. For example, in the case of a processor having four CPU cores, each having two cores sharing a secondary cache, the number of groups is two and the number of processor cores each group has is two. . At this time, the size information of the shared cache memory shared by each core is also acquired [ACT101].

  The processing for calculating the number of processor cores is performed for each group. First, confirmation processing is performed to determine whether the processing for all the processor groups has been completed, that is, whether the processing has been looped several times [ACT102].

  Next, the number of CPU cores belonging to the shared group is set as the division number of the group. That is, “number of divisions in group ← number of processors in shared group” is set [ACT 103].

  Subsequently, the shared cache size of this group is divided by this division number, and the size of the area on the shared cache allocated to one core is calculated [ACT 104]. At this time, information on the data size of the processing parameter is acquired and subtracted from the cache size assigned to one processor core, thereby calculating the memory size that can be used as a work area [ACT 105].

  With respect to the work size calculated here, an average compression rate is acquired [ACT 106], and this is multiplied by the work size to calculate the size of the image and the prediction used as the coding area. That is, the fixed processing parameter size is calculated [ACT 107].

  In the work area, since the area of the encoded data is smaller, this is set as the minimum area size of the work memory [ACT 108].

  Here, the minimum area size of the work memory is compared with the minimum transfer size of the DMA transfer between the secondary cache and the shared memory [ACT109], and the minimum transfer of the DMA transfer between the shared memories is larger than the minimum area size of the work memory. When the size is small [ACT109-YES], the number of group divisions is reduced by 1 [ACT110], and the minimum area size of the work memory is calculated again and compared [ACT111-YES] to [ACT104],. [ACT109].

  When the work size is larger than the minimum DMA size [ACT109-NO], the division number of this processor is added to the division number of the parallel processing, and the processing shifts to the division number determination processing of the next processor group [ACT112].

That is,
Cache shared processor group information acquisition
Have you processed the number of processor supply groups (has looped several times)?
Number of partitions in group ← Number of processors in shared group
Calculate the number of cache shared processors
Average compression rate data acquisition
Fixed processing parameter size calculation
Work area size calculation
Calculation of minimum data area size in work area
Minimum data area size> DMA minimum transfer size?
Number of divisions within group ← Number of divisions within group-1
Number of divisions within group ≠ 1?
Number of partitions ← Number of partitions + Number of partitions in group
These steps are performed.

  The processing method for determining the number of parallel divisions for other processor groups is also performed in the same manner as described above.

  As described above, according to the present invention, in a system for storing a huge image with high resolution, such as inside a copying machine (MFP), in recent years, when image data is compressed and stored in order to save a storage area. As the printer engine speeds up and the speed of computers such as external devices increases, image data can be compressed and expanded at higher speed. That is, a multi-processor system having a plurality of CPUs and a many-core processor having a plurality of CPUs in one CPU are remarkably widespread, and using such a processor, a method for compression / acceleration of the huge image, The system can be applied especially for JPEG compression with a high compression rate. In other words, by applying the embodiment of the present invention, it is possible to realize a system capable of performing high-speed compression / decompression using a plurality of processors and a single image based on the JPEG method.

  In the image processing apparatus (MFP) of the present invention, when compressing, the image data expanded in the memory is divided in the vertical direction, and each is subjected to JPEG compression processing in parallel by another processor. Each processor has a local memory, and whenever a certain amount of compressed data is stored, a certain amount of code data is written asynchronously, and information indicating that the processor itself has written is recorded on the memory. After the processing of all the processors is completed, this writing order data is appended to the compressed image data to form a compressed image file. At the time of decoding, each processor decodes the image data onto the memory while reading the compressed data by a certain amount with the writing order data below. As described above, according to the present invention, since the dependency relationship of the compressed data is closed to the area to be processed by each processor, it can be processed without synchronizing with other processors. The corresponding performance is obtained.

  Therefore, large-scale image data can be compressed and decompressed at high speed by parallel processing in a many-core or multiprocessor system.

  That is, by applying the image compression / decompression method of the present invention, in a system equipped with a multi-core CPU, when performing compression / decompression in parallel, the execution rate of processing executed by each core is high, and synchronous processing related to memory access is performed. The overhead can be kept low.

  Even when compression / decompression processing is performed on another system, the execution rate of the system with the higher degree of parallelism can be increased in each system that performs compression / decompression. Total throughput can be improved.

  In addition, the processing speed is reduced by suppressing the number of swap operations between the secondary cache memory that is commonly accessed from a plurality of cores, the primary cache memory in the core, and the shared secondary cache memory. Can be suppressed.

  In addition, it is possible to prevent the processing speed from being lowered by performing data transfer of DMA transfer between the secondary cache memory and the shared memory with an appropriate size.

  Therefore, the compression / decompression process by the multi-core CPU can be processed at high speed without impairing the image quality.

  In addition, this invention is not limited to each embodiment mentioned above, A various deformation | transformation or change is possible in the range which does not deviate from the summary in the stage of the implementation. In addition, the embodiments may be implemented by appropriately combining them as much as possible, or by deleting a part thereof. In that case, various effects resulting from the combination or deletion can be obtained.

For example, the image data is developed in the memory and the image is divided in the vertical direction toward the direction in which the addresses are continuous, and each of the divided areas is subjected to color conversion, DCT conversion, quantization, When Huffman coding is performed and each Huffman coded data exceeds a certain amount, the coded data written by each processor by a certain amount and each processor at the time of creating the coded data Write PE data that can identify the processor written when writing,
This can be realized as an image compression format in which writing PE data continues after the encoded data.

And multiple processors with local memory,
Shared memory accessible to each processor;
A memory control unit for storing a part of the input image or the entire image in a memory;
A part or the whole of the stored input image is divided into image areas along the vertical direction in the direction in which the image data is expanded in memory and the addresses are continuous, and the image data of each divided area is When color conversion, DCT, quantization, and Huffman code processing are performed in a form assigned to any one of the processors, and the Huffman code data generated as a result of processing exceeds a certain amount, a certain amount of Huffman code data Is written to the memory, and when the compression processing of all the processors is completed, the write PE data that can identify the processor is written, and the write PE is written out of the compressed data of the format. Huffman decoding and inverse quantization while reading only the data corresponding to the processor that is reading the data An image processing unit which operates by the inverse DCT, the decompression program processes the compressed image subjected to the inverse color conversion processing,
Can be realized.

  DESCRIPTION OF SYMBOLS 1,101 ... Image processing apparatus (image processing system), 11 ... System bus, 21 ... CPU (main control apparatus), 23 ... Shared memory, 25 ... HDD (image data storage device), 103 ... Memory controller, 105 ... Shared Memory 107 107 HDD (image data storage device) 109 System bus 111a to 111n Secondary cache memory PE1 PEn Arithmetic unit (processor core, core processor, unit processing device)

JP 05-143552 A JP-A-11-170657 JP 2003-348355 A

Claims (7)

In an image processing method in which image data of a plurality of regions divided in a matrix in two directions orthogonal to each other in the JPEG method is independently compressed in parallel by two or more processors.
The individual image data includes an information for specifying a processor when the image data of the area is compressed. In the JPEG method, the compressed image data is expanded for each of a plurality of regions divided in a matrix in two directions orthogonal to each other, and is returned to the position before the division based on information at the time of compression attached to the image data. In an image processing method in which decompression is performed in parallel by two or more processors,
An image processing method characterized in that information at the time of compression attached to individual image data includes information that can identify a processor at the time of compression. Multiple processors with local memory,
A shared memory accessible to each of the processors;
A storage control unit that stores a part of the input image or the entire image in a memory together with unique information of the processor;
A part or the whole of the stored input image is expanded in memory to divide the image area along the vertical direction in the direction in which the addresses are continuous, and the image data of each divided area is Assign to any one, perform color conversion, DCT, quantization, Huffman code processing, and if the Huffman code data generated as a result of processing exceeds a certain amount, write a certain amount of Huffman code data Write the data that can identify the issued processor to the memory, and when the compression processing by all the processors is completed, write the processor information at the time of writing that can identify the processor, read the processor information at the time of writing from the compressed data, process Huffman decoding, inverse quantization, inverse DCT while sequentially reading only the data corresponding to the processor performing An image processing unit for decompression the compressed image subjected to the inverse color conversion processing,
An image processing apparatus comprising:   4. The image processing apparatus according to claim 3, wherein the image data on the input buffer is divided in the vertical direction and each is processed in parallel by different processors.   4. The image processing apparatus according to claim 3, wherein, when the compression processing is performed in parallel, when a certain amount of compressed data is accumulated in a local memory of the processor, the data is written without being synchronized with another processor.   4. The image processing apparatus according to claim 3, wherein at the time of the compression, each processor writes data that can identify that it has written data when the processor writes code data.   4. The image processing apparatus according to claim 3, wherein when decoding the image data, a plurality of processors can perform parallel image restoration while sequentially reading the data based on the processor information at the time of compression written in the input data.

Images