JP4907488B2 - Image processing apparatus, image processing method, and computer-readable recording medium storing program for executing the method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and a computer-readable recording medium storing a program for executing the method, and more particularly to a method for compressing / decompressing image data, and is particularly included in each of a plurality of threads. The present invention relates to a technique for executing image processing using a processor capable of executing instructions in parallel.

  In recent years, with the progress of computer technology, various hardware resources such as a high-speed processor and a large-capacity memory have been developed. One of them is a processor in which one processor can logically act as a plurality of processors. This processor can execute instructions included in each of a plurality of threads in parallel. By performing image processing with such a processor, speeding up of image processing is expected. Some proposals have been made on such image processing methods.

As one example, Patent Document 1 proposes an image processing method and an image processing program that divide and execute a plurality of threads for executing image processing into groups according to differences in processing methods used for executing the threads. Yes. Patent Document 2 proposes an image processing method, a printing apparatus, an image processing system, and a recording medium in which the number of processors assigned to halftoning is changed and the assigned processors are changed by resolution.
JP 2005-259042 A Patent No. 3,797,013

  However, the image processing methods disclosed in Patent Documents 1 and 2 include a code that can be executed in parallel and a portion that cannot be executed in parallel, such as JPEG2000, and changes depending on the code to be processed. , Suitable division is not possible, and the speed cannot be increased. In other words, the decoding process of codes composed of codes that can be decoded independently for each region represented by the JPEG2000 image processing method, etc., includes a part that can be processed in parallel and a part that cannot be processed in parallel. It was complicated and speeding up was difficult.

  The present invention is intended to solve these problems, and an image processing apparatus, an image processing method, and the method capable of executing image processing capable of executing instructions included in a plurality of threads in parallel at high speed An object of the present invention is to provide a computer-readable recording medium storing a program for executing the above.

In order to solve the above problem, the image processing apparatus of the present invention that divides image data into regions and decodes code data that has been subjected to compression processing that can be independently decoded for each divided region. Header processing means for decoding information, code division means for dividing the code into independently decodable codes, decoding processing means for decoding received code data, and activation means for providing the divided code data to the decoding processing means Image data acquisition means for receiving the decoding result of the decoding processing means, thread number acquisition means for acquiring the number of threads that can be executed simultaneously, and execution thread number acquisition means for acquiring the number of execution threads of the decoding processing means ing. According to the image processing apparatus of the present invention , the starting unit controls the starting of the decoding processing unit based on the comparison result between the measurement result obtained by measuring the throughput of the instruction sequence executed by the CPU and the predetermined value. The code data is divided based on the number of executable threads and the number of execution threads of the decoding processing means, and a plurality of decoding processing means are simultaneously executed to generate image data. Therefore, it is possible to execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed.

  In addition, the activation unit can execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed by changing the number of decoding processing units activated during the decoding process.

Further, the image processing apparatus of the present invention that divides the image data into regions and generates encoded data that has been subjected to compression processing that can be independently decoded for each of the divided regions. Header generating means for generating the above, image dividing means for extracting the target image data from the original image for each encoded data that can be independently decoded, encoding processing means for encoding the received image data, An activation unit that provides the divided image data to the encoding processing unit; a code data acquisition unit that receives an encoding result of the encoding processing unit; a thread number acquisition unit that acquires the number of threads that can be executed simultaneously; The processing means includes an execution thread number acquisition means for acquiring the number of execution threads, and a code generation means for combining the plurality of acquired encoded data to constitute the code data. According to the image processing apparatus of the present invention , the starting unit controls the starting of the decoding processing unit based on the comparison result between the measurement result obtained by measuring the throughput of the instruction sequence executed by the CPU and the predetermined value. A feature is that the image data is divided based on the number of executable threads and the number of execution threads of the encoding processing means, and a plurality of encoding processing means are simultaneously executed to generate codes. Therefore, it is possible to execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed.

  Furthermore, the activation means can execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed by changing the number of encoding processing means activated in the middle of the encoding process.

  Furthermore, the compression code is preferably JPEG2000. Moreover, it is preferable that the size of an area | region is a tile. Furthermore, the size of the region is preferably precinct.

According to another image processing method of the present invention, image data is divided into regions, and code data subjected to compression processing that can be independently decoded for each divided region is decoded. The image processing method of the present invention includes a header processing step for decoding image information in code data, a code division step for dividing the code information into codes that can be decoded independently, a decoding processing step for decoding received code data, In the starting process for providing the divided code data to the decoding process, the image data acquiring process for receiving the decoding result by the decoding process, the thread acquiring process for acquiring the number of threads that can be executed simultaneously, and the decoding process An execution thread number acquiring step for acquiring the number of execution threads. Furthermore , according to the image processing method of the present invention , the starting step controls the starting of the decoding processing step based on the comparison result between the measurement result obtained by measuring the throughput of the instruction sequence executed by the CPU and a predetermined value. The code data is divided based on the number of executable threads and the number of execution threads in the decoding process, and a plurality of decoding processes are simultaneously executed to generate image data. Therefore, it is possible to execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed.

Furthermore, according to another image processing method as another invention, image data is divided into regions, and encoded data subjected to compression processing that can be independently decoded for each divided region is generated. The image processing method of the present invention extracts a target image data from the original image for each encoded data that can be decoded independently, and a header generation step for setting the encoding conditions of the image and generating header information. An image division process, an encoding process for encoding the received image data, an activation process for providing the divided image data to the encoding process, and code data for receiving an encoding result by the encoding process An acquisition process, a thread acquisition process that acquires the number of threads that can be executed simultaneously, an execution thread acquisition process that acquires the number of execution threads in the encoding process, and a plurality of acquired encoded data The code generation process which comprises. Furthermore , according to the image processing method of the present invention , the starting step controls the starting of the decoding processing step based on the comparison result between the measurement result obtained by measuring the throughput of the instruction sequence executed by the CPU and a predetermined value. Then, the image data is divided based on the number of executable threads and the number of execution threads in the encoding process, and a plurality of encoding processes are simultaneously executed to generate a code. Therefore, it is possible to execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed.

  As another invention, a computer-readable recording medium storing a program for causing the computer to execute the image processing method is characterized. Therefore, it is possible to construct an image processing system for general use without changing the existing system.

  The image processing apparatus of the present invention divides code data based on the number of executable threads and the number of execution threads of the decoding processing means, and generates image data by simultaneously executing a plurality of decoding processing means. Further, the image data is divided based on the number of executable threads and the number of execution threads in the encoding process, and a plurality of encoding processes are simultaneously executed to generate a code. Therefore, it is possible to execute image processing capable of executing instructions included in a plurality of threads in parallel at high speed.

  FIG. 1 is a block diagram showing the configuration of the JPEG2000 encoder. In the figure, an input image is first subjected to subband decomposition by wavelet transform (hereinafter abbreviated as DWT) 101, and then quantized by a quantization unit 102. As shown in FIG. 2 showing an example of subband decomposition when the decomposition level is 2 (the number of resolution levels is 3), there are resolution levels from level 0 to level 2. At this time, a coefficient belonging to a smaller resolution level includes lower frequency information. The quantized wavelet coefficients are encoded by the EBCOT algorithm. In the code block dividing unit 103, each subband is divided into square blocks (for example, 64 × 64) called code blocks. These code blocks are encoded independently. Next, the coefficient modeling unit 104 performs coefficient modeling based on the pit plane for the wavelet coefficient sequence of each code block. Thus, an embedded code string in which coefficient bits are arranged in order of importance is generated. All bit planes from the MSB to the LSB are decomposed into three sub-bit planes (paths) depending on the context. The boundary of each sub-bit brain is called a truncation point, and becomes the minimum division unit when data is discarded later. The arithmetic coding unit 105 and the rate control unit 106 perform adaptive arithmetic coding on the embedded code string generated by coefficient modeling. Thereafter, the arithmetic bit string is appropriately cut off at the cut-off point that is the boundary of the sub-bit brain to obtain the target bit rate. Next, the layer forming unit 107 performs code layer formation next when it is necessary to sequentially display images with a plurality of image quality, that is, when SNR scalable is required. Each layer includes a part of the embedded code of each code block. The higher the layer, the more important components are included in image reproduction. Then, the packet generation unit 108 decomposes each layer into units called a plurality of bodies, and adds header information to each to generate a packet. Here, each body has information of the corresponding resolution level. Therefore, the total number of packets generated is the product of the number of layers and the number of resolution levels. The header information includes information such as the length of the arithmetic code string of each code block and the number of sub-bit brains. As shown in FIG. 3, the final JPEG2000 code string is obtained by collecting all packets and adding global header information. However, JPEG2000 stipulates that the above-described various header information and sub-bit plane, which is the minimum unit of data division, have a size that is an integral multiple of 1 byte.

  FIG. 4 is a block diagram showing the configuration of the image processing apparatus according to the embodiment of the present invention. In the figure, an HDD 202, a RAM (in a PC) 203, and a CPU (in a PC) 204 are connected via a data bus 201, and code data is decoded in the following flow. Code data recorded on the HDD 202 is read onto the RAM 203 in accordance with a command from the CPU 204. The CPU 204 reads the code on the RAM 203 and performs a decoding process. The CPU 204 writes the decrypted data in another area on the RAM 203. The decrypted data is recorded on the HDD 202 in accordance with a command from the CPU 204. The CPU 204 is a CPU capable of executing a plurality of threads in parallel.

  Here, FIGS. 5 to 8 show the relationship between execution instructions and threads in the CPU. Note that instructions to be executed are instruction 0 to instruction 9, and threads that can be executed in parallel by the CPU are THREAD0 to THREAD4.

  FIG. 5 shows an example in which the CPU can execute only one THREAD at the same time. Such a CPU is referred to as a single thread CPU. 6 to 8 show examples of CPUs that can execute four THREADs in parallel. Such a CPU is referred to as a multi-thread CPU. In the case of FIG. 6, the CPU can execute four THREADs in parallel. However, for example, the instruction 1 uses the result of the instruction 0, and therefore the thread cannot be executed in parallel. It will be the same. FIG. 7 shows a case where THREAD can be partially executed in parallel (THREAD5 to 8). A part of multi-thread processing can be executed. In this case, the processing time can be reduced by 30%. FIG. 8 shows a schematic diagram of processing when all THREADs can be executed in parallel. All processing is completed with three instructions, and the processing time can be reduced by 70%.

  FIG. 9 is a flowchart showing the flow of the image processing operation in the first embodiment of the present invention. Here, code data obtained by compressing an image is decoded to generate image data. In the figure, the number N of threads that can be executed in parallel by the PC that performs the decoding process is acquired (step S101). Then, it is determined whether or not the acquired number of threads N> 1 (step S102). When N> 1, multi-thread processing is executed (step S102; YES, step S103). On the other hand, when N ≦ 1, single thread processing is executed (step S102; NO, step S104). An end process is performed (step S105). Note that this termination processing includes program termination processing, output of generated image data, and the like. It is not necessarily executed. The single thread processing is the same operation as a general application program, and generates a sequence of instructions for operating the CPU one by one and causes the CPU to execute it.

Next, the decoding operation of multithread processing will be described with reference to FIG.
First, multithread processing is initialized (step S201). As an initial setting, the number of running threads N1 = 0. Further, the number NT of independently decodable data units (for example, JPEG2000 tiles) of the code data to be decoded is calculated. It is assumed that the number of independent processing units already processed ND = 0. Next, the number NN of unprocessed independent data is calculated from NT and ND (step S202).
NN = NT-ND

Thread division is performed as follows. An unused thread NT is calculated from the number of executing threads ND and the number of executable threads N.
NT = N-ND

  When NT> 0, a part of the code data that can be decoded independently is provided to the decoding processing apparatus, and the decoding process is executed (step S203). Part of the provided code data may indicate the position of the code data, such as an offset from the original code data. It may be extracted from the actual code data. When extracting, header information may be combined and provided so as to be a complete code.

  Here, without considering the fact that it is a part of the entire image, it is assumed that there are multiple independent image data. Care must be taken when generating possible codes. For example, in the JPEG2000 code, when applying a Low or High filter of wavelet transform, the applied filter changes depending on the position in the original image, so the offset information must be accurately reflected and a code must be generated. Similarly, it is necessary to pay attention to code block division and precinct division.

  When all the tiles have been processed, the process ends (step S204). (N1 = 1 and NN = 0)

  In addition, this Embodiment only showed an example and only showed an example of the partial encoding process. When generating data for decoding to be executed by the CPU, in addition to a method for providing a normal code, a method for dividing each independent data into different partial codes and a method for dividing a plurality of independent data into partial codes There are also methods. Also, when generating these codes, the partial codes that are efficiently executed according to the conditions such as the CPU type, number, number of cores, PC memory size, compression code area size, etc. By generating the above, speeding up is realized. In the present embodiment, the method of generating a partial code has been described. However, even when a processing speed is taken into consideration, even in a decoding processing apparatus that can execute a plurality of threads, a decoding process is performed without generating a partial code. If it is determined that the process is faster, normal thread generation means is used. It is also possible to adopt a configuration including a decoding processing device having different performance. In that case, since it is very difficult to generate a partial code to be executed by each decoding processing apparatus, it is effective to generate a suitable partial code using a previous processing result or the like.

  Next, the encoding process of the image processing operation in the present embodiment will be described. Here, the image is compressed to generate code data. In FIG. 9, the number N of threads that can be executed in parallel by the PC that performs the encoding process is acquired (step S101). Then, it is determined whether or not the acquired number of threads N> 1 (step S102). When N> 1, multi-thread processing is executed (step S102; YES, step S103). On the other hand, when N ≦ 1, single thread processing is executed (step S102; NO, step S104). An end process is performed (step S105). This end process is a process of generating code data by combining the encoding results of the encoding apparatuses executed a plurality of times. However, when the data recording position is instructed to each encoding device in advance and the encoding process is executed, the combination process may not be necessary. The single thread processing is the same operation as a general application program, and generates a sequence of instructions for operating the CPU one by one and causes the CPU to execute it.

Next, the sign operation of multithread processing will be described with reference to FIG.
First, multithread processing is initialized (step S201). As an initial setting, the number of running threads N1 = 0. Also, the number NT of data units (for example, JPEG2000 tiles) that can be decoded independently of the image data to be encoded is calculated. It is assumed that the number of independent processing units already processed ND = 0. Next, the number NN of unprocessed independent data is calculated from NT and ND (step S202).
NN = NT-ND

  When NT> 0, a part of the image data is provided to an encoding device that generates independently decodable code data, and the encoding process is executed (step S203). A part of the provided image data may be indicated by a position such as an offset from the original image data. What was actually extracted from the original image data may be generated.

  Here, without considering the fact that it is a part of the entire image, it is assumed that there are multiple independent image data. Care must be taken when generating possible codes. For example, in the JPEG2000 code, when applying a Low or High filter of wavelet transform, the applied filter changes depending on the position in the original image, so the offset information must be accurately reflected and a code must be generated. Similarly, it is necessary to pay attention to code block division and precinct division.

  When all the tiles have been processed, the process ends (step S204). (N1 = 1 and NN = 0)

  In addition, this Embodiment only showed an example and only showed an example of the partial encoding process. When generating data for encoding to be executed by the CPU, in addition to a method for providing a normal code, a method for providing different images for each independent data, and a method for providing a plurality of regions collectively for encoding processing There are also. Also, when generating these images, partial images that are executed efficiently according to conditions such as the type, number of CPUs, the number of cores, the size of the PC memory, the size of the compression code area, etc. By generating the above, speeding up is realized. In the present embodiment, the method of always generating a partial image has been described. However, even in an encoding processing apparatus that can execute a plurality of threads when processing speed is considered, encoding is performed without generating a partial image. If it is determined that the processing is faster, normal thread generation means is used. It is also possible to adopt a configuration including an encoding processing device having different performance. In this case, since it is very difficult to determine the optimal encoding process division method, suitable encoding is effective using the previous processing results and the like.

FIG. 11 is a flowchart showing the flow of multi-thread processing of the image processing operation in the second embodiment of the present invention.
First, multi-thread processing is initialized (step S301). As an initial setting, the number of running threads N1 = 0. Further, the number NT of independently decodable data units (for example, JPEG2000 tiles) of the code data to be decoded is calculated. It is assumed that the number of independent processing units already processed ND = 0. The activation device acquires the activation control method stored in the storage device (step S302). In the storage device, setting through U / I, setting from past results, setting of default value, control method for CPU type, control method for CPU number, control method for CPU core number, control method for each tile size, There are many possible ways to control each precinct size. Next, it implements according to the starting method (step S303). When all the tiles have been processed, the process ends (step S304).

  Next, FIG. 12 is a block diagram showing the configuration of a specific apparatus for starting a program for executing each embodiment of the image processing method of the present invention. That is, the figure shows hardware constructed from a microprocessor or the like that executes software according to the image processing method in the above embodiment. In the figure, the image processing system includes an interface (hereinafter abbreviated as I / F) 301, a CPU 302, a ROM 303, a RAM 304, a display device 305, a hard disk 306, a keyboard 307, and a CD-ROM drive 308. A general-purpose processing apparatus is prepared, and a readable recording medium 309 such as a CD-ROM stores a program for executing the image processing method of the present invention. Further, a control signal is input from an external device via the I / F 301, and an instruction by an operator or a program of the present invention is automatically activated by the keyboard 307. The CPU 302 performs image processing according to the above-described image processing method in accordance with the program, stores the processing result in a storage device such as the RAM 304 or the hard disk 306, and outputs it to the display device 305 or the like as necessary. As described above, by using the recording medium recorded with the program for executing the image processing method of the present invention, it is possible to construct an image processing system for general use without changing the existing system.

  In addition, this invention is not limited to the said embodiment, It cannot be overemphasized that various deformation | transformation and substitution are possible if it is description in a claim.

It is a block diagram which shows the structure of a JPEG2000 encoder. It is a figure which shows the mode of the subband decomposition | disassembly by DWT in JPEG2000. It is a figure which shows the code string of JPEG2000. 1 is a block diagram illustrating a configuration of an image processing apparatus according to an embodiment of the present invention. It is a figure which shows the relationship between a thread | sled and an instruction | indication. It is a figure which shows the relationship between a thread | sled and an instruction | indication. It is a figure which shows the relationship between a thread | sled and an instruction | indication. It is a figure which shows the relationship between a thread | sled and an instruction | indication. It is a flowchart which shows the flow of the image processing operation | movement in the 1st Embodiment of this invention. It is a flowchart which shows the decoding operation | movement of multithread processing. It is a flowchart which shows the flow of the multithread process of the image processing operation | movement in the 2nd Embodiment of this invention. It is a block diagram which shows the structure of the specific apparatus for starting the program which performs each Example of the image processing method of this invention.

Explanation of symbols

  201; bus, 202; HDD, 203; RAM, 204; CPU.

Claims (8)

An image processing device that divides image data into regions and decodes code data subjected to compression processing that can be independently decoded for each divided region,
Header processing means for decoding image information in code data;
Code dividing means for dividing the code into independently decodable codes;
Decoding processing means for decoding received code data;
Starting means for providing the divided code data to the decoding processing means;
Image data acquisition means for receiving a decoding result of the decoding processing means;
Thread acquisition means for acquiring the number of threads that can be executed simultaneously;
Execution thread number acquisition means for acquiring the number of execution threads of the decryption processing means,
The activation means controls the activation of the decoding processing means based on a comparison result between a measurement result obtained by measuring the throughput of an instruction sequence executed by the CPU and a predetermined value,
An image processing apparatus that divides code data based on the number of executable threads and the number of execution threads of the decoding processing unit, and generates image data by simultaneously executing a plurality of decoding processing units . Before Symbol activation means, the image processing apparatus according to claim 1, wherein changing the number of the decoding processing unit to start in the middle of the decoding process. Dividing the images data in the region, an image processing apparatus for generating an encoded data subjected to the decodable compression independently for each divided area,
Header generation means for setting the encoding condition of the image and generating header information;
Image segmentation means for extracting target image data from the original image for each encoded data that can be independently decoded;
Encoding processing means for encoding the received image data;
Starting means for providing the divided image data to the encoding processing means;
Code data acquisition means for receiving an encoding result of the encoding processing means;
Thread acquisition means for acquiring the number of threads that can be executed simultaneously;
Execution thread number acquisition means for acquiring the number of execution threads of the encoding processing means;
A code generating means for combining the plurality of acquired encoded data and configuring the encoded data;
The activation means controls the activation of the decoding processing means based on a comparison result between a measurement result obtained by measuring the throughput of an instruction sequence executed by the CPU and a predetermined value,
An image processing apparatus that divides image data based on the number of executable threads and the number of execution threads of the encoding processing means, and generates a code by simultaneously executing a plurality of the encoding processing means. The image processing apparatus according to claim 3 , wherein the activation unit changes the number of the encoding processing units activated in the middle of the encoding process. The image processing apparatus according to any one of claims 1 to 4, the compression code is characterized in that it is a JPEG2000. Dividing the images data in the region, an image processing method for decoding code data subjected decodable compression independently for each divided area,
A header processing step for decoding image information in the code data;
A code division step of dividing into independently decodable codes;
A decoding process for decoding the received code data;
A starting step of providing the divided code data to the decoding processing step;
An image data acquisition step for receiving a decoding result by the decoding processing step;
Thread number acquisition process for acquiring the number of threads that can be executed simultaneously,
An execution thread number acquiring step for acquiring the number of execution threads in the decryption processing step,
The starting step controls the starting of the decoding processing step based on a comparison result between a measurement result obtained by measuring the throughput of an instruction sequence executed by the CPU and a predetermined value,
An image processing method comprising: dividing code data based on the number of executable threads and the number of execution threads in the decoding process step; and generating image data by simultaneously executing a plurality of the decoding process steps. An image processing method that divides image data into regions and generates encoded data subjected to compression processing that can be independently decoded for each divided region,
A header generation step for setting image encoding conditions and generating header information;
An image dividing step of extracting image data of interest from the original image for each encoded data that can be independently decoded;
An encoding process for encoding the received image data;
A starting step of providing the divided image data to the encoding processing step;
A code data acquisition step for receiving an encoding result by the encoding processing step;
Thread number acquisition process for acquiring the number of threads that can be executed simultaneously,
An execution thread number acquisition step of acquiring the number of execution threads in the encoding processing step;
A code generation step of combining the plurality of acquired encoded data and configuring the encoded data,
The starting step controls the starting of the decoding processing step based on a comparison result between a measurement result obtained by measuring the throughput of an instruction sequence executed by the CPU and a predetermined value,
An image processing method, wherein image data is divided based on the number of executable threads and the number of execution threads in the encoding processing step, and a plurality of the encoding processing steps are simultaneously executed to generate a code. A computer-readable recording medium storing a program for causing the computer to execute the image processing method according to claim 6 or 7 .

Images