KR101736041B1 - Method for Speeded up image processing using distributed processing of embedded CPU and GPU - Google Patents

Abstract

More particularly, the present invention relates to an image processing method and an image processing method using an embedded CPU-GPU distribution process. More particularly, the present invention relates to an image processing method and an image processing method, The present invention relates to an image processing speed-up method using an embedded CPU-GPU distributed processing which improves the efficiency of image processing through distributed processing of a CPU-GPU and multiple algorithm structures of a GPU without loss and duplication of frames through storage and reading.

Description

[0001] The present invention relates to an embedded CPU-GPU distributed processing method,

[0001] The present invention relates to image processing speedup in embedded and mobile systems, and more particularly, to a method and apparatus for processing image data in a CPU (Central Processing Unit) and a GPU (Graphic Processing Unit) Distributed processing, a plurality of buffers located between a CPU and a GPU, and a method of accelerating image processing through a multiple algorithm structure.

With the development of semiconductor technology and the refinement of the process, the performance of the CPU has gradually increased since it first appeared. The way to improve CPU performance was to increase the number of transistors in the CPU. However, this approach limits the CPU clock to 4GHz. As the number of transistors integrated in the CPU increases, the power consumed by the CPU increases, and as the power consumption increases, the heat increases accordingly. Semiconductor, which is the material of the CPU core, The current will flow and the overcurrent will flow and the core will burn. To overcome these limitations, a Multi Core Processor with multiple processors running on the same performance within a single CPU has emerged. The multicore processor is a concept that is applied not only to a CPU but also to a GPU. The GPU, which first appeared in the 1980s to accelerate the computer's graphics processing capabilities, continued to improve performance and now exceeds CPU with its simple computing power. These GPUs are also blocked by performance-enhancing CPUs, and GPUs with multiple independent cores, such as CPUs, have emerged. Several methods for utilizing CPUs and GPUs made of such multi-core processors have been proposed. A typical example is the concept of parallel processing and GPGPU (General Purpose Graphic Processing Unit). Parallel processing is the process of dividing a computer into several tasks and assigning them to each independent processor in the CPU and GPU to process them simultaneously. This method has been used for high-performance computing for a long time. It was a concept that was applied to supercomputer. But recently, interest in heat generation and power consumption has increased in computer use. In addition to the emergence of multi-core processors and a strong paradigm of computer architecture I got attention. GPGPU is a replacement for the CPU, not the graphics processing that the GPU originally intended. There are not many devices for calculation because the CPU has many devices to handle the processing of instructions like non-sequential execution to process signals from various devices of computer. In particular, cache memory, which acts as a buffer to improve the speed difference between memory and CPU, occupies more than half of high-performance CPU area, so the number of devices that actually perform calculations is relatively small compared to the number of GPUs. On the other hand, the GPU is composed of a device for enhancing the simple calculation ability of most of the devices in order that the purpose of the GPU is dedicated to image processing having a large amount of calculation. This difference causes a difference in the computational power between the CPU and the GPU, and the concept of GPGPU has emerged. These GPGPUs are useful for large-scale simple computation tasks such as image processing, and show performance degradation in conditional or branching programming.

To use parallel processing and GPGPU with the above-mentioned characteristics, it is necessary to use not only a CPU and a GPU capable of a program but also a layer or a library which is converted by software, and mainly uses Compute Unified Device Architecture (CUDA) created by nVIDIA or OpenCL and OpenGL do.

On the other hand, with the development of portable electronic devices, mobile devices equipped with cameras become popular, and various applications using image processing in embedded and mobile are spreading. However, mobile has a limited power supply compared to PC, there are physical limitations such as low computing power, small memory and heat due to the difference in size of CPU and GPU mounted in PC and mobile, and embedded GPU has difference between CPU and bandwidth, There are many architectural limitations such as lack of interfaces, fewer cores and the number of shader units. In order to solve this problem, studies are being continued to apply GPGPUs used in PCs to embedded and mobile applications, and graphics processing applications that can be used in embedded and mobile applications such as OpenGL ES 2.0 are being developed.

As described above, various applications using image processing in embedded and mobile are spreading. The efforts to use the above-mentioned parallel processing and GPGPU in an embedded and mobile environment are representative. Prior Art 1 (CPU and GPU Parallel Processing for Mobile Augmented Reality) discloses image processing through parallel processing of a CPU and a GPU in a mobile environment. However, there is an advantage in that the total execution time is reduced due to the lack of waiting time of each processor in the image processing using the conventional parallel processing, rather than the processing of one frame sequentially. However, And the frame to be operated is overlapped.

Another disadvantage of the prior art is a single algorithm processing structure. Normal image processing is completed by several stages of algorithms, but a single algorithm processing structure processes only one algorithm and outputs it to the screen. When a plurality of algorithms are processed using such a single algorithm processing structure, as shown in FIG. 2, data transmission and conversion between the CPU and the GPU occur as many as the number of algorithms. There are many devices in the PC that have higher bandwidth compared to the embedded and mobile environment and compensate the speed difference between the CPU and the GPU. However, the mobile and the processor are low in bandwidth and the devices that compensate the speed difference between the CPU and the GPU are also limited in performance There is a problem that it is inefficient.

(Prior Art 1) "CPU and GPU Parallel Processing for Mobile Augmented Reality", Image and Signal Processing (CISP), 6th International Congress, ppp. 133-137, Dec, 2013

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above problems, and it is an object of the present invention to provide a method and a system for providing a plurality of buffers between a CPU and a GPU in an embedded and mobile environment, And to improve the efficiency of image processing.

A method of processing image data based on CPU-GPU parallel processing in an embedded and mobile environment, includes a decoding step (S100) of obtaining image data from a camera or decoding image data stored in a memory, a decoding step (S100) A data conversion step (S200) for converting data into a texture type to be used in the GPU, sequentially storing the converted texture type data in a plurality of buffers (10) located between the CPU and the GPU, (S300) for sequentially calling data stored in the buffer (10) alternately and applying a predetermined algorithm, and an output step (S400) for outputting data through the algorithm step (S300) The step S100 and the data conversion step S200 are performed by the CPU and the algorithm step S300 and the output step S400 It is performed by the GPU.

The buffer 10 usable in the data conversion step S200 is the buffer 10 other than that being used in the algorithm step S300.

The algorithm applied to the data received from the buffer 10 in the algorithm step S300 is one or more using a frame buffer object.

The decoding step (S100) and the data conversion step (S200) are modularized and processed in parallel.

There is provided a computer-readable recording medium storing a program for implementing an image processing speed-up method using the embedded CPU-GPU distribution processing of the present invention.

In addition, a program stored in a computer-readable recording medium is provided to implement an image processing speed-up method using an embedded CPU-GPU distribution process.

The method for accelerating image processing using embedded CPU-GPU distributed processing according to the present invention is a method for speeding up image processing using an embedded CPU-GPU distributed processing, in which data is stored and read alternately from a plurality of buffers located between a CPU and a GPU in an embedded and mobile environment, It is effective.

In addition, the efficiency of image processing is improved through the multi-algorithm processing structure and the CPU-GPU distributed processing in the GPU installed in the embedded and mobile environment.

1 is a flowchart according to an embodiment of the present invention;
Figure 2 shows a conventional single algorithm processing structure.
FIG. 3 is a multiple algorithm processing structure according to an embodiment of the present invention; FIG.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Prior to this, terms and words used in the present specification and claims should not be construed as limited to ordinary or dictionary terms, and the inventor should appropriately interpret the concepts of the terms appropriately The present invention should be construed in accordance with the meaning and concept consistent with the technical idea of the present invention.

Therefore, the embodiments described in the present specification and the configurations shown in the drawings are merely the most preferred embodiments of the present invention and are not intended to represent all of the technical ideas of the present invention. Therefore, various equivalents It should be understood that water and variations may be present.

FIG. 1 is a view briefly showing a method of speeding up image processing using an embedded CPU-GPU distributed processing according to an embodiment of the present invention. A method of determining an encoding mode according to an embodiment of the present invention will be described in detail with reference to FIG.

As shown in FIG. 1, an image processing speed-up method using an embedded CPU-GPU distributed processing according to an embodiment of the present invention is a method of processing image data based on CPU-GPU parallel processing in an embedded and mobile environment, Decoding step S100, a data conversion step S200, an algorithm step S300, and an output step S400. The decoding step S100 and the data conversion step S200 are performed by the CPU, and the algorithm step S300 and the output step S400 are performed by the GPU.

The decoding step S100 decodes the image data obtained from the camera or stored in the memory into data usable in the CPU. The image received from the camera is an analog video signal and needs to be converted into a digital signal that can be handled by a computer. The operation of converting the video signal of the analog system into the digital code separated by the presence or absence of the unit pulse is referred to as encoding and the operation opposite thereto is referred to as decryption. This encoding includes not only conversion of an analog signal into a digital signal but also compression of a vast amount of analog signal and storing it in a memory or a hard disk. The operation of decrypting in the decrypting step (S100) is the operation of converting the encoded and stored data into data capable of image processing.

The data decoded in the decoding step (S100) is a data type used by the CPU. In order to use the decoded data in the GPU, it is necessary to convert the decoded data into a texture which is a data type used in the GPU. For this reason, the data conversion step S200 converts the image data decoded in the decoding step S100 into a texture type to be used in the GPU, and converts the converted texture type data into two buffers 10 located between the CPU and the GPU Sequentially stored in turn.

The decoding step (S100) and the data conversion step (S200) are modularized and processed in parallel. Modularization and parallel processing of the two steps are intended to reduce image data type conversion time. The post-modular parallel processing can be performed when a multi-core CPU having a plurality of independent cores in a CPU is used. In this case, parallel processing refers to parallel processing by allocating tasks to independent cores in a CPU, rather than parallel processing of a CPU and a GPU.

In the algorithm step S300, the data stored in the two buffers 10 are alternately called in sequence and a predetermined algorithm is applied, and the algorithm is one or more using a frame buffer object.

Figure 2 illustrates a conventional single algorithm processing structure. When a single algorithm processing structure as shown in FIG. 2 is used, the number of data transfer between the CPU and the GPU is increased by the number of algorithms in the image processing by the multiple algorithms, and since data is transferred between the CPU and the GPU, Type conversion is also necessary. The time to convert this data and the time to transmit data are inefficient because they are included in the total working time by the number of algorithms.

FIG. 3 illustrates a multiple algorithm processing structure using a frame buffer object in order to overcome this. In the case of passing data from the CPU to the GPU, the CPU applies one or more predetermined algorithms in the GPU for output. Specifically, an image transferred to the frame buffer of the GPU is created and copied to a frame buffer object of the texture memory. The image transmitted to the frame buffer is consumed without outputting it to the screen by using the Off-Screen Rendering command. The data stored in the frame buffer object is processed through a plurality of predetermined algorithms and is output through the frame buffer.

The buffer 10 used in the data conversion step S200 and the algorithm step S300 is an area of a memory that temporarily stores data while transferring data from one place to another, 10) is used in order to prevent the missing frames of the working frame due to the difference of the operation speed of the CPU and the GPU in the parallel processing, to increase the accuracy of the image and to shorten the working time by preventing overlapping frame work. In the conventional sequential processing structure, when the CPU converts the data of the first frame and passes the data to the GPU, the GPU outputs the data after performing the predetermined image processing, and then the CPU converts the data type of the second frame . The sequential processing structure has the advantage that there is no lost frame, but there is a disadvantage that the total working time increases because one processor is waiting while another processor is working. In order to overcome this, the CPU starts the operation of the second frame immediately without waiting time when the CPU finishes the operation of the first frame and sends data to the GPU, and the GPU starts the operation with the data of the first frame that the CPU finishes. . This is because the total operation time is slowed down because each processor does not have a waiting time, but it has a disadvantage that the frame to be operated is missed or overlapped due to the difference in the operation speed. However, if the buffer 10 is used between the CPU and the GPU, it compensates for the difference in operation speed between the CPU and the GPU, thereby preventing the frame omission and improving the accuracy of the image processing and preventing unnecessary use of the GPU . The reason why there are two buffers 10 is considering the insufficient CPU capacity in the embedded and mobile environments. In the data conversion step S200, the CPU sequentially stores data in each buffer 10, and the GPU sequentially calls data in each buffer 10 in the algorithm step S300 do. When the CPU and the GPU are alternately stored and loaded into the two buffers 10 in sequence, the buffer 10 being used by the GPU can not be used by the CPU. This is because if the CPU uses the buffer 10 while the GPU is loading data from the buffer 10, data may be deleted and data may be omitted if new data is used.

The output step (S400) outputs the data processed in the algorithm step (S300) to the screen.

Table 1 below compares the results of the proposed method with those of the sequential processing structure and the parallel processing structure. The experiment was carried out with an ARM v7 based dual core 1.3Ghz CPU, an SGX 543MP3 GPU consisting of triple cores and 1G memory with mobile devices running Android, and the programming language was OpenGL ES 2.0. The experimental results show that the image of 500 frames long is applied to both the grayscale algorithm and the Sobel edge detector algorithm at the same time.

As shown in Table 1, it can be seen that the sequential processing structure is slower than other working methods, but has no lost frames. In contrast, the parallel processing structure has faster processing time than the sequential processing structure, but the data is lost because the resolution is 480P and 720P because the frame is less than 500, and at 1080P resolution, some frames are doubled to increase unnecessary working time. The method proposed in the present invention combines the merits of sequential processing and parallel processing. In the case of resolutions of 480P and 720P, it is slower than parallel processing but has no data loss. In the case of 1080P resolution, fast. Based on the results of the experiment, it can be seen that the method proposed by the present invention is faster than the sequential processing structure or the parallel processing structure and the data loss can be minimized as the workload increases.

It will be appreciated by those skilled in the art that the image processing speedup method using the embedded CPU-GPU distribution processing described above can be easily included in a recording medium that can be read through a computer by tangibly embodying a program of instructions for implementing the method There will be. In other words, it can be implemented in the form of a program command that can be executed through various computer means, and can be recorded on a computer-readable recording medium. The computer-readable recording medium may include program commands, data files, data structures, and the like, alone or in combination. The program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention or may be those known and available to those skilled in the computer software. Examples of the computer-readable medium include magnetic media such as hard disks, floppy disks and magnetic tape, optical media such as CD-ROMs and DVDs, and optical disks such as floppy disks. Magneto-optical media and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, USB memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the operations of the present invention, and vice versa.

It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

10: buffer
S100: Decryption step
S200: Data conversion step
S300: Algorithm step
S400: Output step

Claims (6)

A method of processing image data based on CPU-GPU parallel processing in an embedded and mobile environment,
A decoding step (S100) of decoding image data obtained from a camera or stored in a memory;
The image data decoded in the decoding step S100 is converted into a texture type to be used in the GPU, and the converted texture type data is sequentially stored in the plurality of buffers 10 located in parallel with each other between the CPU and GPU Conversion step S200;
An algorithm step (S300) for sequentially calling the data stored in the buffer (10) in the data conversion step (S200) sequentially and applying a predetermined algorithm; And
An output step (S400) of outputting the data through the algorithm step (S300);
/ RTI >
The decoding step S100 and the data conversion step S200 are performed by the CPU, the algorithm step S300 and the output step S400 are performed by the GPU,
Wherein the buffer for storing the image data in the data conversion step (S200) is a buffer other than that being used in the algorithm step (S300).
delete The method according to claim 1,
Wherein the algorithm applied to the data received from the buffer (10) in the algorithm step (S300) is one or more using a frame buffer object (Frame Buffer Object).
The method according to claim 1,
Wherein the decoding step (S100) and the data conversion step (S200) are modularized and parallel-processed.
delete A program stored in a computer-readable recording medium for implementing an image processing speed-up method using an embedded CPU-GPU distribution process according to any one of claims 1, 3 and 4.

Images