KR20070072746A - Method and image processing apparatus and processor using camera control processor having small size memory - Google Patents

Abstract

An image rotation method, a recording medium, a processor and an image processing apparatus using a camera control processing having a small-capacity memory are provided to freely rotate a captured image even though a small-capacity CCP(Camera Control Processor) memory is used. A raw image captured by an image sensor is encoded into a JPEG(Joint Photographic Experts Group) file(S402). The JPEG file includes a restart marker for each of predetermined basic pixel lines. When rotation of the JPEG file is required, the restart marker of the JPEG file is recognized. The JPEG file is split into a plurality of unit files based on the restart marker(S406). The plurality of unit files is rotation-decoded into raw images(S408). The rotation-decoded raw images are accumulated.

Description

Image rotation method, processor and image processing apparatus using a camera control processor having a low capacity memory TECHNICAL FIELD

1 is a diagram showing the configuration of an image processing apparatus according to a preferred embodiment of the present invention.

2 is a diagram illustrating a module configuration of a rotation application according to an exemplary embodiment of the present invention.

3A-3D illustrate an image rotation process in accordance with the present invention.

4 is a flow chart of an image rotation process in accordance with the present invention.

5 is a flowchart of a file division process according to the present invention.

The present invention relates to an image rotation method, a processor and an image processing apparatus using a camera control processor having a low capacity memory, and to an apparatus, a processor, and a method for enabling image rotation using a low capacity CCP memory.

Recently, digital cameras are rapidly evolving due to the high integration of image sensors such as Charged Coupled Device (CCD) sensors and Complementary Metal Oxide (CMOS) sensors and the processing of digital images. It is the situation that is attached.

An image processing device such as a mobile equipped with a camera captures a raw image of a subject through an image sensor. The image sensor includes a Charged-Coupled Device (CCD) sensor and a Complementary Metal Oxide (CMOS) sensor including a plurality of pixels. It may include.

In mobile, a CMOS sensor is mainly used. The image sensor converts a light signal about an image into an analog signal, which is an electrical signal, and converts it into a digital signal through an analog / digital converter.

Here, the digital signal converted to the light signal corresponds to the RGB (Red, Green, Blue) signal detected in each pixel.

Such RGB data is converted into YUV data through an image signal processor (ISP) or the like, and the converted data is generally encoded into a JPEG file and stored in a memory.

JPEG is an image compression standard set by the Joint Photographic Experts Group and is the most widely used graphic file format because of its high compression efficiency.

In portable devices such as mobile, the raw image is encoded by the Camera Control Processor (CCP), which uses 256K bytes of low memory buffer (CCP memory) to reduce manufacturing costs. It is common.

On the other hand, in the past, it was satisfactory only to capture an image through a camera mounted on a mobile device, but recently, as the number of pixels of a mobile camera increases, it is possible to obtain an accurate image, thereby increasing the user's demand for graphic editing. In particular, there is a growing demand for adjusting the angle of an image already taken.

However, when you want to rotate an image, you need to have at least as much low image size as possible in the CCP memory. When the image is set to 680x480 resolution, which is a commonly used video graphic adapter (VGA) resolution, the raw image is a single image. Including RGB 16-bit information in a pixel requires at least 600K bytes of CCP memory capacity to decode the corresponding JPEG file.

Therefore, the rotation of the image is not possible with the conventional CCP memory capacity, and if the CCP memory capacity is increased, there is an undesirable problem because the manufacturing cost of the CCP is increased.

In the present invention, in order to solve the problems of the prior art as described above, image rotation method, processor and image processing using a camera control processor having a low capacity memory that can freely rotate the captured image even if a low capacity CCP memory is used. An apparatus is proposed.

Another object of the present invention is to provide an image rotation method, a processor, and an image processing apparatus using a camera control processor having a low capacity memory capable of increasing memory efficiency and processing speed during image rotation.

In order to achieve the above object, according to a preferred embodiment of the present invention, a method for rotating a JPEG file using a camera control processor having a low capacity memory, comprising: (a) a raw image captured by the image sensor; Encoding into a JPEG file, wherein the JPEG file includes a restart marker for each preset basic pixel line; (b) identifying a restart marker of the jpeg file if rotation of the jpeg file is required; (c) dividing the JPEG file into a plurality of unit files based on the identified restart markers; (d) rotating decoding the plurality of unit files into a row image; And (e) accumulating the rotation decoded row images.

Preferably, when rotation of the JPEG file is required, the size of the JPEG file and the memory size of the camera control processor may be compared.

Preferably, the step (b) may divide the jpeg file into a predetermined number of restart marker units.

More preferably, the raw image may be a YUV 420 image.

The method according to the present invention comprises the steps of: dividing the accumulated raw image within a memory size range of the camera control processor; JPEG encoding the divided raw image; And synthesizing the encoded plurality of JPEG files.

Preferably, the basic pixel line may be at least one of 8, 16, and 32 lines.

According to another aspect of the present invention, there is provided a computer readable recording medium having recorded thereon a program for performing the above method.

According to another aspect of the invention, the image sensor for capturing a raw image of the subject; A camera control processor including a low memory and performing encoding of the raw image and decoding of a JPEG file; When rotation of the Jpek file is required, the restart marker of the Jpek file is identified, and the Jpek file is divided into a plurality of unit files based on the identified restart marker. A rotation processor for accumulating a plurality of row images having decoded a plurality of unit files; A system memory for storing the jpeg file and the raw image; And a controller for controlling the raw image capture.

According to another aspect of the present invention, a rotation processor connected to a camera control processor that encodes a raw image captured by an image sensor into a JPEG and decodes a JPEG file, wherein the rotation of the JPEG file is required when the rotation is required. A restart marker identification unit for identifying a restart marker of the effect file; A file dividing unit dividing the JPEG file based on the identified restart marker and dividing a rotation row image decoded by the camera control processor within the CCP memory size range; And an encoding file synthesizing unit for synthesizing a plurality of rotation jpeg files encoded by the camera control processor.

Hereinafter, exemplary embodiments of an image rotation method, a processor, and an image processing apparatus using a camera control processor having a low capacity memory according to the present invention will be described in detail with reference to the accompanying drawings.

The present invention relates to a method for efficiently rotating an image in a camera control processor having a low memory, and may be performed in software by a predetermined rotation application, but is not limited thereto. A module for rotation is mounted. It may also be performed in hardware on a chip (processor).

In addition to the image rotation process proposed in the present invention can be carried out in various forms, it will be apparent to those skilled in the art that a simple modification of this embodiment is within the scope of the present invention.

1 is a diagram illustrating a configuration of an image processing apparatus according to an exemplary embodiment of the present invention.

As shown in FIG. 1, an image processing apparatus according to the present invention may include an image sensor 100, a camera control processor (CCP) 102, a system memory 108, and a controller (CPU) 110. The control processor 102 may include a Jpec codec 106 and a CCP memory 104 which is a memory buffer.

The present invention can be mainly applied in a snapshot method in which an image captured by an image sensor is directly encoded by a J-CODEC without going through a CCP memory, and the captured image is rotated using a low-capacity CCP memory.

The method according to the present invention may be performed by a predetermined application, and FIG. 2 illustrates a module configuration of the rotation application according to the present invention. Hereinafter, the rotation method according to the present invention will be described in detail with reference to FIGS. 1 to 2. Explain.

First, the encoding module 200 performs a process of capturing the raw image captured by the image sensor 100 and transferred to the Jpec codec 106 of the CCP 102.

Jpef files include Start Of Image (SOI), JFIF Marker, Define Quantization Tables (DQT), Start Of Frame (SOF), Define Huffman Tables (DHT), Start Of Scan (SOS), Scan Data, and End Of Image It consists of several blocks, like).

Here, each block is divided by a marker, and various markers are used in the JPEG file structure, and a double restart marker is a marker existing in the scan data, and restores the damaged JPEG file to the maximum. It is a marker used for.

The process of encoding a raw image into a JPEG can be accomplished by sampling, DCT transform, quantization and Huffman coding, which is performed with data converted from RGB to Y, Cb and Cr. Here, the Y component is luminance information, and the Cb and Cr components are chroma information.

In encoding such YUV data, in general, one pixel includes all Y, Cb, and Cr components, or Y may be sampled so that each pixel includes Cb and Cr, one in two pixels. The sampling period is 1: 1: 1 and the latter sampling period is 2: 1: 1.

The sampled YUV data is expressed as a sum of cosine functions through a discrete cosine transform (DCT) process. For this purpose, the sampled data must be divided into basic blocks of a predetermined size. In this case, the 8x8 block becomes the basic block, and in the case of 2: 1: 1, the 16x16 block becomes the basic block for the DCT.

When the 8x8 block is the basic block, 8 lines, and when the 16x16 block is the basic block, 16 lines are referred to as a basic pixel line.

The DCT process is sequentially performed on a basic pixel line basis. In this case, the J-CODEC generates a restart marker for each basic pixel line to be encoded.

Therefore, when the DCT process is completed and finally the JPEG file is generated through quantization and Huffman coding, as shown in FIG. 4, when the base compressed file of the DCT process is an 8x8 block in the file compressed with the JPEG A restart marker is included every eight lines in the vertical direction of the pixel, and the capacity of the image data is reduced after the compression process is completed.

However, if the basic block is a 16x16 block, a restart marker may be included every 16 lines, and without limitation, the restart marker may be included every 24 or 32 lines according to the basic block of the DCT process.

Here, the quantization process is a process of removing unnecessary components from the sum components of the cosine generated in the DCT process, and the Huffman coding process is a process of converting quantized data into Huffman codes using a frequency and a type.

The JPEG file encoded as described above is stored in the system memory 108.

There may be a case where rotation is required by the user's request or preset information about the JPEG file stored in the system memory 108.

The rotation rotates the captured image by 90180/270 degrees. The size comparison module 202 according to the present invention compares the size of the JPEG file with the size of the CCP memory, and the size of the JPEG file exceeds the CCP memory size. If so, the restart marker identification module 204 identifies the restart marker included in the JPEG file.

As described above, the restart marker may exist for each basic pixel line that is a reference for encoding, and may exist for every 8, 16, and 32 lines, for example.

When the file division module 206 divides the JPEG file into units including a preset number of restart markers, the basic pixel line is 8 lines, and is configured to divide the JPEG file into one restart marker unit. In addition, the jpeg file may be divided into eight lines.

In contrast, since the JPEG file is divided into restart marker units, it may be divided into multiples of 8. In FIG. 3A, the width of the JPEG file is 96 lines at 640 (Height) x 480 (Width). The file division is shown.

Here, the division into 96 lines is because 640x96 does not exceed 256K bytes, which is a general CCP memory size.

In the case of partitioning as described above, the CCP memory 104 may secure enough storage space of the divided jpeg file and the decoded row image.

Thereafter, the rotation decoding module 208 according to the present invention decodes the divided jpeg file into a row image again, and rotates and decodes the jpeg file of FIG. 3A so that a row image as shown in FIG. 3B can be generated.

3B illustrates an example of decoding a JPEG file in a YUV 420 format.

The row image accumulation storage module 210 accumulates and stores the row image of the divided jpeg file in the system memory 108 as described above.

When all of the rotated row images of one JPEG file are accumulated by cumulative storage, the accumulated row images need to be re-encoded, so the file division module 206 according to the present invention stores the accumulated row images in the CCP memory size. Split within the range, the structure is as shown in Figure 3c.

In this case, since the height is changed from 640 to 480, the row image segmentation may be performed for each 160 lines, but is not limited thereto.

After the accumulated row image segmentation, the encoding module 200 according to the present invention stores the segmented row image in the CCP memory 104, and then divides the row image sequentially as shown in FIG. 3D. To be encoded.

According to the prior art, when rotating a 640x480 image with VGA resolution, at least the CCP memory requires a capacity of 600K bytes of low image size and 600K bytes of size of rotated image, i.e. 1.2M bytes of information. In the situation where 256K bytes of CCP memory is used to lower the manufacturing cost, image rotation was not possible in the past.

However, according to the present invention, when the rotation is required, as described above, by splitting and decoding the already encoded jpeg file, and by dividing and decoding the decoded raw image again, image rotation is performed without exceeding the CCP memory capacity. Can be done.

As described above, the plurality of JPEG files encoded after image rotation are synthesized into one JPEG file through the encoding file synthesizing module 212 and stored in the system memory 108.

In the above description, the method according to the present invention has been described as being software-processed through an application, but is not limited thereto. In the rotation processor including a size comparison unit, a restart marker identification unit, a file division unit, and an encoding file synthesis unit, It can be handled in hardware.

That is, according to the present invention, the rotation processor encodes a raw image captured by the image sensor into a JPEG, and is connected to a camera control processor that decodes a JPEG file, so that when the rotation of the JPEG file is required, the size comparison unit may be configured. Comparing the size of the effect file and the CCP memory size, wherein a restart marker identification unit identifies a restart marker of the effect file, a file division unit divides the effect file based on the identified restart marker, and The process of dividing the rotation row image decoded by the camera control processor within the CCP memory size range may be performed.

Subsequently, the encoding file synthesizing unit may generate a single JPEG file by synthesizing the plurality of rotation JPEG files encoded by the camera control processor.

4 is a flowchart of an image rotation process according to the present invention.

4 illustrates an image rotation process in the case of a snapshot. Referring to FIG. 4, when the image sensor 100 according to the present invention captures an image sensor of a subject (S400), the captured raw image is Immediately input to the CODEC code 106 of the CCP 102 is encoded into a JPEG file (S402).

Thereafter, when receiving a user's request (S404), the image processing apparatus according to the present invention divides the JPEG file (S406), and performs rotation decoding of the divided file (S408).

In FIG. 4, file division and image rotation are performed when a user receives a request. Alternatively, image rotation may be performed according to preset information.

5 is a flowchart of a file division process according to the present invention.

FIG. 5 illustrates a process of processing in a rotation application according to the present invention. Referring to FIG. 5, when the rotation application according to the present invention requires image rotation, a process of comparing a size of a JPEG file and a CCP memory size is illustrated. In operation S500, when the size of the JPEG file is larger, the restart marker of the JPEG file is identified (S502).

Thereafter, the JPEG file is divided in units of a predetermined number of restart markers (S504), and the split JPEG file is rotated and decoded (S506).

The rotation decoded row image file is accumulated and stored in the system memory 108 (S508), and the rotation application divides the accumulated stored row image again within the CCP memory size range (S512).

The divided row images are encoded by the CCP 102 (S512), and the rotation application performs a process of synthesizing the encoded files of the plurality of row images into one (S514).

Preferred embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art will be able to make various modifications, changes, and additions within the spirit and scope of the present invention. Additions should be considered to be within the scope of the following claims.

As described above, according to the present invention, an image can be rotated using a low-capacity CCP memory.

According to the present invention, since the JPEG file is divided and decoded and the raw image is divided and encoded, the memory efficiency and the processing speed can be increased when the image is rotated.

Claims (13)

A method of rotating a JPEG file using a camera control processor having a low memory, (a) encoding a raw image captured by an image sensor into a JPEG file, wherein the JPEG file includes a restart marker for each preset basic pixel line; (b) identifying a restart marker of the jpeg file if rotation of the jpeg file is required; (c) dividing the JPEG file into a plurality of unit files based on the identified restart markers; (d) rotating decoding the plurality of unit files into a row image; And (e) accumulating the rotation decoded raw images. The method of claim 1, And when the rotation of the JPEG file is required, comparing the size of the JPEG file with the memory size of the camera control processor. The method of claim 1, In the step (b), the image file is divided into a predetermined number of restart marker units. The method of claim 1, And the raw image is a YUV 420 image. The method of claim 1, Dividing the accumulated row image within a memory size range of the camera control processor; JPEG encoding the divided raw image; And And synthesizing the encoded plurality of jpeg files. The method of claim 1, And the basic pixel line is at least one of 8, 16 and 32 lines. A computer-readable recording medium having recorded thereon a program for performing the method according to any one of claims 1 to 6. An image sensor for capturing a raw image about the subject; A camera control processor including a low memory and performing encoding of the raw image and decoding of a JPEG file; When rotation of the Jpek file is required, the restart marker of the Jpek file is identified, and the Jpek file is divided into a plurality of unit files based on the identified restart marker. A rotation processor for accumulating a plurality of row images having decoded a plurality of unit files; A system memory for storing the jpeg file and the raw image; And And a controller for controlling the raw image capture. The method of claim 8, And the rotation processor compares the size of the JPEG file with the memory size of the camera control processor when rotation of the JPEG file is required. The method of claim 9, And the rotation processor divides the jpeg file into a predetermined number of restart marker units. The method of claim 9, And the raw image is a YUV 420 image. The method of claim 1, And the rotation processor divides the accumulated row image within a memory size range of the camera control processor, and synthesizes the encoding jpeg file when the camera control processor encodes the divided row image. A rotation processor connected to a camera control processor that encodes a raw image captured by an image sensor into a JPEG and decodes a JPEG file. A restart marker identification unit for identifying a restart marker of the jpeg file when rotation of the jpeg file is required; A file dividing unit dividing the JPEG file based on the identified restart marker and dividing a rotation row image decoded by the camera control processor within the CCP memory size range; And And a encoding file synthesizing unit configured to synthesize a plurality of rotation jpeg files encoded by the camera control processor.

Images