KR20080006774A - Method and device for transmitting thumbnail data - Google Patents

Abstract

A method and an apparatus for transmitting thumbnail data are provided to transmit the thumbnail data through a serial interface, thereby preventing transmission delay of image data. In a thumbnail data transmitting method of image signal processors respectively connected to an image sensor and a backend chip, the method comprises the following steps of: sequentially receiving image signals of each frame from an image sensor; and generating thumbnail data corresponding to an image signal of an n-numbered frame if a current mode is a preview mode, and transmitting the generated thumbnail data to the backend chip.

Description

METHOD AND DEVICE FOR TRANSMITTING THUMBNAIL DATA}

1 is a block diagram of an image processing apparatus according to an example of the prior art.

2 is a block diagram of an image processing apparatus according to another example of the prior art.

3 is a block diagram of an image processing apparatus according to a preferred embodiment of the present invention.

4 is a block diagram of an image processing apparatus using the serial interface of FIG. 3 as an SPI;

5 is a block diagram of an image processing apparatus using the serial interface of FIG. 3 as I 2 C; FIG.

6 is a flow chart of a preview process in accordance with the present invention.

7 is a flowchart of an image capture process according to the present invention.

The present invention relates to a method and apparatus for transmitting thumbnail data, and more particularly, to a method and apparatus for efficiently transmitting thumbnail data.

Recently, digital cameras have become very popular due to the high integration of image sensors and image processing technologies such as charge-coupled devices (CCD) and complementary metal oxide semiconductors (CMOS).

In addition, the digital camera is also equipped with a portable device such as a mobile phone is a situation where it is possible to conveniently take a picture anywhere.

Recently, image processing apparatuses allow a user to immediately preview an image to be captured, and also enhance storage and transmission efficiency of the image by encoding the captured image.

In addition, the image processing apparatus also supports a function of providing a captured image in a thumbnail size.

The thumbnail image is an image generated by scaling the captured image to a small size so that a user can view and manage a large amount of images easily and quickly.

Meanwhile, a recent image processing apparatus includes an image signal processor that processes an image signal (eg, Bayer data) captured by an image sensor and a back end chip that controls the image signal processor separately. Improve image processing efficiency.

1 and 2 are block diagrams of an image processing apparatus according to the prior art.

1 and 2 illustrate an apparatus in which an image signal processor chip and a back end chip transmit and receive a control signal through a serial interface. FIG. 1 illustrates a serial peripheral interface (SPI), and FIG. 2 illustrates an inter-IC. Shows a case where) is used.

Hereinafter, the prior art will be described with reference to FIG. 1.

As shown in FIG. 1, a conventional image processing apparatus includes an image sensor 100, an image processing unit 102, a memory 104, a scaler 106, a mux 108, 114, a Jpepe encoder 110, and an SPI interface. 112 and a back end chip 116.

Here, the image processor 102, the memory 104, the scaler 106, the first mux 108, the JPEG encoder 110, the SPI interface 112, and the second mux 114 process image signals. And a component included in the image signal processor chip in that it functions to transmit to the back end chip 116.

The image sensor 100 converts and outputs a light signal related to a subject into an electric signal (hereinafter, referred to as an image signal).

In general, the image sensor 100 receives a light signal through a color filter array (CFA) in which R, G, and B filters are arranged at a predetermined ratio, and converts the light signal into an image signal to the image processor 102. Output

The image processor 102 processes the image signal, and the processing may include processes such as interpolation, color correction, gamma correction, and color space conversion of the image signal.

According to the prior art, the image signal is converted into preview data such as RGB data or YUV data during preview. The preview data is sent to the back end chip 116 via the second mux 114 bypassing the scaler 106 and the Jpeck encoder 110.

On the other hand, if there is an image capture command of the user, the back end chip 116 transmits a capture start signal to the SPI interface 112.

The transmission of the preview data is stopped when the capture start signal is received, and the image processor 102 processes the video signal to output image data having a higher resolution than the preview data.

The scaler 106 scales the image data to a full size (size corresponding to the display unit standard of the image processing apparatus or a preset size) and outputs the image data to the JPEG encoder 110.

The JPEG encoder 110 performs a process of compressing the full-size image data according to a preset algorithm, and the encoded image data is output to the back end chip 116 through the second mux 114.

Meanwhile, according to the related art, thumbnail data is also transmitted when full size image data is transmitted.

The scaler 106 according to the related art outputs the image data scaled to the full size to the JPEG encoder 110 and simultaneously stores the image data in the memory 104.

After the full-size image data is sent to the back end chip 116, the scaler 106 scales the image data stored in the memory 104 to the thumbnail size, and through the first mux 108, the JPEG encoder ( Output to 110).

The JPEG encoder 110 compresses the thumbnail data and transmits the thumbnail data to the back end chip 116 through the second mux 114.

As described above, according to the related art, the data transmission is delayed because the full size image data and the thumbnail data are sequentially transmitted, which causes a problem of degrading the performance of the back end chip.

That is, according to the related art, since the scaling to the thumbnail size and the encoding of the thumbnail data are performed after the transmission of the full-size image data is completed, there is a problem in that the back end chip 116 is wasted time to receive the thumbnail data.

In addition, according to the related art, since the memory for scaling the thumbnail data must be provided in the image signal processor chip, there is a problem in that the chip volume becomes large and the manufacturing cost increases.

In the present invention, in order to solve the problems of the prior art as described above, to provide a thumbnail data transmission method and apparatus that can increase the performance of the back-end chip during the transmission of the thumbnail data.

Another object of the present invention is to provide an image data transmission method and apparatus in which an image signal processor does not need a separate memory for transmission of thumbnail data.

In order to achieve the above object, according to a preferred embodiment of the present invention, a thumbnail data transmission method of an image signal processor connected to an image sensor and a back end chip, respectively, (a) an image of each frame from the image sensor Receiving signals sequentially; And (b) when the current mode is a preview mode, generating thumbnail data corresponding to an image signal of an n th frame (where n is an arbitrary natural number) and transmitting the thumbnail data to the back end chip. A thumbnail data transmission method is provided.

According to another aspect of the invention, the image sensor for outputting a video signal corresponding to the subject in units of frames; A back end chip for receiving a preview and image capture command of a user; And sequentially receiving image signals of each frame from the image sensor, and when the current mode is a preview mode, generates thumbnail data corresponding to the image signal of the n th frame (where n is an arbitrary natural number). And an image processor chip for transmitting to an end chip, wherein the back end chip matches and stores thumbnail data of the nth frame and full size image data of the n + k (k is one or more natural numbers) frames. An image processing apparatus is provided.

According to another aspect of the present invention, an image signal processor chip connected to an image sensor and a back end chip, in preview mode, receives an image signal corresponding to n (n is an arbitrary natural number) frame from the image sensor to the image An image processor for generating data, wherein the image data is preview data; A scaler for scaling the preview data to a thumbnail size; A serial interface for receiving a control signal from the back end chip and transmitting thumbnail data of the scaled n-th frame to the back end chip; And an image data output unit configured to output the preview data. When the capture start signal is received from the back end chip, the image data output unit outputs the full size image data of the n + k (k is one or more natural numbers) frames. An image signal processor chip is provided for transmitting to the back end chip.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the drawings, similar reference numerals are used for similar elements.

Terms including ordinal numbers such as first and second may be used to describe various components, but the components are not limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. The term and / or includes any item of a plurality of related items or a combination of a plurality of related items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, and the same or corresponding components will be denoted by the same reference numerals regardless of the reference numerals and redundant description thereof will be omitted. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

Hereinafter, a preferred embodiment of a method and apparatus for transmitting thumbnail data according to the present invention will be described in detail.

3 is a block diagram of an image processing apparatus according to an exemplary embodiment of the present invention.

As shown in FIG. 3, the image processing apparatus according to the present invention includes an image processor 302, a scaler 304, an encoder 306, a serial interface 308, and a mux 310 (hereinafter, referred to as an image data output unit). An image signal processor chip, an image sensor 300, and a back end chip 312 connected thereto.

The image sensor 300 includes a pixel array including a plurality of unit pixels, and converts and outputs a light signal relating to a subject into an electrical signal by photoelectric conversion. The image sensor 300 may be, for example, a CCD or a CMOS device.

The dual CMOS image sensor uses a switching method of making metal oxide semiconductors by the number of pixels and using them to sequentially detect signals output from each pixel.

The CMOS image sensor has a simple driving method, a low manufacturing cost, and can greatly reduce power consumption, compared to a CCD, and thus can be mainly applied to the image sensor 300 according to the present invention. It will be apparent to those skilled in the art that this may be possible.

Of course, it will be apparent to those skilled in the art that any image pickup device capable of converting and outputting a light signal relating to a subject into an electrical signal can be applied without limitation.

As described above, the image sensor 300 converts the light signal accumulated corresponding to the subject through photoelectric conversion into an electrical signal (hereinafter, referred to as an image signal) and outputs it to the image processor 302. In this case, the image sensor 300 also outputs a vertical synchronization signal for identifying which frame the image signal received by the image processor 302 relates to.

The image processor 302 performs processing such as interpolation, color correction, gamma correction, and color space conversion on the image signal output from the image sensor 300 on a frame basis.

Interpolation converts a video signal, which is Bayer data, into real color. For example, 8-bit RGB data is converted into real color data of 24 bits.

Color correction improves the quality of the color, and gamma correction adjusts the gamma value that depends on the display unit (e.g., monitor and printer) displaying the captured image so that the image of the same color is output on any display unit. .

Color space conversion converts RGB data into data in which color space including luminance and chromaticity information is converted, and RGB data is converted into YUV (or YIQ) data by color space conversion.

YUV is a color expression method used in TVs, and is classified into Y values indicating luminance and U and V indicating chromaticity. The Y value is error sensitive, so we code many bits in the U and V values, and typical Y: U: V is 4: 2: 2.

In the preview mode of an image, YUV data is mainly used as preview data, but RGB data may be directly used without changing color space.

The scaler 304 scales (sizes) the image data output from the image processing unit 302 to a preset size. In this case, the scaler 304 may scale the image data to a full size (size corresponding to the display unit standard of the image processing apparatus), or may scale to the thumbnail size.

The encoder 306 performs a process of compressing the scaled image data according to a compression standard set in advance or set by a user, for example, JPEG, MPEG, BMP, or the like.

In general, the encoder 306 uses a Joint Photographic Experts Group (JPEG) standard and encodes image data through a sampling, discrete cosine transform (DCT), quantization, and encoding process.

Meanwhile, the serial interface 308 receives a control signal such as a preview start signal or a capture start signal from the back end chip 312.

As the serial interface 308, as shown in FIGS. 4 and 5, the SPI 400 or I2C 500 may be used, but is not limited thereto.

The image data output unit 310 selectively outputs preview data or encoded image data to the back end chip 310.

The back end chip 312 performs a process of displaying preview data and storing image data, and transmits a user's image capture command or a preview command to the serial interface 308.

Conventionally, full-size image data and thumbnail data have been transmitted to the back end chip through a mux when capturing an image. According to a preferred embodiment of the present invention, the thumbnail data is transmitted to the back end chip 312 in advance in the preview mode. In addition, the image transmission efficiency may be improved by transmitting the full-size image data to the back end chip 312 at the time of image capture.

For convenience of explanation, an operation process of the image processing apparatus according to the present invention will be described by dividing into a preview time and an image capture time.

As described above, the image sensor 300 outputs a vertical synchronization signal together with the image signal. The image processing unit 302 may process the image signal in units of frames by the vertical synchronization signal.

According to the present invention, when previewing, the image sensor outputs an image signal corresponding to the nth frame (n is an arbitrary natural number) to the image processor 302.

The image processor 302 processes the video signal of the n-th frame to generate image data.

According to the present invention, the image data is branched and output to the scaler 304 and at the same time bypasses the scaler 304 and is output to the image data output unit 310.

The image data output unit 310 transmits image data (preview data) to the back end chip 312 to be used for preview.

On the other hand, the preview data output to the scaler 304 is output to the encoder 306 after being scaled to the thumbnail size.

Encoder 306 encodes and delivers the scaled thumbnail data to serial interface 308.

The serial interface 308 transmits the encoded thumbnail data to the back end chip 312, and the back end chip 312 stores thumbnail data corresponding to the nth frame in a predetermined memory.

As such, the image signal processor chip according to the present invention transmits not only the preview data but also the thumbnail data to the back end chip 312 in parallel.

If there is a user's image capture command, the back end chip 312 transmits a capture start signal to the serial interface 308.

When the capture start signal is received, the image sensor 300 outputs an image signal corresponding to the n + k (k is one or more natural numbers) th frame to the image processor 302.

According to the present invention, when the capture start signal is received, the image processing unit 302 stops the transmission of the image data and the thumbnail data for preview and transmits the image data generated through the processing to the scaler 304.

At this time, the scaler 304 scales the image data in full size and outputs it to the encoder 306.

Encoder 306 encodes the full size image data, preferably in the JPEG encoding.

The encoded image data is transmitted to the back end chip 312 through the image data output unit 310, and the full size image data corresponding to the n + `kth frames is stored in a predetermined memory of the back end chip 312. .

Preferably, the thumbnail data of the n th frame and the full size image data of the n + k th frame are matched with each other and stored.

In this case, k is preferably 1, that is, the n-th frame is immediately adjacent to the frame, but is not necessarily limited thereto.

Hereinafter, for convenience of description, a case in which thumbnail data of the nth frame and image data of the full size of the n + 1 frame are matched with each other will be described.

According to the present invention, when the user selects the thumbnail data of the nth frame, the back end chip 312 decodes the image data of the n + 1th frame and outputs it to the display unit.

In general, since the difference between the frames adjacent to each other is very small, the user may regard the thumbnail data of the n th frame and the full size image data of the n + 1 th frame as the same image.

According to the present invention, the transmission delay of the image data can be prevented by transmitting the thumbnail data at the time of preview and transmitting the full-size image data at the time of image capture.

In addition, since the image signal processor chip does not need to have a separate memory to scale to the thumbnail size, the chip manufacturing cost can be lowered.

Meanwhile, since the present invention continuously transmits thumbnail data to the back end chip in the preview mode, it is necessary to appropriately manage the thumbnail data in the back end chip.

First, when the thumbnail data of the n + 1th frame is received after receiving the thumbnail data of the nth frame, this means that the image capturing corresponding to the thumbnail data of the nth frame has not been performed. The thumbnail data of the nth frame can be deleted.

Second, when the back end chip receives full size image data of the n + 1th frame after receiving thumbnail data of the nth frame, the thumbnail data of the nth previous frame may be deleted.

In general, the image capture command of the user is made in the middle of outputting a predetermined frame from the image sensor 300. If present, thumbnail data of the n th frame is generated and transmitted to the back end chip, and then n + 1 full size image data is transmitted.

Therefore, according to the present invention, only the thumbnail data received when there is a user's image capture command may be selectively stored.

6 is a flowchart of a preview process according to the present invention.

Referring to FIG. 6, the image processor 302 receives an image signal of an nth frame (step 600) and performs a process of processing an image signal (step 602).

As described above, the processing of the image signal may include processes such as color interpolation, color correction, gamma correction, and color space conversion.

One of the preview data output from the image processing unit 302 is bypassed (step 604) and output to the back end chip 312 through the image data output unit (mux 310).

Meanwhile, the scaler 304 may receive preview data from the image processor 302, and the scaler 304 scales it to a thumbnail size (step 608).

The thumbnail data is encoded by the encoder 310 (step 610), and the encoded thumbnail data is output to the back end chip 312 via the serial interface 308.

The back end chip 312 stores thumbnail data corresponding to the nth frame in a memory.

7 is a flowchart of an image capturing process according to the present invention.

FIG. 7 illustrates a case in which the image processor 302 receives an n + 1th frame image after an nth frame image of FIG. 6.

Referring to FIG. 7, when a user issues an image capture command, the serial interface receives a capture start signal from the back end chip 312 (step 700).

At this time, the image processing unit 302 receives an image signal corresponding to the n + 1th frame (step 702), and processes it to generate image data (step 704).

The generated image data is input to the scaler 304, and the scaler 304 scales the input image data of the n + 1 th frame to full size (step 706).

The scaled image data is encoded by the encoder 306 (step 708) and sent to the back end chip 312 via the image data output 310 (step 710).

The full size image data of the n + 1th frame is matched with the thumbnail data of the nth frame and stored in a predetermined memory in the back end chip 312, and decoded when the user requests the full size of the thumbnail image of the nth frame. And output through the display unit.

Preferred embodiments of the present invention described above are disclosed for purposes of illustration, and those skilled in the art will be able to make various modifications, changes, and additions within the spirit and scope of the present invention. Additions should be considered to be within the scope of the following claims.

As described above, according to the present invention, there is an advantage in that a transmission delay of image data can be prevented by using a serial interface for transmitting thumbnail data.

In addition, according to the present invention, the thumbnail data and the full-size image data are transmitted at the time of preview and image capturing, respectively, thereby increasing the transmission efficiency of the image data.

Claims (16)

A thumbnail data transmission method of an image signal processor connected to an image sensor and a back end chip, respectively. (a) sequentially receiving an image signal of each frame from the image sensor; And (b) when the current mode is a preview mode, generating thumbnail data corresponding to an image signal of an n th frame (where n is an arbitrary natural number) and transmitting the generated thumbnail data to the back end chip; How to transfer thumbnail data. The method of claim 1, (c) if the current mode is a capture mode, generating full size image data corresponding to an image signal of an n + k (k is a natural number of 1 or more) frames and transmitting the generated image data to the back end chip; Thumbnail data transmission method, characterized in that. The method of claim 2, The thumbnail data transmission method, characterized in that switching from the preview mode to the capture mode when a capture command is input. The method of claim 2, And the back end chip matches and stores thumbnail data of the nth frame and full size image data of the n + kth frame. The method of claim 2, Encoding thumbnail data of the nth frame; And And encoding full size image data of the n + kth frame. The method of claim 1, And the thumbnail data is transmitted through a serial interface for receiving a control signal from the back end chip. The method of claim 6, The serial interface is a thumbnail data transmission method, characterized in that at least one of I2C. An image sensor for outputting an image signal corresponding to a subject in units of frames; A back end chip for receiving a preview and image capture command of a user; And The video signal of each frame is sequentially received from the image sensor, and when the current mode is the preview mode, the back end is generated by generating thumbnail data corresponding to the video signal of the n th frame (where n is an arbitrary natural number). Include an image processor chip that transmits to the chip, And the back end chip matches and stores thumbnail data of the nth frame and full size image data of the n + kth frame. The method of claim 8, And the back end chip deletes the thumbnail data of the n th frame when receiving the thumbnail data of the n + k th frame. The method of claim 8, And when the full size image data of the n + kth frame is received, thumbnail data of the nth previous frame is deleted. The method of claim 8, And the back end chip stores thumbnail data of a frame input when a user captures an image capture command. An image signal processor chip coupled to an image sensor and a back end chip, comprising: An image processing unit for generating image data by receiving an image signal corresponding to n (n is an arbitrary natural number) frame from the image sensor in the preview mode, wherein the image data is preview data; A scaler for scaling the preview data to a thumbnail size; A serial interface for receiving a control signal from the back end chip and transmitting thumbnail data of the scaled n-th frame to the back end chip; And An image data output unit for outputting the preview data, When the capture start signal is received from the back end chip, the image data output unit transmits full size image data of an n + k (k is a natural number of 1 or more) frames to the back end chip. Processor chip. The method of claim 12, And an encoder for encoding the thumbnail data and the full size image data. The method of claim 12, And the serial interface is at least one of SPI and I2C. The method of claim 12, And the back end chip matches and stores thumbnail data of the nth frame and full size image data of the n + kth frame. The method of claim 12, And the thumbnail data transmission is stopped when transmitting the full size image data of the n + kth frame.

Images