KR20100007628A - Image sensor interface apparatus and digital photographing apparatus comprising the same - Google Patents

Abstract

PURPOSE: An image sensor interface apparatus and a digital photographing apparatus with the same are provided to connect a high speed image sensor with an existing DSP, thereby converting high speed data inputted from the image sensor and transmitting the converted high speed data to the DSP. CONSTITUTION: A receiver(211) receives the first data from an image sensor at the first transmission rate. A data converter unit(212) converts the first data into the second data with the second transmission rate slower than the first transmission rate. A resize unit(213) changes capacity of the second data. A controller(214) outputs the second data to a DSP(Digital Signal Processor) at the second transmission rate. The first data is serial communication data. The second data is parallel communication data.

Description

Image sensor interface apparatus and digital photographing apparatus having same

The present invention relates to an interface device and a digital photographing device having the same.

In general, an image sensor and a digital signal processor (DSP) are provided in a digital photographing apparatus such as a digital camera or a mobile phone camera.

The image sensor receives light from a subject, converts the light signal into an electrical signal, and outputs the light signal to the DSP. Then, the DSP performs predetermined video signal processing.

Image sensors refrain from being able to input images at high speed. With the development of the image sensor, the development of a DSP capable of receiving image data output from the image sensor at high speed should be accompanied. However, since the DSP receives the image data to perform various signal processing and the like, it takes much more effort and time than developing an image sensor. That is, according to the development of the image sensor, there is a problem that requires considerable effort and time to design a new DSP every time.

The present invention seeks to provide an image sensor interface device interposed therebetween in order to connect a high speed image sensor to a conventional DSP. Another object of the present invention is to provide a digital photographing apparatus including the image sensor interface device.

The present invention provides a receiver for receiving first data from an image sensor at a first transmission rate, and a first conversion of the first data into second data having a second transmission rate later than the first transmission rate. And a data converter unit, a resizing unit for converting the capacity of the second data, and a control unit for outputting the second data to the digital signal processing unit DSP at the second transmission speed.

In the image sensor interface device according to the present invention, the first data may be serial communication data as high speed data, and the second data may be data for parallel communication as low speed data.

The image sensor interface device according to the present invention may further include a memory for storing the second data.

The image sensor interface device according to the present invention may further include a second data converter configured to convert the first data into third data which is later than the first transmission rate and has a third transmission rate that is different from the second transmission rate. have.

The present invention also provides an image sensor, a digital signal processor (DSP) for signal processing image data input from the image sensor, and is connected between the image sensor and the DSP and converts image data input from the image sensor. It provides a digital photographing apparatus comprising an image sensor interface device having a data converter for converting the image data that can be received by the DSP.

In the digital photographing apparatus according to the present invention, the data converter may convert first image data for high speed communication input from the image sensor into second image data for low speed communication.

In the digital photographing apparatus according to the present invention, the image sensor interface device may include a control unit for controlling the second image data to be output to the DSP.

In the digital photographing apparatus according to the present invention, the image sensor interface device may further include a resize unit for reducing the capacity of the second image data.

The digital photographing apparatus according to the present invention may further include a display unit which displays an image corresponding to the second image data whose capacity is reduced by the resize unit during a live view mode.

In the digital photographing apparatus according to the present invention, the image sensor interface device may further include a memory for storing the second image data.

The digital photographing apparatus according to the present invention may further include a storage unit which compresses and stores the second image data stored in the memory in the DSP.

The present invention can provide an image sensor interface device which is connected between a high speed image sensor and a conventional DSP to convert the high speed data input from the image sensor and transmit the data to the DSP. Therefore, it is not necessary to design a DSP suitable for a high speed image sensor, and afterwards, the DSP development may be promoted by designing it as a DSP internal block through the verification process of the interface device.

In addition, the present invention can provide a digital photographing apparatus including the image sensor interface device.

The image sensor interface device of the present invention and a digital photographing device including the same will be described in detail with reference to the accompanying drawings.

Prior to describing the present embodiment, the image data described in the present invention represents data for implementing a frame image forming a still image or a moving image, and are used in the same meaning throughout the present specification.

First, the digital photographing apparatus will be described.

1 is a block diagram illustrating a digital photographing apparatus according to an exemplary embodiment.

Referring to FIG. 1, the digital photographing apparatus includes an image sensor 100, an image sensor interface device 200, a digital signal processing unit (DSP, 300), a display unit 400, a storage unit 500, and an operation unit 600. Equipped.

The image sensor 100 receives light input through a lens, an aperture, and the like, and generates an electric signal corresponding to the light. In the present embodiment, the electrical signal is directly input to the image sensor interface device 200 as a digital signal. The image sensor 100 is an ultrafast image pickup device capable of capturing 30 or more images per second, and outputs data corresponding to the image at high speed. Accordingly, the image sensor 100 may output the data at high speed through serial communication such as low voltage differential signals (LVDS).

Data output at high speed from the image sensor 100 is transmitted to the image sensor interface device 200. The image sensor interface device 200 may include an interface 210 and a memory 220. In this case, the memory 220 is a memory dedicated to the image sensor interface device.

The interface 210 converts the high speed data output from the image sensor 100 into data that can be received by the DSP 300 and outputs the data to the DSP 300. The interface 210 may convert the first data for high speed serial communication output from the image sensor 100 into second data for parallel communication and output the same to the DSP 300.

The DSP 300 performs gamma correction, color filter array interpolation, color matrix, color correction, and color enhancement to convert an image signal according to human vision. Image signal processing such as (color enhancement) is performed. In addition, when the function is set, the DSP 300 may perform an auto white balance or auto exposure algorithm. The video signal subjected to such processing can be output to the display unit 400 and displayed as a predetermined image. Alternatively, the video signal may be compressed and output to the storage unit 500 for recording and storing.

The DSP 300 transmits and receives an image sensor 100 or an interface 210, a control signal, a sync signal, a clock signal, and the like.

In addition to displaying the image, the display unit 400 may display an operation state of the digital photographing apparatus. As the display unit 400, a liquid crystal display (LCD), an organic light emitting display (OLED), an electrophoretic display (EDD), or the like may be used.

The storage unit 500 may store not only compressed image data but also a program for operating the digital photographing apparatus. Different semiconductor memory elements can be used depending on the type of each storage object, the degree of storage of the storage object, and the like.

The manipulation unit 600 is a place where a user inputs a control signal for controlling the operation of the digital photographing apparatus, and may be implemented by various function keys, buttons, a remote controller, a touch screen, and the like.

2 is a block diagram illustrating a digital photographing apparatus according to another exemplary embodiment of the present invention. Similar to the digital photographing apparatus according to an embodiment of the present invention shown in FIG. 1, except that an analog image signal is output from the image sensor 100 to convert the analog image signal into a digital signal. There is a difference to include.

An analog video signal is output from the image sensor 100 to the ADC 110 at a high speed, and the ADC 110 converts the analog video signal into digital image data in real time and transmits the same to the interface 210. The digital image data is data for high speed communication, and is converted into data for low speed communication at the interface 210. The low speed communication data is output to the DSP 300. For example, the interface 210 may convert serial data for serial communication into parallel data.

The DSP 300 transmits and receives a control signal with the image sensor 100, the ADC 110, and the interface 210, and synchronizes the clock signal (sync) and the clock signal with the interface 210 and the ADC 110. clock).

Other components and operations are the same as those shown in FIG.

Hereinafter, an image sensor interface device according to the present invention will be described in more detail with reference to FIG. 3.

Referring to FIG. 3, the image sensor interface device 200 may be divided into an interface 210 and a memory 220.

The interface 210 includes a receiver 211, a data converter 212, a resize unit 213, and a controller 214.

The receiver 211 receives high-speed data from the image sensor and decodes it as necessary.

The data converter 212 converts first data input from the image sensor into second data that can be received by the DSP. The first data for high speed communication is converted into second data for low speed communication. For example, high speed serial communication data or high speed parallel communication data can be converted into low speed parallel communication data. As an example of high speed serial communication, it may be one having two lines per channel, having at least one channel or more, having a speed of 54 Mpixel / sec or more per channel, and capable of multiple channels. As an example of low-speed parallel communication, it may have 8 to 10 data lines per channel, may have a minimum of one channel and a maximum of four channels, and may have an average speed of 100 Mpixel / sec or less per channel. Therefore, an image sensor capable of capturing 30 or more images per second or having 9 million pixels outputs a large amount of data, thereby enabling data transmission at low power and high speed using the serial communication.

The data converter 212 also converts the second data into image data that can be processed by the DSP, and performs signal processing such as removing noise.

The resize unit 213 reduces the capacity of the second data. The image size corresponding to the second data is reduced. In live view mode, the image size can be adjusted to the required size for display on the display.

The controller 214 controls the overall operation of the interface 210 and the memory 220, and controls to transmit the second data from the interface 210 to the memory 220 or the DSP 300.

The memory 220 is an external memory dedicated to the interface 210, and is a space in which the data converter 212, the resize unit 213, and the controller 214 temporarily store data or arithmetic results for arithmetic processing. In addition, the memory 220 is a space for temporarily storing the converted data for output to the DSP 300, and the temporarily stored data may be data before resizing.

As an example of the image sensor interface device of the present invention illustrated in FIG. 3, a block diagram according to FIG. 4 will be described.

According to FIG. 4, the image sensor interface device according to the present embodiment may include an interface 210a and a memory 220. In this embodiment, the interface 210a and the memory 220 exist as separate chips and are connected to each other. However, the present invention is not limited thereto, and the interface 210a and the memory 220 may be formed as one chip. In the present invention, the interface 210a exists as a separate chip independently of the DSP, and is detachable from the DSP.

The interface 210a is a receiver for receiving high-speed data input from an image sensor and includes a serial interface 211a and a sync decode 211b.

The high speed serial interface 211a is a module for receiving high speed data output from an image sensor and is a place where communication such as LDVS can be performed. It is also possible to use a high speed parallel interface capable of performing high speed parallel communication.

In the case of high speed parallel communication or high speed serial communication, the sync decode 211b is a module that decodes the data when transmitting data including a sync code without a sync signal indicating the start and end of an image.

In addition, the interface 210a includes an OB Average 212a, a Bayer Formatter 212b, and an FPN removal 212c as a data converter.

The OB Average 212a is a block for obtaining an average value of the optical black region, and delivers the OB average value to the DSP through the control unit 214.

 Bayer Formatter 212b converts the Bayer Pattern of data transmitted from the image sensor. Therefore, the high speed communication data input from the image sensor is converted into low speed communication data suitable for output to the DSP.

The FPN removal 212c removes fixed pattern noise and removes vertical fixed pattern noise and horizontal fixed pattern noise generated in the image sensor.

Bayer resize (213a) is the minimum required for live-view to minimize power-consuming problems when transmitting raw data of Bayer Pattern for live--view from interface 210a. Adjust the size of the image by the size. Bayer resize 213a reduces the capacity of data transmitted at high speed from the image sensor, enabling low speed transmission to the DSP.

The interface 210a includes a signal controller 214a, a controller 214b, and a DDR controller 214c as a controller.

The signal controller 214a may generate a sync signal required for data transmission to the DSP and control data of another module according to the sync signal. In addition, the signal controller 214a outputs data converted for low-speed communication to the DSP, and is connected to the DDR controller 214c to transfer data stored in the memory 220 to the DSP.

The controller 214b generally controls the interface 210 according to the control signal input from the DSP using low speed serial communication such as SPIO, I2S, or I2C.

The DDR controller 214c is a module that controls the memory 220 dedicated to the interface 210a.

FIG. 5 is a view illustrating the outside of an interface chip among the image sensor interface devices shown in FIG. 4.

According to FIG. 5, the interface 210a exists as a separate chip from the memory 220. The interface 210a includes terminals connected to the memory 220, a DSP, an image sensor, and the like outside the chip. do.

Specifically, the interface 210a chip includes an image sensor terminal (Sensor I / F) for receiving high speed data from an image sensor, and a DSP terminal (DSP I / F) for converting the high speed data into low speed communication data and outputting the DSP to the DSP. ), A memory terminal (DDR I / F) for storing the data for the low-speed communication to the memory dedicated to the interface (210a). In addition, the interface 210a external to the chip includes a terminal (Sync & Clock) for transmitting and receiving the VD, HD, Clock in or out of the DSP. It also has a terminal (Enable & Interrupt & Clock) that transmits an enable signal to the DSP to indicate that the data is valid and an interrupt signal to indicate that the user has set a high point. And a terminal (low speed serial communication) for transmitting a control signal for controlling the memory.

6 is a diagram for describing an exemplary operation of the digital photographing apparatus according to the present invention. In the present embodiment, an operation of the digital photographing apparatus illustrated in FIG. 1 will be described.

The digital photographing apparatus shown in FIG. 6 is similar to the digital photographing apparatus shown in FIG. 1, except that the memory for the interface 210 is the first memory 220, and the DSP 300 is stored as the storage unit 500. A second memory 510 temporarily storing a calculation process or a result of a calculation process, etc., and a storage medium 520 for recording information such as image data are illustrated in detail.

Referring to ① path in FIG. 6, the first image data is input to the interface 210 at high speed from the image sensor 100, and the first image data is converted into the second image data for low speed communication at the interface 210. To transmit the second image data to the DSP 300. In this case, the second image data is converted into one having a general Bayer pattern suitable for the DSP 300. That is, the first image data having a special Bayer pattern according to the high speed image sensor 100 is converted into second projection data having a Bayer pattern suitable for the DSP 300. Also, in order to transmit the second image data to the DSP 300 at a low speed, the capacity of the image data is reduced by reducing the capacity of the first image data input from the image sensor 100. Accordingly, the first projection data input from the image sensor 100 may be converted into second image data and transmitted to the DSP 300 in real time. The second image data transmitted to the DSP 300 may be subjected to predetermined signal processing to display the display unit ( 400). In particular, the image displayed on the display unit 400 during the live view mode is for a user to identify a subject, and may input image data suitable for the resolution of the display unit 400. Accordingly, the interface 210 changes the Bayer pattern of the first image data to generate the second image data, and reduces the capacity of the second image data according to the resolution of the display unit 400. Can be sent to.

② In the case of the high-speed continuous shooting, since the input of the image data of the interface 210 is performed at a high speed and the output is performed at a low speed, the input image data is temporarily stored in the interface first memory 220, and sequentially A method of outputting image data. For example, first image data corresponding to the first image is input to the interface 210, and first image data corresponding to the second image is input in real time. The interface 210 may generate second image data by converting first image data corresponding to the first image, and reduce the capacity of the first image data to output to the DSP 300 and the display unit 400. Since the first image data corresponding to the second image may already be input to the interface 210 while the signal processing is performed on the first image data corresponding to the first image, the first image data corresponding to the first image may be stored in the interface first memory 220. It can be. After signal processing is performed on the first image data corresponding to the first image, the first image data corresponding to the second image is read from the first memory 220 to perform the corresponding signal processing, and the DSP 300 and It can be output to the display unit 400.

3) As for the path, after the high speed continuous shooting is completed, the image data stored in the first memory 220 is transferred to the DSP 300, and the predetermined signal is processed by the DSP 300 to perform the second memory 510 with YCC or RGB data. ) Can be stored. In this case, the DSP 300 may transmit the changed or unchanged capacity of the second image data.

7 is a view for explaining another embodiment of the digital photographing apparatus according to the present invention. In the present embodiment, an operation of the digital photographing apparatus at the time of motion picture capturing is illustrated.

Referring to the path of ① in FIG. 7, frame images are continuously input to the image sensor 100 during video shooting, and first image data corresponding to the frame images is input to the interface 210. The interface 210 converts the first image data into second image data for low speed communication, and reduces the capacity of the first image data and transmits the same to the DSP 300. Then, the second image data input to the DSP 300 is temporarily stored in the second memory 510, and the predetermined image signal processing necessary for displaying a moving image is performed to be transmitted and displayed on the display unit 400.

According to the path of ②, an image necessary at a point in time desired by the user during video recording may be stored in the first memory 220 for the image sensor. The video is photographed along the path of ① and displayed on the display unit 400. The image stored in the first memory 220 may be second image data before resizing.

According to the path (3), after the moving picture is finished, the video stored in the first memory 220 is brought to the DSP 300 to perform video signal processing such as a still picture. It is transmitted to the DSP 300 and the second memory 510 for image signal processing.

In addition, the display unit 400 may display only the image of the extracted region desired by the user among the frame images forming the video through the controller of the interface 210. For example, full frame images may be input from the image sensor 100 in real time, and the images may be temporarily stored in the first memory 220 for the interface through the interface 210. In addition, the controller 210 may select and image the image data corresponding to the partial extraction area of the frame images through the controller of the interface 210, and transmit only the selected image data to the DSP 300 through the interface 210. The display device may be temporarily stored in the second memory 510 for processing of a predetermined image signal, displayed on the display unit 400, and recorded on the storage medium 500 by performing compression processing.

8 is a diagram illustrating an image sensor interface device according to another exemplary embodiment. In this embodiment, an image sensor interface device having a plurality of receivers is illustrated. This embodiment is similar to the image sensor interface device shown in FIG. 4, except that the receiver includes a plurality of high speed serial interfaces. That is, the receiver includes a first high speed serial interface 211a and a second high speed serial interface 211c. Accordingly, the first image data may be received through the first high speed serial interface 211a or the second high speed serial interface 211c according to the image sensor.

As mentioned above, although preferred embodiment of this invention was described with reference to attached drawing, it cannot be overemphasized that this invention is not limited to the said example. It will be apparent to those skilled in the art that various changes or modifications can be made within the scope of the claims, and they are naturally understood to belong to the technical scope of the present invention.

1 is a block diagram illustrating a digital photographing apparatus according to an exemplary embodiment.

2 is a block diagram illustrating a digital photographing apparatus according to another exemplary embodiment of the present invention.

3 is a block diagram illustrating an image sensor interface device according to the present invention.

4 is a block diagram illustrating an image sensor interface device according to an exemplary embodiment.

FIG. 5 is a view illustrating the outside of an interface chip among the image sensor interface devices shown in FIG. 4.

6 is a diagram for describing an exemplary operation of the digital photographing apparatus according to the present invention.

7 is a view for explaining another embodiment of the digital photographing apparatus according to the present invention.

8 is a diagram illustrating an image sensor interface device according to another exemplary embodiment.

Claims (10)

A receiver configured to receive first data from the image sensor at a first transmission speed; A first data converter for converting the first data into second data having a second transmission rate later than the first transmission rate; And A resizing unit for converting the capacity of the second data; And a controller for outputting the second data to the digital signal processor (DSP) at the second transmission speed. The image sensor external interface device of claim 1, wherein the first data is data for serial communication, and the second data is data for parallel communication. The apparatus of claim 1, further comprising a second data converter configured to convert the first data into third data that is later than the first transmission rate and has a third transmission rate that is different from the second transmission rate. And the control unit outputs the third data to the digital signal processing unit (DSP) at the third transmission speed. Image sensor, A digital signal processor (DSP) for signal processing image data input from the image sensor, And an image sensor interface device connected between the image sensor and the DSP and configured to convert image data input from the image sensor and convert the image data into image data receivable by the DSP. The digital photographing apparatus of claim 4, wherein the data converter converts the first image data for high speed communication input from the image sensor into second image data for low speed communication. The digital photographing apparatus of claim 5, wherein the image sensor interface device includes a control unit for controlling the second image data to be output to the DSP. The digital photographing apparatus of claim 6, wherein the image sensor interface device further comprises a resize unit to reduce the capacity of the second image data. The digital photographing apparatus of claim 7, further comprising a display unit configured to display an image corresponding to the second image data whose capacity is reduced by the resize unit during a live view mode. The digital photographing apparatus of claim 6, wherein the image sensor interface device further comprises a memory configured to store the second image data. The digital photographing apparatus of claim 9, further comprising a storage unit which compresses and stores the second image data stored in the memory in the DSP.

Images