KR20090059566A - Video input device for image encoder - Google Patents

Abstract

A video input device for an image encoder is provided to support a camera mode for inputting natural images in real time in a DVR and a real-time mobile terminal application. A video input device for an image encoder comprises a host register(10), a file input mode unit(32), a camera input mode unit(33), a data selection unit(34) and a data-reading control unit(35). The host register receives an initial value required for the operation of the video input device. The file input mode unit receives video data of a file format from a host processor based on the initial value. The camera input mode unit receives video data of a signal level from a camera device based on the initial value. The data selection unit selects the output of one of the file input mode unit and camera input mode unit.

Description

Video input device for video encoder {VIDEO INPUT DEVICE FOR IMAGE ENCODER}

The present invention relates to a video input device for inputting image data used in an H.264 / AVC standard image encoder (Encoder).

The present invention is derived from a study conducted as part of the IT growth engine technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. ].

With the development of multimedia technologies such as digital storage media, TV broadcasting, real-time mobile communication, and semiconductor technologies, the development of IP (Intellectual Property) or SoC (System-on-Chip) for high-compression, high-performance video algorithms is under way.

In particular, the H.264 / AVC video compression standard, which has about twice the compression performance compared to MPEG-4, has been adopted as a standard in various applications, and video encoder IPs based on this have been actively developed.

Among them, the H.264 / AVC video encoder receives an input digital signal composed of Y (luminance signal), Cb (color difference signal), and Cr (color difference signal) components through a video input device, and receives IME (Integer Motion Estimation), Perform Fine Motion Estimation / Motion Compensation (FME / MC), Intra Prediction (IP) functional modules, and use Context-Adaptive Variable Length Coding (CAVLC), Context-Based Adaptive Binary Arithmetic Coding (CABAC), or De-Blocking (DB). Performing a function module to output a compressed video stream, wherein the video input device used in the H.264 / AVC video encoder is a digital video storage (DVR) or mobile communication terminal application that requires real-time natural video In the form of receiving camera data directly from a CMOS image sensor (CIS) or CCD camera, and in applications that compress existing image content such as TV broadcasting. As a format for receiving image data in a format, only a video input device has been provided which restricts the input format differently according to an application.

However, in the case of using a video input device having a limited format of the input image, in particular, camera data is directly received in the RTL (Register Transfer Level) simulation or the field verification stage at the FPGA (Field Programmable Gate Array) level. It requires a real-time camera interface, and also requires an input device that receives test image data in a predetermined file format for comparison with a reconstructed image (hereinafter referred to as reference image) separately. Inconsistency was inferior.

In order to solve the above-described problem, an object of the present invention is to provide a video input device capable of simultaneously supporting a camera input mode for natural image data input and a file input mode for image input in a predetermined file format.

In addition, an object of the present invention is to provide a video input device that can receive an appropriate image in the image data encoder IP development (RTL simulation or FPGA level functional verification) step without using an additional test interface device by using the file mode.

In order to achieve the above object, a video input device of an image encoder according to the present invention includes a host register for receiving an initial value necessary for the operation of the video input device; A file input mode unit configured to receive video data in a file format from a host processor based on the initial value; A camera input mode unit configured to receive video data of a signal level from a camera device unit based on the initial value; A data selector for selecting one of the file input mode and the camera input mode; And a read control unit for controlling to transmit the selected data to the image encoder.

The file input mode unit and the camera input mode unit include Y, U, and V buffers for storing pixel data. The file input mode and the camera input mode have the following characteristics.

1) File input mode: It is used for function verification by receiving test image data in the form of a file from a file device such as a host processor. Or as an input device in an image encoder application for an input image in the form of a file.

2) Camera input mode: Used as an input device in an application that receives a video signal from a CIS or CCD and delivers a natural image to an H.264 encoder. Or for real-time operation verification of H.264 encoder IP (eg CIF: 30fps, VGA: 30fps).

As described above, the video input apparatus according to the present invention may simultaneously support a camera mode for inputting a natural image and a file mode for inputting an image in a predetermined file format.

Accordingly, the video input device according to the present invention supports a camera mode for real-time natural video input in a digital video storage device (DVR) and a real-time mobile terminal application, and a predetermined file in a multimedia broadcasting application that requires encoding of image content. It supports file mode that allows video input.

In addition, the video input device according to the present invention can receive various test image data in a predetermined file format through a file mode operation without an additional test interface device in an RTL simulation or an IP verification function using an FPGA. The function verification time of the encoder IP can be shortened.

Therefore, the video input device according to the present invention can be widely used from the development of the image data encoder IP to the image data encoder of various applications.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram schematically illustrating a video encoding system 100 according to an embodiment of the present invention.

As shown in FIG. 1, the video encoding system 100 according to the present invention includes a host processor 10 and a camera device unit 20 for providing video data of a file format and a signal level, respectively; A host register 31 for receiving initial values required for operation from the host system, a file input mode unit 32 for receiving video data having a file format of YCbCr from a file device such as a host processor, and YCbCr video data for a real camera. A camera input mode unit 33 for receiving input at a signal level from a CIS or CCD, a data selector 34 for selecting outputs of the file input mode and the camera input mode, and prepared data in H.264 image data. A video input unit 30 including an H.264 read control unit 35 which is in control of transmission to the encoder 40; And an H.264 video encoder 40.

In addition, the video input device according to the present invention includes the video input unit 30.

Selection of a camera input mode for real-time natural video input and a file input mode for video input in a predetermined file format in the video encoding system or the video input device according to the present invention is performed by the camera input mode or the host register 31. This is done by setting a register corresponding to each of the file input modes.

FIG. 2 is a diagram illustrating in detail a file input mode unit 32 included in the video encoding system or the video input device of FIG. 1. The operation of the file input mode unit 32 will now be described with reference to FIGS. 1 and 2.

The file input mode unit 32 models a virtual CIS or CCD camera and transmits YCbCr [4: 2: 0] or [4: 2: 2] format data to the H.264 encoder 40 by YCbCr [4: 2]. : 0] This is for converting and transmitting the format. Internally, vertical sync signal (Vsync) and horizontal sync signal (Hsync) suitable for 30 f / s are generated. At the time of Hsync generation, it generates a write request signal to send horizontal line data to the host processor (or file device) 10.

In the case of the YCbCr [4: 2: 0] format, since the two types of horizontal line data sets, consisting of data consisting of YCbCr pixels and only Y pixels, must be transferred by the number of H lines, the host processor 10 may be configured to perform the horizontal line. In response to the data request signal (or write request), the YCbCr data corresponding to one line of the image is first, and only the Y data is sequentially stored in the Y0, U, and V buffers of the file input mode unit 31. Write each to the Y1 buffer.

The Y buffer is divided into two regions, the first Y buffer and the second Y buffer (Y0 and Y1) for double buffering, and implemented as a two-port SRAM, and the first line YCbCr data is And alternately write the next line data to Y0, which is the first Y buffer, and to Y1, which is the second Y buffer. When the first YCbCr data has been written to the Y0, U, and V buffers, the H.264 readout control unit 35 takes the horizontal line data to the vim_dma 42 of the H.264 encoder 40 (read request signal). ). In response to the signal, the H.264 encoder 40 checks the state of the DMA through the internal DMAC 43 and generates a control signal for initiating the DMA transfer. Read to and write to frame memory (frame memory: 50).

When the second Y data is written to the Y1 buffer by the host processor 10, the H.264 read control unit 35 transmits a read request signal to the H.264 encoder 40 in the same manner as above, and the H.264 encoder 40 reads data from the Y1 buffer in units of 32 bits through the internal DMAC 43 and writes it to the frame memory 50.

Next, FIG. 3 shows a state transition diagram for configuring a function of the file input control unit of the file input mode unit 32 of FIG. 2 for modeling a virtual camera device and automatically generating Vsync and Hsync.

The file input controller of FIG. 3 consists of six transition states, three counters, and five host registers. Referring to the function of the host register of Figure 3, 'start' is a register for transferring the start command of the file input control unit, 'Width_Hight' transfers the number of pixels constituting one line and the number of lines constituting one screen 'V_high' is the size of the high section of the Vsync signal, 'V_low_toH' is the interval from when the Vsync signal goes low until the Hsync signal goes high, and 'H_low 'Is a register to transfer the interval from when the Hsync signal goes low until the Hsync signal goes high.

Next, in more detail for each state, 'reset' is the initial state, 'Vsync' is a transition state that is operated according to the start command of the 'start' register to keep the Vsync signal high and count the number of LCPS. When the 'Count_line' counter value becomes 'V_high' host register value, it moves to the next state 'VtoH' state. In the 'VtoH' state, the Vsync signal goes low and remains until the 'Count_line' counter value reaches the 'V_low_toH' host register value. The 'Hsync' state is used to make the Hsync signal high. The 'Hsync' state is maintained until the 'Count_pixel' counter value for counting the number of pixels reaches the width defined in the 'Width_Hight' register. The 'HtoH' state is to create the interval from when the Hsync signal goes low to high again, and is maintained until the 'Count_pixel' counter value becomes the 'H_low' register value. The 'Hsync' and 'HtoH' states are repeated until the 'Count_h' counter to count the number of horizontal lines reaches the Hight value defined in the 'Width_Hight' register to generate the Vsync and Hsync signals for one frame. It transitions to the HtoV 'state, and then returns to the' Vsync 'state for generating the Vsync signal, which is a frame start signal.

FIG. 4 is a timing diagram of the file input mode described above in the VGA class 640 × 480, and the operation of the file input mode unit 32 will be described below with reference to FIGS. 3 and 4.

The file input controller generates Vsync by generating a start signal by the 'start' register of FIG. 3. Stores pixel data in the file format input in the interval where Vsync is low and Hsync is high in the corresponding buffer (i.e. Y → Y0 buffer, Cb → U buffer, Cr → V buffer, then Y → sequentially generates buffer write signals for the Y1 buffer, and the like. In the period when the first Vsync is low and Hsync is high, the Y data is stored in the first Y buffer Y0, and the Cb and Cr data are stored in the U and V buffers. In the period when the second Vsync is low and Hsync is high, Only Y data of the YCbCr data is stored in the second Y buffer Y1, and the two operations are alternately performed.

When the first YCbCr data has been written to the Y0, U, and V buffers, the H.264 read control unit 35 ends in the horizontal line unit toward the vim_dma 42 of the H.264 encoder 40 at the time of Hsync falling in FIG. Generates a signal (read request signal) to take data from. In response to this signal, the H.264 encoder 40 reads data from the Y0, U, and V buffers through the internal DMAC 43 and writes it to the frame memory 50, and at the time of the second Hsync fall, the data from Y1. Is read and recorded in the frame memory 50.

FIG. 5 is a diagram specifically illustrating a camera input mode unit 33 included in the video input unit 30 of FIG. 1.

The camera input mode unit 33 of the video input unit 30 receives the video data from the camera device unit 20 at the signal (VICLK, VIVSYNC, VIHSYNC, VIY [7: 0]) level and receives Y, U, and V buffers. To this end, it consists of a camera input control unit, Y, U, V buffer and host register to control the overall operation.

The operation of the camera input mode unit 33 having such a configuration will be described below with reference to FIG. 6 showing the timing diagram.

In an embodiment of the present invention, when using the OV9650 product of Omnivision Co., Ltd., the U0Y0V0Y1U1Y2V1Y3... Format 8-bit pixel data is synchronized via VICLK and entered via the VIY bus in bytes. The camera input control unit, which is a control unit, recognizes the start of a frame from the VIVSYNC signal input from the camera input, and stores pixel data input in a section where the VIHSYNC signal is high in a corresponding buffer in real time (that is, U0 to U buffer, A buffer write signal for Y0 → Y buffer, V0 → V buffer, Y1 → Y buffer, etc. is generated sequentially.

When implementing the Y, U, and V buffers, each buffer is configured to easily convert and store 8-bit data inputs to 32 bits by connecting two 2-port SRAMs having 8-bit data widths in parallel. Like the file input mode unit 32, the Y buffer is divided into two buffers Y0 and Y1 for double buffering. In the first VIHSYNC section, Y data is in the first buffer area Y0, and U and V data are U and V. In the second VIHSYNC interval, the Y data is stored in the second buffer area Y0, and the U and V data are stored in the U and V buffers, and the two operations are alternately performed.

When data has been written to the Y0, U, and V buffers, the H.264 read control unit 35 brings the data in the horizontal line unit to the vim_dma 42 of the H.264 encoder 40 at the falling time of VIHSYNC in FIG. Generate a signal (read request signal). In response to this signal, the H.264 encoder 40 reads data from the Y0, U, and V buffers in 32-bit units via the internal DMAC 43 and writes it to the frame memory 50.

When the second Y data is completed from the camera device unit 20 to the Y1 buffer, the H.264 read control unit 35 transmits a read request signal to the H.264 encoder 40 in the same manner as described above. The encoder 40 reads data from the Y1 buffer in 32-bit units through the internal DMAC 43 and writes it to the frame memory 50.

The present invention has been described in detail with reference to preferred embodiments. However, one of ordinary skill in the art appreciates that various modifications can be made without departing from the spirit of the invention as set forth in the claims below.

1 is a diagram illustrating a video encoding system according to an embodiment of the present invention.

FIG. 2 is a diagram specifically illustrating a file input mode unit included in the video input apparatus of FIG. 1.

FIG. 3 is a state transition diagram constituting a function of a file input controller of the file input mode unit of FIG. 2.

FIG. 4 shows a timing diagram when implementing a file input mode for the VGA (640 × 480) size of FIG. 1.

FIG. 5 is a diagram specifically illustrating a camera input mode included in the video input device of FIG. 1.

6 illustrates an input signal timing diagram when the camera mode of FIG. 1 is implemented.

* Description of the symbols for the main parts of the drawings *

10: host processor 20: camera unit

30: video input unit 31: host register

32: file input mode unit 33: camera input mode unit

34: data selector 35: H.264 read control

40: H.264 video encoder

41: control unit of the H.264 video encoder

42: DMA control unit for the video input unit of the H.264 video encoder

43: DMA control unit of the H.264 video encoder

44: frame memory controller of an H.264 video encoder

50: frame memory of the H.264 video encoder

Claims (11)

As a video input device of an image encoder, A host register to receive an initial value required for operation of the video input device; A file input mode unit configured to receive video data in a file format from a host processor based on the initial value; A camera input mode unit configured to receive video data of a signal level from a camera device unit based on the initial value; A data selection unit for selecting one of the file input mode unit and the camera input mode unit; And A read control section for performing control for transmitting the selected data to the video encoder; And a video input device of an image data encoder. The method of claim 1, The file input mode unit generates a horizontal sync signal Hsync and a vertical sync signal Vsync according to a value defined by the host register, and generates a video data request signal to the host processor whenever a horizontal sync signal occurs. A video input device of a video data encoder. The method of claim 2, The host processor converts YCbCr [4: 2: 2] or [4: 2: 0] data of a video image into YCbCr [4: 2: 0] format in response to the video data request signal to the file input mode unit. The video input device of the video data encoder characterized by the above-mentioned. The method of claim 3, wherein The file input mode unit includes Y, U, and V buffers for storing YCbCr data of the video image. The Y buffer is a buffer for storing Y pixels of video image data, and comprises two buffers (Y0, Y1) for double buffering. The method of claim 4, wherein The read control unit transmits data of the corresponding line unit to the image encoder each time YCbCr data of one line is completed in the Y0, U, and V buffers and whenever Y data of one line is completed in the Y1 buffer. And generating a signal for reading the video input device. The method of claim 1, The camera device unit generates VICLK, VIVSYNC and VIHSYNC signals, The camera input mode unit recognizes a start of a frame whenever the VIVSYNC signal is generated, and receives YUV data of a video image from the camera device unit. The method of claim 1, The camera input mode unit includes Y, U, and V buffers for receiving and storing YUV [4: 2: 2] data of video image data. The Y buffer is a buffer for storing Y pixels of video image data, and comprises two buffers (Y0, Y1) for double buffering. The method of claim 7, wherein The camera input mode unit U0Y0V0Y1U1Y2V1Y3... Storing 8-bit pixels of YUV data in a format in bytes in the Y, U, and V buffers, And the camera input controller of the camera input mode unit sequentially generates a buffer write signal for recording the pixel data of the format into respective Y, U, and V buffers. The method according to claim 4 or 7, The read controller generates a read request signal for transferring data stored in the file input mode unit or the camera input mode unit to the image encoder, When the read request signal is generated, the data stored in the Y0, U, and V buffers of the file input mode unit or the camera input mode unit is transferred first. Then, when the read request signal is generated, the data stored in the Y1 buffer of the file input mode unit or the camera input mode unit is transferred. The video input device of the image data encoder, characterized in that the transmission. The method according to claim 4 or 7, And the Y buffer is configured by connecting two two-port SRAMs having an 8-bit data width in parallel to each other. The method of claim 2, The file input mode unit includes a file input control unit, The file input control unit includes three counters and five host registers, and uses them to control data request signal generation to the host processor and reception of video data from the host processor, The host register is a register for transmitting a start command of the file input controller, a register for transmitting the number of lines of a video image and the number of pixels constituting the one line, and a register for transmitting a size of a high section of a Vsync signal. A register that transfers the interval from when the Vsync signal goes low until the Hsync signal goes high, and a register that transfers the interval until the Hsync signal goes high when the Hsync signal goes low Including, And the counter includes a counter for counting the number of LCPSs, a counter for counting the number of pixels, and a counter for counting the number of lines.

Images