JP2006041648A - Moving image processing unit - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a moving image processing unit having high performance and suitable for a semiconductor integrated circuit. <P>SOLUTION: A first difference arithmetic circuit receives pixel signals adjacent to each other time-serially and forms a difference signal, an accumulation circuit accumulates the output signals of the first difference arithmetic circuit, a latch section latches the accumulated value of the accumulation circuit corresponding to the pixel signals within a prescribed range, and a comparison section compares the latched accumulated value with the accumulated value of the accumulation circuit, formed by one frame delay and forms a scene change signal when the change exceeds a prescribed amount. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a moving image processing unit, for example, to a technique effective when used in a semiconductor integrated circuit device including a moving image compression circuit.

MPEG-4 and H.264 In moving picture compression techniques such as H.264, there are intra-frame coding called I picture and inter-frame coding called P picture. In the P picture, since the difference between frames is taken and the image data is compressed, the compression efficiency is good in a continuous scene, but the coding efficiency is poor when the scene is changed. In the image at the time of the scene change, the compression efficiency of the I picture is improved. Regarding the scene change detection method, there is JP-A-2000-324499, and regarding the motion vector detection method, there is JP-A-2002-354443.
JP 2000-324499 A JP 2002-354383 A

  In the technique of Patent Document 1, the absolute value of the difference between the current frame and the image signal of the previous frame stored in the frame memory is cumulatively added, and a frame memory that holds an image signal for one frame is provided. I need it. In Patent Document 2, an absolute value of a difference between an input video signal and a signal delayed by a one-frame delay circuit is cumulatively added, and a delay circuit corresponding to one frame is required, which is configured by a semiconductor integrated circuit. There is a problem that the circuit scale becomes large when trying to do so.

  An object of the present invention is to provide a moving image processing unit that has high performance and is suitable for a semiconductor integrated circuit. The above and other objects and novel features of the present invention will be apparent from the description of this specification and the accompanying drawings.

  The outline of a typical invention among the inventions disclosed in the present application will be briefly described as follows. A difference signal is formed by a first difference calculation circuit in response to pixel signals that are adjacent in time series, and the output signal is cumulatively added by a cumulative addition circuit, and the cumulative addition of the cumulative addition circuit corresponding to a predetermined range of pixel signals is performed. The value is held by the holding unit, and the cumulative addition value of the cumulative addition circuit formed one frame later is compared by the comparison unit, and a scene change signal is formed when the amount of change exceeds a predetermined amount.

  A moving image processing unit having a simple configuration and high encoding efficiency can be obtained.

  FIG. 1 is a block diagram showing an embodiment of a moving image processing unit according to the present invention. The moving image processing unit of this embodiment includes a capture block unit 100, an image memory 114, and a moving image compression circuit 113. A feature of this embodiment is that the capture block unit 100 includes a pixel difference calculation circuit 110 and a cumulative addition calculation circuit 111 as a scene change detection unit. Here, the capture block unit 100 includes an original capture circuit, the difference calculation circuit, and a cumulative addition circuit.

  A camera DSP (digital signal processor) 115 receives a video signal formed by an image pickup device such as a CCD (not shown), performs predetermined signal processing, and forms image data in order from the upper left of the image pickup screen in the horizontal direction. To the capture circuit 112. The capture circuit 112 captures image data sent from the camera DSP 115, that is, extracts digital image data for one screen (frame) and stores it in the image memory 114. In this embodiment, the digital pixel data captured by the capture circuit 112 is stored in the image memory 114 and simultaneously sent to the difference calculation circuit 110 through the signal line 120. The difference calculation circuit 110 forms a difference between adjacent pixel data in time series by the capture. This difference data is input to the cumulative addition circuit 111 through the signal line 121. The cumulative addition circuit 111 forms a cumulative addition value for one screen and sends it to the moving image compression circuit 113 through the signal line 122.

  FIG. 2 shows a block diagram of an embodiment of the difference calculation circuit 110 of FIG. The difference calculation circuit 110 includes a pixel holding unit 200 that holds pixels sent from the capture circuit 112, a pixel holding unit 201 that holds pixels from the pixel holding unit 200, and is temporally adjacent. The difference calculation unit 202 is configured to perform a difference calculation of luminance components for each pixel adjacent in the direction. In the difference calculation circuit 110, the pixel data sent through the signal line 120 is first held in the pixel holding unit 200. When the next pixel data is sent, the pixel data held in the pixel holding unit 200 is transferred to the pixel holding unit 201 at the same time as the pixel data is held in the pixel holding unit 200. That is, the pixel holding units 200 and 201 are configured by a shift register or the like.

  When one pixel data consists of a plurality of bits, the pixel holding units 200 and 201 include a plurality of storage circuits (latches) corresponding to each bit, and the pixel data for a plurality of bits sent in parallel is as described above. The operation as a shift register is performed. Then, the difference calculation unit 202 performs a calculation of the difference between the two pixel data held in the two pixel holding units 200 and 201 and an absolute value of the result of the difference calculation, and the absolute value of the difference value is a signal. The signal is sent to the cumulative addition circuit 111 through the line 121.

  FIG. 3 shows a block diagram of an embodiment of the cumulative addition circuit 111. The cumulative addition circuit 111 includes an addition calculation unit 300 and a cumulative addition value holding unit 301. An addition operation is performed on the absolute value of the difference value sent through the signal line 121 and the cumulative addition value so far. The calculation result is stored again in the cumulative added value holding unit 301. This is repeated for one frame, and a cumulative addition value for one frame is obtained. The accumulated addition value is sent to the moving image compression unit 113 shown in FIG.

  In FIG. 3, a scene change detection circuit provided in the moving image compression unit 113 is also exemplarily shown. The scene change detection circuit of this embodiment includes a holding unit 302 and a comparison unit 303, and the holding unit 302 holds the cumulative added value of the previous frame. The comparison unit 303 compares the cumulative addition value of the previous frame with the cumulative addition value of the frame formed by the cumulative addition circuit 111, and determines whether or not the difference exceeds a predetermined amount. If the predetermined amount exceeds a predetermined amount, the scene change detection signal is set to one level, and if the predetermined amount is not exceeded, the scene change detection signal is set to the other level.

  FIG. 4 is a block diagram showing a main part of an embodiment of a video camera unit provided with a moving image processing unit according to the present invention. The camera DSP includes an image sensor such as a CDD, a driving circuit for the image sensor, and a DSP (digital signal processor) that performs signal processing output from the image sensor. The capture block corresponds to the capture block unit 100 of FIG. 1, the moving image compression unit corresponds to the moving image compression circuit 113 of FIG. 1, and the internal memory is used for temporarily holding data. In addition to the circuit shown in FIG. 1, an image processing block, a CPU (central processing unit) core, and a bus state controller as an interface circuit with an external memory are provided and incorporated in one system LSI. The The SDRAM is an external memory of the system LSI and constitutes the image memory 114 of FIG. 1 used for moving image compression processing. The image data formed by the moving image compression process is not particularly limited, but is recorded on a recording medium such as a hard disk memory or a DVD.

  The difference calculation circuit 110, the cumulative addition circuit 111, and the scene change detection circuit provided in the capture block unit 112 process the pixel data input in time series in real time to form the scene change detection signal as described above. The data holding unit used therefor can also be configured with a register or memory circuit having an extremely small storage capacity, and is smaller in circuit scale than the memory circuit or delay circuit for one frame as in the above-mentioned patent publications 1 and 2. Can be realized. Therefore, even if the system LSI constituting the moving image processing unit is mounted, the circuit scale of the system LSI does not have to be increased.

  FIG. 5 is a schematic diagram for explaining the operations of the difference calculation circuit 110 and the cumulative addition circuit 111 of FIG. The difference values between the pixels sent in time series in order from the upper left of the imaging screen are taken one by one, and all the difference values for one frame are added. This added value is sent to the moving image compression unit 113. In other words, in the figure, adjacent pixels such as first and second, second and third, third and fourth, etc. are subtracted (-), and each subtraction value is A cumulative addition value for one frame is obtained by sequentially adding over one frame.

  FIG. 6 is a schematic diagram for explaining the scene change detection operation and the encoding operation in the moving image compression unit 113 of FIG. The moving image compression unit 113 calculates a change amount for each frame with respect to the cumulative addition value formed by the difference calculation circuit 110 and the cumulative addition circuit 111. If the change amount is large, it is determined as a scene change, and I Encoding with pictures. In other words, as shown in the figure, the amount of change in the cumulative addition value for each frame indicated by a black circle is seen. In this case, it is determined that the scene is changed, and encoding is performed using an I picture.

  FIG. 7 is a flowchart for explaining scene change information detection and coded picture determination operations according to the present invention. In step (1), image data is captured. In step (2), a difference calculation is performed between adjacent pixels using time-series image data formed in the capture process. In step (3), the difference values are cumulatively added. In step (4), it is determined whether one frame has been completed. If it is not completed (no), the difference with the next image data and its cumulative calculation are repeated. If it is determined in step (4) that the end of one frame is (yes), in step (5), the change in the cumulative addition value is compared with a predetermined change amount, and the change amount is large (yes). If it is determined, it is determined in step (6) that it is a scene change, and the image data captured in the image memory is encoded with an I picture. If it is determined in step (5) that the amount of change is small (no), it is determined in step (7) that the images are continuous, and the image data captured in the image memory is encoded with a P picture. . Since an I picture can be inserted at the time of a scene change in this way, the compression efficiency can be improved compared to the case of encoding with a P picture.

  For example, in the conventional image compression such as MPEG, I pictures are inserted at regular intervals in accordance with a predetermined specification such as encoding with an I picture once in 15 frames and encoding with other P pictures. In this case, since the I picture is inserted regardless of the content of the scene of the moving image, it may be encoded with the P picture even in the case of a scene change, which may reduce the compression efficiency. On the other hand, in the moving image processing unit of the present invention, when capturing a moving image to be compressed, the difference values in the horizontal direction (time series) of the pixels are cumulatively added, and the cumulative added value for each frame. A scene change is determined based on the amount of change, and an I picture insertion timing is determined.

As in this embodiment, the cumulative addition value of the time-series pixel difference value is greatly related to the complexity of the image, and when the content of the image changes greatly, such as in the case of a scene change, the cumulative addition value also increases. Change. Therefore, the presence / absence of a scene change can be detected from the change amount of the cumulative addition value. In this embodiment, the horizontal direction, that is, the time-series pixel difference calculation and the cumulative addition calculation are performed, so that the scene change information can be detected simultaneously with the capture operation. This eliminates the need for extra memory access such as writing image data for one frame and reading images, enabling scene change detection with a small circuit scale and improving the efficiency of efficient video compression as described above. Can be planned. Further, in this embodiment, the time-series pixel difference value takes only the luminance component, the color difference component takes the difference value, and the absolute value is not cumulatively added. As a result, the amount of calculation can be reduced. FIG. 8 shows a block diagram of another embodiment of the difference calculation circuit used in the present invention. In this embodiment, the difference is further obtained from the output signal of the difference calculation circuit 110 by a similar difference calculation circuit 110 ′. Similar to FIG. 2, the difference calculation circuit 110 includes a pixel holding unit 200 that holds pixels sent from the capture circuit 112, a pixel holding unit 201 that holds pixels from the pixel holding unit 200, and is temporally adjacent. In other words, the imaging screen includes a difference calculation unit 202 that performs a difference calculation of luminance components for each pixel adjacent in the horizontal direction and a calculation for obtaining an absolute value of a result of the difference calculation. In the difference calculation circuit 110, the pixel data sent through the signal line 120 is first held in the pixel holding unit 200. When the next pixel data is sent, the pixel data held in the pixel holding unit 200 is transferred to the pixel holding unit 201 at the same time as the pixel data is held in the pixel holding unit 200. That is, the pixel holding units 200 and 201 are configured by a shift register or the like.

  Also in the difference calculation circuit 110 ', the difference data sent through the signal line 121 is first held in the difference holding unit 200'. When the next difference data is sent from the difference calculation circuit 110, the difference data is held in the difference holding unit 200 ′ and at the same time the difference data held in the difference holding unit 200 ′ is held as a difference. To the unit 201 ′. That is, the difference holding units 200 ′ and 201 ′ are composed of shift registers and the like as described above. Then, the difference calculation unit 202 ′ further calculates the difference between the two difference data held in the two difference holding units 200 ′ and 201 ′, and the absolute value of the second difference value, in other words, the absolute value of the second differentiation. The sum of the values is sent to the cumulative addition circuit 111. It is possible to determine whether or not a scene change has been detected in the same manner as described above by determining the amount of change in the cumulative addition value for each frame calculated in this way. By detecting a scene change in this way, it becomes possible to detect a scene change with higher accuracy than accumulating and adding the absolute values of the difference values once.

  Although the invention made by the inventor has been specifically described based on the above embodiment, the present invention is not limited to the above embodiment, and various modifications can be made without departing from the scope of the invention. The scene change detection is realized by hardware in which a circuit as shown in FIG. 3 is provided in the moving image compression circuit. In addition, in the system LSI shown in FIG. , 201, the cumulative addition value holding unit 301 and the holding unit 302, and the CPU is used as the difference calculation circuit 202 and the addition calculation unit 300 to perform signal processing for scene change detection by software using a program written in the internal memory. It may be what you do. The image data may be received from the camera DSP, or may be received from an analog video signal read from a television broadcast or a VTR. In this case, the analog video signal is first converted into a digital signal in the capture circuit. The present invention can be widely used as a moving image processing unit.

It is a block diagram which shows one Example of the moving image processing unit which concerns on this invention. FIG. 2 is a block diagram illustrating an example of a difference calculation circuit 110 in FIG. 1. FIG. 2 is a block diagram showing an embodiment of the cumulative addition circuit 111 of FIG. 1. It is a principal part block diagram which shows one Example of the video camera part provided with the moving image processing unit which concerns on this invention. FIG. 2 is a schematic diagram for explaining operations of a difference calculation circuit 110 and a cumulative addition circuit 111 in FIG. 1. It is a schematic diagram for demonstrating the scene change detection operation | movement and encoding operation | movement in the moving image compression part 113 of FIG. It is a flowchart figure explaining the scene change information detection which concerns on this invention, and an encoding picture determination operation | movement. It is a block diagram which shows another Example of the difference calculating circuit used for this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 100 ... Capture block part, 110, 110 '... Difference calculating circuit, 111 ... Cumulative addition circuit, 112 ... Capture circuit, 113 ... Moving image compression circuit, 114 ... Image memory, 115 ... Camera DSP, 120-122 ... Signal line, 200, 201: Pixel holding unit, 200 ', 201' ... Difference holding unit, 202, 202 '... Difference calculating unit, 300 ... Addition calculating unit, 301 ... Cumulative added value holding unit, 302 ... Holding unit, 303 ... Comparison unit .

Claims (8)

A first difference calculation circuit which receives a pixel signal adjacent in time series and forms a difference signal;
A cumulative addition circuit for receiving an output signal of the first difference calculation circuit;
A holding unit for holding a cumulative addition value of the cumulative addition circuit corresponding to a predetermined range of pixel signals;
A comparison unit that compares the cumulative addition value of the holding unit with the cumulative addition value of the cumulative addition circuit formed with a delay of one frame;
A moving image processing unit, wherein a scene change signal is formed when an output signal of the comparison unit exceeds a predetermined amount. In claim 1,
A capture circuit that receives a time-series pixel signal and a moving image compression unit;
The first difference calculation circuit and the cumulative addition circuit are provided in the capture block unit together with the capture circuit,
The moving image compressing section performs encoding with a P picture in a frame in which the scene change signal is not formed, and performs encoding with an I picture in a frame in which the scene change signal is formed. Image processing unit. In claim 2,
The moving image processing unit according to claim 1, wherein the pixel signal in the predetermined range is a pixel signal for one frame. In claim 3,
The first difference calculation circuit and the cumulative addition circuit obtain the difference between the time-series pixel signals and the cumulative addition value in synchronization with the writing of the pixel data of the pixel signals to the image memory by the capture circuit. A featured video processing unit. In claim 3,
Furthermore, a second difference calculation circuit is provided,
The output signal of the first difference calculation circuit is input to the second difference calculation circuit, and a further difference signal of the difference signals adjacent in time series is formed and transmitted to the cumulative addition circuit. A featured video processing unit. In claim 3,
The moving image processing unit, wherein the time-series image signal input to the capture block unit is formed by a camera circuit. In claim 3,
The moving image processing unit further comprises a CPU core, an image processing block, and an external interface circuit, and is constituted by a one-chip semiconductor integrated circuit device. In claim 3,
The moving image processing unit, wherein the first difference calculation circuit obtains an absolute value of the difference signal.

Images