JP5672116B2 - Video processing device - Google Patents

Description

  The present invention relates to a video processing apparatus for performing motion estimation between frames before and after a video created by a pull-down method.

  In recent years, in video display devices such as televisions, by inserting interpolated frames between frames of input video, a technology that outputs a higher frame rate than the input frame rate and provides a smooth and less blurred video Has become commonplace. As one of them, a dejudder process for making a video such as a movie appear smoothly is attracting attention.

  Video signals such as movies are originally shot at 24 frames per second, but the video format in Japan is generally 60 frames per second. For this reason, in order to reproduce a video shot in 24 frames at 60 frames, a 2-3 pull-down process for alternately displaying two consecutive frames and three consecutive frames is performed.

  In the display by the 2-3 pull-down process, since continuous frames are alternately output in 2 frames and 3 frames, the moving image is slightly fluctuated. Also, since there are 24 frames with different images per second, the motion per frame is larger than that of 60 frames of video signals, and the blurring of moving images is conspicuous.

  Therefore, the motion between 24 video frames is estimated, interpolation frames are generated and inserted, and 60 frames, that is, the same number of frames as the standard video format, are generated. There is no shakiness of the video during processing, and there is less blurring of the video.

  For example, as such a moving image format conversion technique, there is a technique as disclosed in Patent Document 1.

Japanese Patent Laid-Open No. 2005-16787

  In order to generate an interpolated frame in the dejudge process, it is necessary to detect a motion based on a difference between original frames. There are a variety of motion detection methods, but in general, when performing highly accurate detection, in the detection between consecutive frames before and after, the previous frame reference for detecting the corresponding position in the subsequent frame of the target pixel of the previous frame And detection of a subsequent frame reference for detecting a corresponding position in the previous frame of the target pixel of the subsequent frame are performed.

  Further, in order to cope with fast movement of the target pixel, it is necessary to detect in a wide range. For this purpose, it is necessary to have a somewhat wide search range in the frame that is not the reference frame. To expand the search range, a large amount of memory resources are required.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a video processing apparatus that can use a memory effectively, can handle a wide range of searches without increasing the memory, and can perform highly accurate detection.

In order to solve the above problems, the present invention provides the following video processing apparatuses (a), (b) and (c) .

(A) a process selection unit (21) to which video data composed of a plurality of image frame groups having a plurality of identical image frames is input; a first memory (13) for temporarily storing the image frames; To the frame memory (12) for delaying the image frame by one or a plurality of frames, the second memory (14) for temporarily storing the image frame or the image frame output from the frame memory, and the second memory A selector (22) for selectively supplying an image frame; an address control unit (20) for controlling an address reading position of the image frame stored in the frame memory; and an image frame stored in the first memory. As a reference, a first motion candidate detection unit (15) that detects the motion of the target pixel from the image frame stored in the second memory. If, as a reference image frame stored in the second memory, the second motion candidate detection unit to detect motion of the target pixel from the image frame stored in the first memory (16), whether the frame memory the movement candidate position of the object image containing the al acquired third motion candidate detection unit for detecting the image frame stored in the first memory and the second memory and a (17), wherein the processing selection section first When the second image frame that is different from the first image frame included in the image frame group is detected, the selector is configured to supply the first image frame from the frame memory to the second memory. And when the second image frame is detected to be continuously input at least two times, the second image frame is supplied to the second memory. Video processing apparatus and controls the address control unit controls the motor.
(B) When the process selection unit detects that two or more second image frames following the first image frame are input, the address control unit reads the target pixel from the frame memory. image according to control by, and controls the selector to supply the video data read position following the image data of the image frame stored in the first memory to the second memory (a) Processing equipment.
(C) said first image frame group is composed of a plurality of same image frame, the second image frame group is composed of a plurality of same image frame, the video data, the first The image processing apparatus according to (a) or (b), wherein the plurality of image frame groups and the second image frame group are alternately arranged.

  According to the video processing apparatus of the present invention, by using memory resources efficiently, it is possible to perform both high-accuracy frame detection and frame detection corresponding to fast movement without expanding the memory.

1 is an overall configuration diagram of a video processing apparatus according to an embodiment of the present invention. It is a block diagram of the video processing apparatus which concerns on a highly accurate flame | frame detection. It is a block diagram of the video processing apparatus which concerns on the wide-range frame search of quick operation | movement. It is a figure which shows the memory use state at the time of 2-3 pulldown processing. It is a figure which shows the memory use state at the time of 2-3 pulldown processing.

  Next, embodiments of the present invention will be described with reference to the drawings. In the following description of the drawings, the same or similar parts are denoted by the same or similar reference numerals. However, it should be noted that the drawings are schematic. Furthermore, it should be noted that the mathematical formulas described below are examples and can be appropriately changed within a range in which the essence is the same.

(Embodiment)
(Video processing device)
A video processing apparatus 100 according to an embodiment of the present invention will be described with reference to FIG.

  The video processing apparatus 100 includes an input signal receiving unit 11, a frame memory unit 12, a first motion search memory unit 13, a second motion search memory unit 14, a first motion candidate detection unit 15, and a second motion candidate detection unit 16. A third motion candidate detection unit 17, a motion candidate selection unit 18, a motion determination unit 19, an address control unit 20, a process selection unit 21, and a selector unit 22.

    The input signal receiving unit 11 receives a video input signal composed of a plurality of image frame groups having a plurality of identical image frames. In this embodiment, the input signal receiving unit 11 is a video input signal that has been subjected to 2-3 pulldown processing, that is, a first image frame group having two identical image frames, and a second image having three identical image frames. A video input signal including an image frame group, a third image frame group having two identical image frames, a fourth image frame group having three identical image frames, and the like is received. The frame memory unit 12 temporarily stores the video input signal received from the input signal receiving unit 11 and delays the image frame by one or more frames.

  The first motion search memory unit 13 is a first memory that temporarily stores image frames, and temporarily stores a motion search video input signal received from the input signal receiving unit 11.

  The second motion search memory unit 14 is a second memory that temporarily stores an image frame or an image frame output from the frame memory unit 12, and a motion search video input received from the input signal receiving unit 11. Temporarily store signals. The first motion search memory unit 13 and the second motion search memory unit 14 have the same memory width, for example, a horizontal memory or a vertical memory for 30 lines.

  The first motion candidate detection unit 15 detects a corresponding motion candidate position in the subsequent frame with reference to the previous frame, between successive frames. More specifically, the first motion candidate detection unit 15 uses the image frame stored in the first motion search memory unit 13 as a reference and uses the image frame stored in the second motion search memory unit 14 as a target pixel. Detecting the movement of The second motion candidate detection unit 16 detects a corresponding motion candidate position in the previous frame with reference to the subsequent frame among consecutive frames before and after. More specifically, the second motion candidate detection unit 16 uses the image frame stored in the second motion search memory unit 14 as a reference and uses the image frame stored in the first motion search memory unit 13 as a target pixel. Detecting the movement of The first motion candidate detection unit 15 and the second motion candidate detection unit 16 are used in highly accurate motion detection.

  The third motion candidate detection unit 17 detects the corresponding motion candidate position of the target image acquired from the frame memory unit 12 from the first motion search memory unit 13 and the second motion search memory unit 14. The third motion candidate detection unit 17 is used in a wide range detection of fast motion.

  The motion candidate selection unit 18 selects a motion candidate position based on the corresponding motion candidate position acquired from the first motion candidate detection unit 15 and the second motion candidate detection unit 16.

  The motion determination unit 19 compares the motion candidate position received from the motion candidate selection unit 18 with the motion candidate position received from the third motion candidate detection unit 17, and determines which motion candidate position is to be adopted as the motion position. .

  The address control unit 20 controls the data address reading position of the image frame stored in the frame memory unit 12 in accordance with an instruction from the process selection unit 21. When reading, the position of the motion search target pixel can be arbitrarily determined by changing the readout position of the video signal data between the case of high-precision motion detection and the case of motion detection corresponding to fast motion. To.

  The process selection unit 21 receives the video input signal from the input signal reception unit 11 and detects whether the input video signal is the next image frame group different from the previous image frame group, and the switching of the image frame group. After that, it is determined whether or not the same image frame is detected. The process selection unit 21 transmits the determination result to the address control unit 20 and the selector unit 22.

  The selector unit 22 is a selector that selectively supplies an image frame to the second memory. Specifically, when the selector unit 22 receives the determination result that it is a switching position image, the selector unit 22 selects a video data transmission source for input to the second motion search memory unit 14 and performs switching.

  1, 2, and 3 show the same configuration, but in order to help understanding, FIG. 2 shows each part used in high-precision motion detection by a solid line. Indicates a solid line for each part used in a wide range detection of fast movement.

(Motion search operation of video processing device)
The motion detection of the video processing apparatus 100 will be described separately for highly accurate motion detection and fast motion wide-range detection.

(A) First, highly accurate motion detection will be described with reference to FIG.

  The video input signal transmitted from the input signal receiving unit 11 is transmitted to the first motion search memory unit 13 and the process selection unit 21. The first motion search memory unit 13 temporarily stores the received video input signal for a predetermined memory width, for example, video information for 30 lines. The process selection unit 21 determines whether the video input signal is an image frame group different from the previous image frame group. More specifically, when the image is a 2-3 pull-down processing method, the images are arranged as A1, A2, B1, B2, B3, C1, C2, D1,... As shown in FIG. Of these, images with the same alphabet, for example, A1 and A2, are set as an image frame group composed of a plurality of the same images. The process selection unit 21 determines that the image frame group A has shifted to a new image frame group B when the B1 image next to the A2 image is received at circle 1 in FIG. The determination result is transmitted to the address control unit 20. If the address control unit 20 determines that the transition is to a new image frame group based on the determination result, the address control unit 20 sends the first motion search memory unit to the address control unit 20 as shown in FIG. The frame memory unit 12 is instructed to read the same memory width line from the same position as the video input signal temporarily stored in the frame 13. The motion search target pixel 5 is located near the center line of the first motion search memory unit 13 and the second motion search memory unit 14 in order to facilitate the motion search. In response to this reading instruction, the frame memory unit 12 reads an image of the same width from the same position as the video input signal stored in the first motion search memory unit 13. Further, the selector unit 22 that has received the determination result that the process selection unit 21 has shifted to the new image frame group causes the second motion search memory unit 14 to output the image data stored in the frame memory unit 12. In other words, when the process selection unit 21 detects that a second image frame different from the first image frame included in the first image frame group is input, the process selection unit 21 extracts the first image frame from the frame memory unit 12. The selector unit 22 is controlled to be supplied to the second motion search memory unit 14. As a result, as shown by circle 1 in FIG. 4B, the first motion search memory unit 13 and the second motion search memory unit 14 temporarily store images of the same width read from the same position. .

(B) Next, an operation for performing highly accurate motion detection processing using images stored in the first motion search memory unit 13 and the second motion search memory unit 14 will be described with reference to FIG.

  The first motion candidate detection unit 15 compares the target pixel 5 serving as a reference in the output signal group of the first motion search memory unit 13 with the output signal group of the second motion search memory unit 14, and the target pixel It is detected which position on the second motion search memory unit 14 the destination of 5 moves. Similarly, the second motion candidate detection unit 16 compares the target pixel 5 serving as a reference in the output signal group of the second motion search memory unit 14 with the output signal group of the first motion search memory unit 13. , The position on the first motion search memory unit 13 to which the target pixel 5 has moved is detected. That is, both the movement of the target pixel of the previous frame and the movement of the target pixel of the subsequent frame between successive frames are detected.

  The motion candidate selection unit 18 acquires a motion candidate detection result from the first motion candidate detection unit 15 and the second motion candidate detection unit 16, and selects a motion candidate position. The selected motion candidate position is transmitted to the motion determination unit 19. The motion candidate position is determined by a known method using motion vector detection or the like.

(C) Next, detection of the fastest movement in a wide range will be described with reference to FIG.

  The video input signal transmitted from the input signal receiving unit 11 is transmitted to the first motion search memory unit 13 and the process selection unit 21. The first motion search memory unit 13 temporarily stores the received video input signal for a predetermined memory width, for example, video information for 30 lines. The process selection unit 21 determines whether or not the same image frame is continuously input after an image frame of an image frame group different from the previously input image frame group is input. For example, in the case of the position shown by circle 2 in FIG. 4A, after detecting the image frame B1 of the image frame group B different from the image frame group A, the image frame B2 continuously input is the previous image. It is determined whether the image frame is the same as the frame B1, and the image frame B2 is determined as a selection target. In FIGS. 4A and 2B, as an example, the image frame B1 is used for high-precision detection and the image frame B2 is used for quick and wide-range detection. In the 2-3 pull-down method, images B1 to B3 are used. Since these are the same, any image may be used for any detection. However, since the memory is not unnecessarily expanded, for example, when the image frame B1 is used for high-precision detection, the frame B2 or B3 of the same image frame group different from B1 may be used for the wide-range detection of fast motion. preferable. Further, when there are three identical images in the 2-3 pull-down method as in the B image frame group, it is desirable to perform processing such as control to prevent unused frames from being written to the frame memory in the processing selection unit 21. As an implementation, for example, the first image of a different image frame group is used for high-precision detection, the second image is used for wide-range detection of fast movement, and the third image is not written to the frame memory. You may design so that it may control.

  The process selection unit 21 detects that the B image frame B1 different from the A image frame is input following the A image frame A2 included in the A image frame group. The image frame B2 is detected. When the process selection unit 21 detects that two or more of the same image frames (here, B image frames) are continuously input, the process selection unit 21 controls the address control unit 20 and the selector unit 22 as follows.

  Based on the determination result by the process selection unit 20, the address control unit 20 controls the frame memory unit 12 to read out a video signal including the target pixel 5 for motion search. The position of the motion search target pixel 5 is set to the center of the first motion search memory unit 13 and the second motion search memory unit 14 in order to facilitate a wide range of motion search. The frame memory unit 12 outputs a video signal including the target pixel to the third motion candidate detection unit 17. Furthermore, the selector unit 22 that has received the determination result indicating that the processing selection unit 21 has shifted to a different image frame group reads the video signal data stored in the first motion search memory unit 13 with a delay of one memory width. The processed video signal data is output to the second motion search memory unit 14. As a result, as shown by circle 2 in FIG. 4 (b), the first motion search memory unit 13 has pixels corresponding to one memory width from the target pixel 5 upward, and the second motion search memory unit 14 has target. A pixel corresponding to one memory width from the pixel 5 downward is temporarily stored. In this embodiment, a horizontal memory is used. However, this may be a vertical memory.

(D) Next, using the target pixel 5 acquired from the frame memory unit 12 and the images stored in the first motion search memory unit 13 and the second motion search memory unit 14, a fast motion is detected in a wide range. The operation for performing the processing will be described with reference to FIG.

  The third motion candidate detection unit 17 compares the target pixel 5 serving as a reference transmitted from the frame memory unit 12 with the output signal groups of the first motion search memory unit 13 and the second motion search memory unit 14. Then, it is detected which position on the first motion search memory unit 13 and the second motion search memory unit 14 corresponds to the destination to which the target pixel 5 has moved. That is, both the movement of the target pixel 5 in the upper line direction and the movement in the lower line direction are detected by one memory width, that is, by two memory widths. The detected motion candidate result is transmitted to the motion determination unit 19.

  The motion determination unit 19 compares the motion candidate position with high accuracy received from the motion candidate selection unit 18 with the motion candidate positions in a wide range received from the third motion candidate detection unit 17, and determines which motion candidate position is the motion position. Decide whether to adopt as

  Similarly, in the motion estimation between the subsequent image frame group B and the image frame group C, when moving from the previous image frame group B to the different image frame group C shown in FIG. (B) A high-precision detection is performed using the frame C1 as shown by a circle 3, and a fast motion wide-range detection as shown by a circle 4 in FIG. I do. In FIG. 4 (a), circles 3 and 4 use frame C1 for high-precision detection and frame C2 for wide-range detection of fast movement. However, as described above, this is an example and high-precision detection. Of course, the same result can be obtained even if the frame C2 is used and the frame C1 is used to detect a wide range of fast motion.

  Thus, by effectively using the two memories, it is possible to perform highly accurate motion detection and wide-range detection without increasing the amount of memory.

  Note that the address control unit 20 performed when the processing selection unit 21 detects that the B image frame B1 different from the A image frame is input following the A image frame A2 included in the A image frame group. And the control for the selector unit 22 and the control for the address control unit 20 and the selector unit 22 when it is detected that the B image frame B1 is input and then the B image frame B2 (or B image frame B3) is input. Is something that may be exchanged.

  According to the video processing apparatus of the present invention, by using memory resources efficiently, both high-accuracy frame detection and frame detection corresponding to fast motion can be performed, and an increase in the amount of memory can be prevented. it can.

5 ... Motion detection target pixel 11 ... Input signal receiving unit 12 ... Frame memory unit 13 ... First motion search memory unit 14 ... Second motion search memory unit 15 ... First motion candidate detection unit 16 ... Second motion candidate Detection unit 17 ... third motion candidate detection unit 18 ... motion candidate selection unit 19 ... motion determination unit 20 ... address control unit 21 ... process selection unit 22 ... selector unit

Claims (3)

A process selection unit to which video data composed of a plurality of image frame groups having a plurality of identical image frames is input;
A first memory for temporarily storing the image frame;
A frame memory for delaying the image frame by one or more frames;
A second memory for temporarily storing the image frame or the image frame output from the frame memory;
A selector for selectively supplying image frames to the second memory;
An address control unit for controlling an address reading position of the image frame stored in the frame memory;
A first motion candidate detection unit that detects a motion of a target pixel from an image frame stored in the second memory with reference to the image frame stored in the first memory;
A second motion candidate detection unit that detects a motion of a target pixel from the image frame stored in the first memory with reference to the image frame stored in the second memory;
It said frame Note the movement candidate position of the object image containing the Li or we acquire, and a third motion candidate detection unit for detecting the image frame stored in the first memory and the second memory,
When the process selection unit detects that a second image frame different from the first image frame included in the first image frame group is input, the process selection unit extracts the first image frame from the frame memory to the second image frame. The selector is controlled to be supplied to the memory, and the selector is controlled to supply the second image frame to the second memory when it is detected that the second image frame is continuously input at least two or more. And controlling the address control unit. When the process selection unit detects that the second image frame is input two or more times in succession, the process selection unit controls the address control unit to read out the target pixel from the frame memory and stores the target pixel in the first memory the video processing apparatus according to claim 1, wherein the controller controls the selector to supply the video data read position following the image data of the image frame to the second memory. The first image frame group includes a plurality of identical image frames,
The second image frame group includes a plurality of identical image frames,
The video data, the video processing apparatus according to claim 1 or claim 2, characterized in that it is constituted by arranging a plurality of the first image frame the second image frame group and group alternately.

Images