JPH02248173A - Moving picture/still picture converter - Google Patents

Abstract

PURPOSE:To attain effective noise reduction and to obtain a still picture by discriminating a moving picture part with a large movement or a still picture part in a small movement or standstill state and applying noise reduction processing in a time axis direction of a moving picture and a noise reduction processing in a 2-dimensional space. CONSTITUTION:A moving picture signal from a video camera and a VTR or the like is supplied to an interface section 12, fed to a memory section 14 via a bus switching section 32 and stored in a picture memory 18. A moving picture/ still picture discrimination section 22 discriminates whether the picture is a picture with a large movement or a still picture with small movement or standstill state in the unit of picture elements. An interframe smoothing section 24 applies noise reduction processing in the time axis direction of the moving picture according to the result of discrimination of the discrimination section 22. Moreover, an in-frame smoothing section 20 applies noise reduction processing of the moving picture in the 2-dimensional space according to the result of discrimination of the discrimination section 22. The processed still picture is output from an interface section 26.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Industrial Application] The present invention relates to a moving image/still image converting device that outputs moving images input from a television camera, VTR, etc. as still images to a CRT, printer, etc.

[Prior Art] In the past, images output by printers and the like were mainly input from still image input devices such as image scanners, but recently, input devices such as televisions, cameras, video cameras, etc. A wide variety of devices such as cameras and VTRs have come into use.

Furthermore, as the definition of television signals such as HDTV becomes higher, there is a growing tendency to output images of high-definition television signals to printers and printing devices.

[Problems to be Solved by the Invention] However, in general, when extracting one scene of a moving image such as the output of a television camera as a still image, there are still many problems in obtaining a good still image.

In particular, there is a problem in that noise components that are not noticeable in moving images are conspicuous in still images. In other words, in a moving image,
While the noise in one frame becomes difficult to see due to the visual system's integral effect in the time axis direction, in a still image extracted from one frame, the visual system's integral effect is small, and image quality deterioration due to noise becomes noticeable.

Therefore, when converting a moving image into a still image, it is necessary to perform effective noise removal. An object of the present invention is to provide a moving image/still image conversion device that performs effective noise reduction.

[Means for Solving the Problems] A moving image/still image conversion device according to the present invention has a moving image/still image conversion device that determines a moving image portion with large movement and a still image portion with small movement or stillness in an input moving image. a still image determination means; a first noise reduction means for performing noise reduction processing in the time axis direction of the moving image according to the determination result of the determination means; and a second noise reduction means that performs noise reduction processing.

[Operation] By the above means, it is possible to reduce noise in both the time axis direction and two-dimensional space, and therefore, it is possible to obtain a clear still image with reduced noise even from a moving image with relatively large noise. can.

[Example] Hereinafter, the present invention will be described in detail with reference to the drawings.

FIG. 1 shows a block diagram of an embodiment of the present invention. 1
0 is a moving image input terminal connected to a device that outputs a moving image signal, such as a video camera or VTR, and 12 is an interface section thereof. A memory section 14 stores image data and its processed data, and includes a flag memory section 16 and an image memory section 18 inside. Reference numeral 20 denotes an intra-frame smoothing unit that performs smoothing within a frame, and 22 determines whether an image has a large amount of movement, a small amount of movement, or a still image in units of a small area of the image (one pixel in this embodiment). A moving image/still image determination unit 24 is an inter-frame smoothing unit that performs smoothing in the time axis direction, that is, in a small image area (one pixel in this embodiment) at the same position over a plurality of frames.
8 is an interface unit for connection to a still image output device such as a CRT video display device or a printer; 30 is an output terminal.

32 is a bus switching unit that switches the input bus of the image memory 18.

FIG. 2 shows a concrete configuration block diagram of the moving image/still image determining section 22 and the inter-frame smoothing section 24.

34 are two frames A1. A subtractor 36 is a comparator that compares the output of the subtractor 34 with a predetermined threshold value. 38 is a smoother that performs smoothing between pixels of two frames A+, I+B+, +; 40 is a smoother that smoothes between pixels of the two frames A+, I+B+, +; This is a selector that selects the pixel value of .

FIG. 3 shows a block diagram of a specific example of the configuration of the intra-frame smoothing section 20. As shown in FIG. 40.42 is a line/line scan converter that converts line scan image data and flag data in blocks.
Block conversion circuit, 46°48.50.52 is a latch circuit, 54 is a selector, 56 is a comparison circuit, 58 is a multiplier,
60 is an RO in which the coefficients Cs and t of the smoothing filter are stored.
M, 62 is an adder, 64 is a latch circuit with a clear function, and 66 is a latch circuit.

The operation in the above configuration will be explained. In order to capture a moving image, first, the bus switching section 32 is connected to the a side.

When the interface section 12 is activated by an external instruction,
A moving image signal from a video camera or VTR is taken into the image memory 18 via the interface section 12 and the bus switching section 32. At this time, the interface section 12 adjusts the image input speed from an image input device such as a video camera and the writing speed of the image memory section 18. Note that for the sake of simplicity, the image memory section 18
It is assumed that the images captured in the image are two screens (two frames).

Next, the bus switching unit 32 is switched to the b side, and pixels A+, 1 . B+, 1 (i=1~M.

j−1 to N) and input to the moving image/still image determination section 22 and the interframe smoothing section 24. Note that M and N are the number of horizontal pixels and the number of vertical pixels of one frame. movie/
The still image determination unit 22 determines whether each pixel is a moving image or a still image, and uses the determination result F1. ．． Output. This judgment result F ,,
, are stored in the flag memory section 16, and are also output to the interframe smoothing section 24.

Inside the moving image/still image determining unit 24, a subtractor 34 converts two frames A1 . ++ B Difference A between l and l
+, + Find B111. The comparator 36 includes a subtractor 3
The output of F1. ．． set to 1, l A
+, + B+, 11 If <k, it is determined that it is a still image and the flag F l, , is set to O.

On the other hand, in the inter-frame smoothing unit 24, the smoother 38
Frame A I, ++ B 1. The selector 40 outputs the pixel value obtained by smoothing the corresponding pixel of +, and the selector 40 selects the determination result F7. ．． Accordingly, in the case of a still image (F, low O), the output of the smoother 38, that is, 2 frames A1. H, B 1. A pixel value obtained by smoothing I is selected, and in the case of a moving image CF+, +=1), a pixel in frame A5 is selected. The output of the selector 40, ie, the output X of the interframe smoothing section 24, is sent to the image memory section 18 via the switch 32 and stored therein. X in the image memory section 18.

The storage location of , may be the storage area of frame AI, , or the storage area of frame B, , , or both.
Also, frame A1. ++ A special frame other than B l, l may be provided in the image memory unit 18 as a storage area.

When the processing for the entire one screen is completed in this way, smoothing in the time axis direction has been performed for the still image portion. When this inter-frame smoothing is being performed, the intra-frame smoothing unit 20 does not output image data to the interface unit 26.

Next, intra-frame smoothing will be explained. The image data XI, , which have been smoothed in the time axis direction, are sequentially read out from the image memory unit 18. The intra-frame smoothing unit 20 captures this and takes in the pixel area (here, 3 pixels to simplify the explanation).
x 3 pixels as one block. ), the smoothing process is performed according to the flags Fi and j in the flag memory unit 16, and the smoothing process is output to the interface unit 26. Output devices such as a display device and a printer are connected to the output terminal 30, and the interface unit 26 transfers data to these output devices and adjusts the processing speed of the intra-frame smoothing unit 20. .

The concept of smoothing in the intra-frame smoothing section 20 will be explained with reference to FIG. First, according to Examples 1 and 2, the pixel of interest
31. The smoothing process for . - In the boat fishing smoothing process, processing blocks X□ and 1. n+, (s, t=-
1, 0, 1) by the smoothing filter coefficient C11, in which case the smoothed pixel data Y59. In this method, when there is a moving image part and a still image part in the same block, the moving object part and the still image part are mixed and smoothed, which has the disadvantage that the edges of moving objects become blurred.

On the other hand, in this embodiment, the pixel of interest Xm. The flag F... is 0 (that is, a still image), in that block, the smoothing filter coefficient Cs,l for the pixels whose flags F m+g, , , ++ are 1 is set to O, and the center filter coefficient C6 is set to 0. , 0, and conversely, the pixel of interest X. ,n
The flags F, fi,. If is 1 (i.e. video), then
Smoothing filter coefficient C1, for pixels whose flag F□s nil is 0 within that block.

is set to 0, and a modified smoothing filter coefficient C"1.1 is used, which is obtained by adding that amount to the center filter coefficient C6,0.

That is, in Example 1, the pixel of interest X0. . Flag F,.

Since is 0, the flag F m+*,. The smoothing filter coefficient C8,1 for the pixel where 1 is 1 is set to 0, and the center filter coefficient C8, is set to 0. Add to.

As a result, the filter coefficient C6,0 becomes 9/16,
The modified smoothing filter coefficients C's,t are shown as a whole at 70 in FIG. Therefore, the pixel of interest is X. .

Pixel value Y that is smoothed. . is Y□, (X--t , -1+2 X,, -1,, 10X,,
, -z -+1+ 2 X,, -1+ 9 X,
, , +X-++, o-1) / 16.

On the other hand, in the case of Example 2, the flag F of the pixel of interest
,. is 1, so the flag F m+h. The smoothing filter coefficient Cal for the pixel where y is O is set to 0, and the filter coefficient C6, which is centered on the smoothing filter coefficient Cal, is set to 0. Add to. As a result, the filter coefficient C8,0 becomes 11/16, and the modified smoothing filter coefficient C'1. 1 as a whole is indicated by the reference numeral 72 in FIG. Therefore, the pixel of interest, X,n,.

Pixel value Y□, which is smoothed. is 1ll11 (2X, fi-+ n+xm-1,,+++11X,,
.

+X□ne,) /16 becomes.

Next, the above operation will be explained with reference to the internal structure of the intra-frame smoothing section 20 shown in FIG. Image memory section 18
and pixel data X1.1 and flag F11.1 read out line sequentially from the flag memory unit 16. are converted into signals in units of blocks by line/block converters 42 and 44, respectively. In this embodiment, it is converted into a 3×3 pixel block. The line/block conversion unit 42.44 first outputs the pixel of interest X1,1 and its flag F I, , and the latch circuit 46.50 temporarily holds each of them.

The line/block converters 42 and 44 then convert the pixel values within the block to Xi+6. . +1 and? Lug Fm+)+nya.

(s, t=-1, 0, 1) are output sequentially, and the latch circuit 4
8.52 holds this temporarily. The comparator 56 outputs the flags Fm and n and the flag F by the outputs of the latch circuits 50 and 52.
m++1. , +1. The selector 54 selects the output X of the latch circuit 48 based on the match signal from the comparator 56.
m+h nil is selected, and the output of latch circuit 46, X, due to the mismatch signal from comparator 56.

, select.

The multiplier 58 multiplies the output of the selector 54 by the output C, , (filter coefficient) of the ROM 60, and the adder 62 adds the output of the multiplier 58 to the output of the latch circuit 64 and outputs the result to the latch circuit 64. . That is, the adder 62 and the latch circuit 64 perform cumulative addition. However, the latch circuit 64 is cleared to 0 when processing the first pixel X m-1, n-1 of the block.

The above series of operations is performed for all pixels (9 pixels) in the block, and the final pixel is X m + 1. When the processing result of +1 is held in the latch circuit 64, the output of the latch circuit 64 is taken into the latch circuit 66, and the smoothed pixel value Yゎ is obtained.

Bind and output.

Then, such block-by-block processing is performed for all pixels
, (i=1 to M, j=1 to N).

In this embodiment, the basic screen of a moving image is treated as a frame, but it is of course possible to use a field. Also,
In this embodiment, two frames of images are loaded into the memory section 14, but two or more frames may be loaded depending on the memory capacity. Although the average value of pixels at the same position in two frames is used for noise reduction processing in the time axis direction of a moving image, the present invention is not limited to this. For example, if three or more frames are used, an appropriate weighted average may be used.

Furthermore, although the weighted average within a block is used as a noise reduction process in a two-dimensional space of a moving image, a simple average or other filtering process may be used.

Further, although a still image is output while performing noise reduction processing in a two-dimensional space after performing noise reduction processing in the time axis direction for the entire region, the present invention is not limited to this. For example, you can use an appropriate buffer memory without using frame image memory, perform noise reduction processing in the time axis direction and noise reduction processing in two-dimensional space for each small region in a pipeline, and still image. It may also be possible to output .

Although the moving image/still image determination circuit 22 makes a determination based on the magnitude of the difference between corresponding pixels of two frames, it is also possible to use techniques such as motion detection and motion vector detection used in television systems, for example.

In this embodiment, smoothing processing is not performed in the moving image section in inter-frame smoothing, but for example, by using a technique such as motion vector detection to determine where in the next frame the pixel or part of that pixel has moved. Smoothing may be performed using

Although the description has been made using a hardware configuration, it goes without saying that some or all of the processing can be performed using software.

In this embodiment, depending on the moving image part and the still image part of the moving image,
We perform noise reduction in the time axis direction using inter-frame smoothing, and noise reduction in two-dimensional space using intra-frame smoothing, which allows us to create clear still images with little noise, even from relatively noisy moving images. can be obtained.

[Effects of the Invention] As can be easily understood from the above description, according to the present invention, clear still images with less noise can be obtained.

[Brief explanation of drawings]

FIG. 1 is a block diagram of the configuration of an embodiment of the present invention, FIG. 2 is a block diagram of the moving image/still image determination section 22 and interframe smoothing section 24 in FIG. 1, and FIG. 3 is a block diagram of the intraframe smoothing section. FIG. 4, a block diagram of the configuration of the unit 20, is a conceptual explanatory diagram of intra-frame smoothing. 10: Input terminal 20: Intra-frame smoothing unit 22: Video/still image determining unit 24: Inter-frame smoothing unit 30.
Output terminal (Example 1) Figure +Xm-1, n+1 ' xm n-1+9Xm, n
'! 11+1. n-1)+11X, n+2x,. +1)

Claims (1)

[Claims] A moving image/still image determining means for determining whether a moving image part with large movement and a still image part having small movement or stillness in an input moving image; The present invention is characterized by comprising: a first noise reduction means that performs noise reduction processing; and a second noise reduction means that performs noise reduction processing in a two-dimensional space of a moving image according to the determination result of the determination means. Video/still image conversion device.