JPH10145788A - Video reproduction device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display a reduced screen in a simple constitution by eliminating dependence of a multi-screen reproduction function of the receiver side on an encoder of the transmitter side. SOLUTION: A display 2 which can show a multi-screen is connected to a decoder 1 connected to a transmission line 6. At the same time, the encoders 4a to 4c having the cameras 3a to 3c and an on-demand video server 5 are connected to the line 6. The decoder 1 decodes the desired one of image data which are outputted from the cameras 3a to 3c and the server 5 and displays plural decoded images 2a to 2d on a reduced multi-screen. Furthermore, a single screen can be shown in its full size by the instruction of a user. As all pixels are concerned in the display of a full-size screen, the vertical primary inverse DCT is applied in an entire range. Then only a string of (x=0, 2, 4, 6) serves as the pixels concerned in the display of a 1/4 screen and accordingly the pixels of other strings are not calculated. In the same way, only the strings of (x=0, 4) for a 1/16 screen and (x=0) for a 1/64 screen are calculated respectively and is displayed on the screen.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a video reproducing apparatus, and more particularly to a video reproducing apparatus for reproducing a plurality of digital images transmitted via a network as a multi-screen video at one place on the same network.

[0002]

2. Description of the Related Art Conventionally, moving image data is
2. Description of the Related Art There is known a technique of performing compression by a method such as PEG (MPEG1, MPEG2), transmitting the data via a network, and reproducing the data.

FIG. 23 is a diagram for explaining an encoding method according to the MPEG method. Referring to FIG.
Adopts an encoding method using a discrete cosine transform (hereinafter referred to as DCT). In encoding by MPEG, one image S is subjected to two-dimensional DCT using a block B of 8 × 8 pixels as one unit. After that, the blocks subjected to DCT by zigzag scan are
After being made a combination of 0 run and level, it is encoded by Huffman encoding and transmitted.

FIG. 24 is a block diagram showing the configuration of a moving picture reproducing apparatus (decoder) of the MPEG system.

Referring to FIG. 1, a reproducing apparatus includes a separator 105 for separating a desired variable-length code (Huffman code) from MPEG data input via a network;
A decoder 107 for decoding the separated variable length code and reproducing the 0 run and the level, and a zigzag scanning unit 10
9, an inverse quantization unit 111 that performs inverse quantization for each 8 × 8 coefficient, an inverse DCT unit 113 that performs inverse DCT,
First and second frame memories 117 and 119 for storing either a picture or a P-picture, a motion compensation unit 115 for performing motion compensation as a prediction process, and a motion-compensated image and inverse DCT unit 113. An adder 121 for adding the output images; a first frame memory 117, a second frame memory 119, or an adder 1
And an output frame memory 123 for storing any one of the outputs 21.

[0006] The image stored in the output frame memory 123 is output and displayed on a display.

[0007] Further, there has been conventionally known a system for displaying images shot by cameras installed at a plurality of points on one screen as multi-screen images. Such a system is used for the purpose of monitoring multiple points at one point.

It is conceivable to apply the MPEG system to such a system, but in the MPEG system, the data processed by the moving picture reproducing apparatus is enormous even for one screen. Therefore, it is difficult to process a plurality of screens in real time with one playback device.

Under such restrictions, the following system is considered as a system for displaying a multi-screen image on one screen using the MPEG system.

(1) System for realizing multi-screen display using a plurality of decoders FIG. 25 is a diagram showing an example of such a system.
Referring to the figure, the system includes a plurality of cameras 3a to 3c.
And encoders 4a to 4c connected to the respective cameras
, A transmission path 6 for transmitting the encoded signal, a separator 10 for separating the transmitted signal into signals from the respective encoders, and a plurality of decoders 12a to 12c for respectively decoding the separated signals. And a reduction / synthesizer 14 for reducing and combining the decoded image, and a multi-screen output device (display) 16 for displaying the reduced and combined image.

Since the digital video of the selected channel is reproduced by each of the decoders 12a to 12c, it is possible to reproduce a multi-screen without particularly considering the processing speed.

(2) System for Controlling Decoder on Receiving Side FIG. 26 is a diagram showing an example of such a system.

Referring to FIG. 1, the system includes a camera 3a.
An encoder 4a connected to the camera; a bidirectional transmission path 20; a selection separator 22 for selecting a desired signal from the transmitted signals and separating the selected signal; A decoder 24 for decoding, a multi-screen output device 16 for displaying the decoded image data on a multi-screen, a multi-screen display from a user, or a detailed screen display (one screen of the multi-screen display is enlarged. The control PC 2 transmits the screen size to the control personal computer (control PC) 18 on the camera side based on the instruction of whether to perform the display.
6 and a control PC 18 for inputting a screen size from the control PC 26 and controlling the encoder 4a.

When performing multi-screen display, the encoder 4a thins out, encodes, and transmits photographed video data based on the screen size. On the receiving side, the transmitted data is reproduced by one decoder (or a plurality of decoders). When displaying the detail screen, the encoder 4a requests transmission of data obtained by encoding the detail screen.

In this system, when performing multi-screen display, data whose data amount is reduced by being thinned out in advance is input to the decoder, so that the processing load on the decoder can be reduced. it can.

(3) System employing Hierarchical Coding FIG. 27 is a diagram showing an example of such a system.

Referring to the figure, the system includes a camera 3a
And an encoder 4E1 for encoding a basic component for displaying a reduced screen in photographed image data, and an encoding for encoding an extended component capable of displaying a detailed screen by adding the basic component. Encoder 4E2 and both encoders 4E1, 4E
Multiplexing unit 28 for multiplexing the encoded data output from the second unit 2, the transmission path 6 for transmitting the multiplexed data, a separating unit 30 for separating the transmitted signal, and a decoder for decoding the basic component 32D1, a decoder 32D2 for decoding the extended component, a combining unit 34 for combining the basic component and the extended component, and the multi-screen output device 16.

On the receiving side, a reduced screen is output by reproducing only the basic component, thereby realizing multi-screen reproduction. When a full-size screen is output for a specific screen, a result obtained by adding an extended component to a basic component is used.

[0019]

However, in the system (1), there is a problem that the reproducing apparatus becomes complicated and large. In the systems (2) and (3), the function of multi-screen reproduction on the receiving side must be made dependent on the function of the encoder on the transmitting side. Therefore, there is a problem that communication procedures (protocols) and a system configuration are complicated, which leads to an increase in cost.

The present invention has been made to solve the above problems, and has as its object to provide a video reproducing apparatus capable of displaying a reduced screen with a simple configuration.

[0021]

According to a first aspect of the present invention, there is provided an image reproducing apparatus, comprising: an input unit for inputting data for displaying an image composed of a plurality of pixels; Computing means for computing data for displaying some of the plurality of pixels based on the calculation, wherein the computing means is a pixel that is included in the plurality of pixels and is a pixel other than some of the pixels It is characterized by omitting the calculation for.

According to the present invention, the operation relating to pixels other than some of the displayed pixels is omitted, so that it is possible to efficiently reproduce a video.

More preferably, the video reproducing apparatus includes a display means for displaying a plurality of reduced images on one screen based on the data calculated by the calculation means.

According to the present invention, since a plurality of reduced images are displayed on one screen, it is possible to provide a user-friendly image reproducing apparatus.

More preferably, the calculating means includes a motion compensating means for compensating for motion by supplementing pixels for which the calculation has been omitted.

According to the present invention, motion compensation is performed by supplementing pixels for which calculations have been omitted, so that effective image compression and reproduction can be achieved.

More preferably, the data for displaying an image composed of a plurality of pixels is data according to the MPEG system.

[0028]

FIG. 1 is a diagram showing an environment in which a decoder (video reproduction device) 1 according to a first embodiment of the present invention is used.

Referring to the figure, decoder 1 is connected to transmission line 6. The decoder 2 is connected to a display 2 capable of performing multi-screen display. The encoders 4a to 4c are connected to the transmission path 6. Cameras 3a to 3c are connected to encoders 4a to 4c, respectively. Transmission line 6 also has
An on-demand video server 5 is connected.

The decoder 1 decodes a desired one of the image data output from the cameras 3a to 3c and the video server 5. The display 2 displays a plurality of decoded images 2a to 2d as multiple screens. In addition, one screen can be displayed on the display 2 as a full size screen according to an instruction from the user.

FIG. 2 is a block diagram showing the structure of the decoder 1. This decoder includes a controller 101 and an operation pixel controller 103 in addition to the configuration of the decoder shown in FIG. The controller 101 sends the channel information to the separator 105, and instructs the separator 105 to separate the image data of the desired channel. The controller 101 controls the variable-length code decoder 10
7 is transmitted. Further, the controller 101 informs the calculation pixel controller 103 of the screen size to be displayed.

The operation pixel controller 103 includes an inverse DCT unit 11
3, a signal for omitting unnecessary calculations (a signal for limiting calculation pixels) is sent. Further, the operation pixel controller 103 sends a control signal to the motion compensation unit 115 so as to appropriately perform motion compensation even when the operation pixel is controlled.

In multi-screen display, it is necessary to reduce and display full-size screen data sent via a transmission path. As shown in FIG. 3, in the present embodiment, a full-size screen shown in (A), a screen whose area ratio is 1/4 times that of the full-size screen shown in (B), It is possible to display a screen of an arbitrary size from the 1/16 times screen shown in (C) and the 1/64 times screen shown in (D).

Pixels displayed on the full-size screen are 0 to 8 in the x direction and 0 in the y direction as shown in FIG.
8 × 8 blocks. (B)
In the display of the 1 / 4x screen shown in
X = 0, 2, 4, 6 and y = 0,
Only pixels at positions 2, 4, and 6 are displayed. (C)
In the display of the 1/16 times screen shown in FIG. 7, only the pixels at the positions of x = 0,4 and y = 0,4 in the 8.times.8 block are displayed. 1/64 shown in (D)
In the display of the double screen, of the 8 × 8 blocks, x
Only pixels at the positions of = 0 and y = 0 are displayed.

This embodiment is characterized in that pixels which are not related to the display of the reduced screen in the screens of (B), (C) or (D) are omitted from the object of the inverse DCT operation. The specific processing will be described below.

First, referring to FIG. 4, a two-dimensional 8-point inverse D
In the CT operation processing, the input DCT coefficient block is defined as F (u, v), and the output block is defined as f (x,
y), the inverse DCT operation is represented by equation (1-1). Here, by changing the order of the terms for the arguments x and v,
Equation (1-2) can be derived.

Therefore, if the equation (1-3) is defined, the equation (1-1) can be expressed as the equation (1-4).

[0038]

(Equation 1)

Here, equation (1-3) shows the one-dimensional inverse DCT transform in the horizontal direction, and equation (1-4) shows the one-dimensional inverse DCT transform in the vertical direction. As described above, the two-dimensional inverse DCT can be divided into one-dimensional inverse DCT in the horizontal and vertical directions.

That is, the DC actually input to the decoder
When the T coefficient block is a block of 0 to 7 in the x direction and 0 to 7 in the y direction shown in FIG.
The conversion can be performed by a combination of the horizontal inverse DCT and the vertical inverse DCT by the processing in the flowchart of FIG.

Referring to the figure, a variable y is set to 0 in step S1. In step S2, it is determined whether y <7. If “YES” in the step S2, 1 in a horizontal direction is applied to eight pixels existing in the y-th row in a step S3.
A dimensional inverse DCT operation is performed. The result is step S
At 4 it is returned to the same y row. In step S5, the variable y is incremented by one. Next, the processing from step S2 is repeated.

If NO in step S2, step S
In step 6, the variable x is set to 0. In step S7, it is determined whether x <7. If “YES” in the step S7, a one-dimensional inverse D is performed on the eight pixels existing in the column x in a step S8.
A CT operation is performed. The result is returned to the same x column in step S9. In step S10, the variable x is incremented by one. Thereafter, the processing from step S7 is repeated.

If NO in step S7, one DCT
The inverse DCT transform of the coefficient block ends.

Referring to FIG. 7, in the present embodiment, first, one-dimensional inverse DCT in the horizontal direction is performed in the range of y = 0 to 8 for an input 8 × 8 DCT coefficient block.
Is applied. Next, the one-dimensional inverse DCT in the vertical direction is performed. At this time, the calculation of pixels not involved in display is omitted.

More specifically, in the full-size screen shown in (A), since all the pixels are involved in the display, the vertical one-dimensional inverse DCT is performed in the entire range of x = 0 to 8.

In the １ ／ screen of (B), x = 0, 2,
Since only columns 4 and 6 are pixels involved in display, x =
No operation is performed on the pixels in columns 1, 3, 4, and 7.

In the 1/16 times screen of (C), x = 0,4
Are the pixels involved in display, only x = 1 to
No operation is performed on the pixels in columns 3, 5, and 7.

In the 1/64 times screen of (D), only the column of x = 0 is a pixel involved in the display, and therefore no operation is performed on the pixels of the column of x = 1 to 7.

More specifically, the processing of the inverse DCT performed in the present embodiment is as shown in the flowchart of FIG.

Referring to FIG. 8, steps S11-S19
Are the same as steps S1 to S9 in FIG. 6, but the processing in step S20 is different. In step S20, b = 1 is set for full-size screen display, b = 2 for 1 / 4-times screen display, b = 4 for 1 / 8-times screen display, and 1/64 times. In the case of screen display, b = 8. In step S20, the value of b is added to x.

8 × 8 converted by the processing of FIG.
The pixels required for display are extracted and displayed as shown in FIG.

As described above, by omitting the calculation of the pixels which are not reproduced on the screen due to the reduction, one screen (one channel) can be obtained.
The processing volume per hit can be reduced. This gives 1
Multi-screen reproduction is possible even with one decoder.

Next, the processing performed by the operation pixel controller 103 of FIG. 2 on the motion compensation unit 115 will be described.

In the case of the motion compensation, in the case of the full-size screen, FIG.
As shown in the figure, four 8 × 8 blocks are put together,
This is performed in 16 × 16 block (macroblock) MB units.

Therefore, when a reduced screen is displayed, motion compensation is performed on macroblocks corresponding to the reduction ratio of the screen.

Referring to FIG. 10, motion compensation is a method of encoding a difference between moving images by using the fact that the moving image has a high correlation between the previous and next screens (A) and (B), such as MPEG. Is the predictive coding used for In order to make the difference smaller, a motion vector MV indicating how much the image in the macroblock has moved and the difference between the screen before and after the motion are considered are coded.

At the time of multi-screen reproduction, as shown in FIG. 11, the motion vector is also set to 1/2, 1
/ 4, 1/8 times.

Further, depending on the value of the motion vector, it may be necessary to refer to a pixel that is not displayed by the reduction operation.

For example, as shown in FIG.
When the pixel indicated by the coordinates of 1, 3, and y = 1, 3 is not displayed, and the pixel of F, G, or H must be referred to, the formula (2-1) and the formula (2-
2) or a value obtained by calculating an average value from the pixels existing around those pixels according to Expression (2-3) is defined as F, G, or H pixel data.

F = (B + C + D + E) / 4 (2-1) G = (B + C) / 2 (2-2) H = (B + D) / 2 (2-3) FIG. FIG. 14 is a block diagram illustrating a configuration of a decoder according to a second embodiment.

Referring to the figure, this decoder is provided with a multiplier 125 between inverse quantization section 111 and inverse DCT section 113 of the decoder shown in FIG. The multiplier 125 multiplies the 8 × 8 block output from the inverse quantization unit 111 with the filter matrix output from the calculation pixel controller 103 according to the displayed screen size. As a result, the high-frequency portion of the DCT coefficient output from the inverse quantization unit 111 can be arbitrarily deleted.

8 × 8 output from inverse quantization section 111
When a full-size screen is displayed for the DCT coefficient block, the filter matrix of Expression (3-1) is multiplied. When displaying a 1/4 screen, the filter matrix of Expression (3-2) is multiplied. When displaying the 1/16 times screen, the filter matrix of Expression (3-3) is multiplied. When displaying the 1 / 64x screen, the filter matrix of Expression (3-4) is multiplied.

[0063]

(Equation 2)

[0064]

(Equation 3)

According to this filter matrix, an 8 × 8 DCT coefficient block for displaying a full size screen is shown in FIG.
4, the signal is input to the inverse DCT unit 113 while being output from the inverse quantization unit 111.

As shown in FIG. 15, the DCT coefficient block in the case of displaying the 1/4 magnification screen is cleared to 0 except for the low frequency part of 4 × 4 at the upper left.

As shown in FIG. 16, the DCT coefficient block in the case of displaying a 1/16 times screen is a 2 × 2
Are cleared to 0 except for the low-frequency part of.

As shown in FIG. 17, the DCT coefficient block for displaying the 1 / 64x screen is cleared to 0 except for the lower left 1 × 1 low frequency portion.

These processes are processes for minimizing an error caused by partially omitting a horizontal operation process performed later.

By these processes, the horizontal one-dimensional inverse DCT and the vertical one-dimensional inverse DCT shown in FIG. 18 are executed on the 8 × 8 DCT coefficient block in which only the low-frequency component is present. You.

In the one-dimensional inverse DCT in the horizontal direction, the inverse DCT is applied to all rows of y = 0 to 7 in the full-size screen of FIG.

In the 1/4 screen of (B), inverse DCT is performed only on the rows of y = 0 to 3. In the 1/16 times screen of (C), inverse DCT is performed only in the rows of y = 0 to 1.
In the 1/64 times screen of (D), inverse DC is applied only to the row of y = 0.
T is given.

The DCT coefficient block subjected to the one-dimensional inverse DCT in the horizontal direction is then subjected to the inverse DCT in the vertical direction.
T is performed, but since this is the same as the processing in the first embodiment shown in FIG. 7, the description will not be repeated here.

FIG. 19 shows the above-described inverse DCT in the horizontal direction.
6 is a flowchart showing processing in DCT and inverse DCT.

Step S101 of this flowchart,
Steps S103 to S110 correspond to steps S11 and S13 to S20 in the flowchart shown in FIG. In step S102, it is determined whether the variable y has exceeded a.

The value of a is 7 for a full size screen, 3 for a 1/4 time screen, 1 for a 1/16 time screen, and 1/64 time for a 1/16 time screen. Is 0 for.

According to the second embodiment, the number of operations on a 1 / 64-times screen is 1/8 that of the prior art. As a result, a single decoder can realize a sufficiently multi-screen display.

Next, an error caused by decoding according to the present embodiment will be described. Referring to FIG. 20, pixels to be decoded are composed of frames from F1 to F13, and these frames are composed of I0, B0, B1, P0, B2, B
Assume that the order is 3, P1, B4, B5, P2, B6, B7 and I1. Here, I, P, and B indicate an I picture, a P picture, and a B picture in the MPEG system, respectively. The order of transmission of these data is
I0, P0, B0, B1, P1, B2, B3, P2, B
4, B5, I1, B6, and B7.

FIG. 21 shows a case where a reduced screen is created according to the first embodiment from the data shown in FIG.
FIG. 9 is a graph showing a difference (Δx) between luminance values of pixels when a reduced screen is created by the G method and an average square error thereof. FIG. Note that the reduced screen according to the conventional MPEG method refers to a reduced screen created by reproducing a full size screen according to the MPEG method and thinning the screen.

In FIG. 21, the error becomes 0 in the I picture, and the error is accumulated with time. However, the error is reset to 0 at the next I picture to appear. As a result, an error with a screen reproduced by the method of the related art can be suppressed to about 6% for a 1 / 16-times screen and up to about 10% for a 1/64 screen. As described above, the deterioration of the image is not visually recognized.

FIG. 22 is a graph showing an error in the second embodiment, and is a graph corresponding to FIG.

In the second embodiment, since the filtering is performed, an error occurs in the operation on the I picture, but the error falls within the range of the error in the first embodiment. Therefore, even in the second embodiment, it is possible to reproduce a screen having no particular problem visually.

As described above, according to the present invention, multi-screen display can be performed without depending on an encoder. For example, a signal which is not coded for multi-screen display, such as a signal from a video server, is transmitted to a plurality of channels. Even in a system such as a cable television transmitted to a transmission line, a system capable of simultaneously performing a multi-channel video monitor on multiple screens can be easily constructed.

Further, according to the above embodiment, it is possible to display a maximum of 64 screens in a full size screen.
Moreover, multi-screen display can be realized with one decoder.

Further, according to the present invention, decoding can be performed according to the capability of the terminal, so that hierarchical decoding can be performed without depending on the capability of the encoder.

Further, since only the processing in the decoder is performed efficiently, a complicated communication procedure (protocol) is not required.

Further, in the system shown in FIG.
Since it is necessary to send an instruction from the decoder side to the camera, communication may be delayed. According to the present invention, such communication delay can be eliminated.

[Brief description of the drawings]

FIG. 1 is a diagram illustrating an environment in which a decoder 1 according to a first embodiment of the present invention is used.

FIG. 2 is a block diagram showing a configuration of an encoder 1 of FIG.

FIG. 3 is a diagram for explaining a screen displayed on a display 2 of FIG. 1;

FIG. 4 is a diagram for explaining one-dimensional inverse DCT transform;

FIG. 5 is a diagram showing one block processed by the inverse DCT transform.

FIG. 6 is a flowchart illustrating processing of one-dimensional inverse DCT.

FIG. 7 is a diagram for explaining a portion that performs one-dimensional inverse DCT and a portion that does not perform one-dimensional inverse DCT in the DCT coefficient block.

FIG. 8 shows the inverse D performed by the decoder according to the first embodiment.
It is a flowchart of CT conversion.

FIG. 9 is a diagram illustrating a macro block M serving as a unit for performing motion compensation.
FIG.

FIG. 10 is a diagram for explaining a motion vector MV in motion compensation.

FIG. 11 is a diagram showing a relationship between a displayed screen size and a size of a motion vector.

FIG. 12 is a diagram for explaining processing when data of a pixel to be referred to in motion compensation is not calculated.

FIG. 13 is a block diagram illustrating a configuration of a decoder according to the second embodiment of the present invention.

FIG. 14 is a diagram showing a configuration of a DCT coefficient block output from a multiplier 125 on a full-size screen.

FIG. 15 is a diagram showing a configuration of a DCT coefficient block output from a multiplier 125 on a 1 / 4-times screen.

FIG. 16 is a diagram showing a configuration of a DCT coefficient block output from a multiplier 125 on a 1/16 × screen.

FIG. 17 is a diagram showing a configuration of a DCT coefficient block output from a multiplier 125 on a 1/64 × screen.

FIG. 18 is a diagram for explaining a portion that performs one-dimensional inverse DCT of a DCT coefficient block and a portion that does not perform one-dimensional inverse DCT in the second embodiment.

FIG. 19 is a flowchart illustrating one-dimensional inverse DCT processing according to the second embodiment;

FIG. 20 is a diagram for explaining an image sequence in the MPEG system.

FIG. 21 is a diagram illustrating an error between a video output according to the first embodiment and a video output according to the related art.

FIG. 22 is a diagram illustrating an error between a video output according to the second embodiment and a video output according to the related art.

FIG. 23 is a diagram showing a unit block in the MPEG system.

FIG. 24 is a block diagram showing a configuration of a decoder adopting the MPEG system in the related art.

FIG. 25 is a diagram showing an example of a multi-screen display system using the decoder shown in FIG.

FIG. 26 is a diagram illustrating a configuration of a system that realizes multi-screen display by controlling an encoder on a receiving side.

FIG. 27 is a diagram illustrating a configuration of a system that realizes multi-screen display employing hierarchical coding.

[Explanation of symbols]

 Reference Signs List 1 decoder 2 display 3a-3c camera 4a-4c encoder 6 transmission line 101 controller 103 arithmetic pixel controller 113 inverse DCT unit 115 motion compensation unit 125 multiplier

Claims (4)

[Claims] An input unit for inputting data for displaying an image composed of a plurality of pixels; and data for displaying some of the plurality of pixels based on the input data. An image reproducing apparatus, comprising: an operation unit that performs an operation, wherein the operation unit is configured to omit an operation relating to pixels included in the plurality of pixels and other than the some of the pixels. 2. The video playback device according to claim 1, further comprising a display unit that displays a plurality of reduced images on one screen based on the data calculated by the calculation unit. 3. The video reproducing apparatus according to claim 1, wherein the arithmetic unit includes a motion compensating unit that compensates for a pixel in which the arithmetic operation is omitted and performs motion compensation. 4. The video reproducing apparatus according to claim 1, wherein the data for displaying the video composed of the plurality of pixels is data according to an MPEG system.