JPH11353499A - Image encoding device and image decoding device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image encoding device and image decoding device which can reduce a data transmission quantity per unit time and make small the data processing capability on a reception side. SOLUTION: The image encoding device had 1st and 2nd polygon image memories 16a and 16b which store complete images at time (t) and time (t+α) based on polygon information, an image deformation part 17 which deforms the complete image at the time (t) stored in the 1st polygon image memory 16a to calculate a predicted image at the time (t+α), and a difference signal calculation part 18 which compares the complete image at the time (t+α) stored in the 2nd polygon image memory with the predicted image to calculate the difference between the both as a difference signal. The image decoding device performs reverse operation to display the complete image from the image calculation part at the time (t) and a composite image at the time (t+α).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a method for transmitting image information,
The present invention relates to an image information encoding device and an image information decoding device for receiving.

[0002]

2. Description of the Related Art Three-dimensional computer graphics (3.
In the field of (dimensional CG), when transmitting moving image information from the transmitting side to the receiving side, the transmitting side generates each frame of the moving image from the polygon information and sends each frame to the receiving side, or the transmitting side generates a polygon. The information was sent, and the receiving side generated each frame (frame) of the moving image from the received polygon information. In three-dimensional CG, an object is represented by a shape in which a large number of polygons (for example, triangles) are three-dimensionally attached. The polygon information includes vertex coordinates of these triangles, color information of the triangles, light source information, and the like.

[0003]

Conventionally, when each frame of a moving image is generated from polygon information on the transmission side and each frame is transmitted to the reception side, the amount of data transmission per unit time is very large.

Further, when polygon information is transmitted from the transmitting side and the receiving side generates each frame of the moving image from the received polygon information, the receiving side calculates the contour of the object from the polygon information for each frame, Since the object was colored, it was necessary to increase the data processing capacity of the receiving side.

Accordingly, an object of the present invention is to propose an image encoding device and an image decoding device capable of reducing the amount of data transmission per unit time and reducing the data processing capacity on the receiving side. I do.

[0006]

According to a first aspect of the present invention, there is provided an image coding apparatus, comprising: a coordinate conversion unit for calculating polygon information comprising a polygon-shaped motion vector and a polygon area in a virtual space; And a polygon information memory for storing the polygon information, a polygon information encoding unit for encoding the polygon information, and contouring and coloring the image from the polygon information to obtain a completed image. An image calculation unit for generating, a first polygonal image memory for storing a completed image at time t, and a time t
A second polygonal image memory storing the completed image at + α, and a completed image at time t stored in the first polygonal image memory using the motion vector,
An image transformation unit that calculates a predicted image at time t + α, a difference signal calculation unit that compares the completed image stored in the second polygon image memory with the predicted image, and calculates a difference between the two as a difference signal; An image encoding apparatus, comprising: a difference signal encoding unit that encodes the difference signal.

[0007] An image coding apparatus according to a second aspect of the present invention includes a difference judging unit for judging whether or not the difference amount of the difference signal is within a designated range, wherein the difference amount is designated. If the difference signal is outside the specified range, the difference signal is encoded.If the difference amount is outside the specified range, the coordinate transformation unit newly calculates the polygon-shaped motion vector and the polygon area, and calculates the polygon information memory. Is updated. The image encoding apparatus according to the first aspect, An image compounding apparatus according to a third aspect of the present invention provides polygon information for compounding a signal obtained by encoding polygon information including a polygon-shaped motion vector and a polygon region in a virtual space. A compounding unit, an image calculating unit for contouring and coloring an image from the compounded polygon information, and generating a completed image; a polygon image memory for storing the completed image at time t; An image transformation unit for transforming the completed image at time t stored in the image memory using the motion vector to calculate a predicted image at time t + α; a completed image at time t + α;
+ Α, a composite signal for generating a composite image at time t + α by adding a composite signal that represents the difference between the predicted image and the encoded signal to the composite signal and the differential image. And an image display unit for displaying the completed image from the image calculation unit at time t and displaying the synthesized image at time t + α.

[0008]

FIG. 1 shows an embodiment of an image encoding apparatus according to the present invention. In FIG. 1, 11 is a coordinate conversion unit, 12 is a polygon information memory, 13 is a polygon information encoding unit, 14 is a polygon shape calculation unit, 15 is a luminance calculation unit, 16a is a first polygon image memory, 16b Is a second polygon image memory, 17 is an image transformation unit, 18 is a difference signal calculation unit, 19 is a difference signal encoding unit, 2
0 and 21 are changeover switches, and 22 and 23 are output terminals.

As shown in FIG. 2, a coordinate transformation unit 11 arranges an object represented by combining a large number of polygons (for example, triangles) in a world coordinate system, calculates a viewpoint coordinate system at time t, and The image data is converted to a three-dimensional screen coordinate system by conversion, and finally to a screen (screen) coordinate system to calculate a polygon area. Further, a polygon motion vector is also calculated from the virtual viewpoint velocity vector v at time t using coordinate transformation, and the polygon area and the motion vector are stored in the polygon information memory 12 as polygon information. Here, if the polygon is a triangle, the object can be represented by a large number of small triangles connected in a three-dimensional manner. The polygon information memory 12 stores the following information. (1) Three-dimensional coordinates of vertices Ta, Tb, Tc of each triangle. (2) Three-dimensional motion vectors of vertices Ta, Tb, Tc of each triangle. (3) Material data of each triangle. (4) Texture data of each triangle. (5) Data relating to the light source.

Here, the three-dimensional motion vector may be a specific numerical value or may be represented by a time function V (t).
Here, the texture means a picture or pattern.
In the field of the dimensional CG, in order to perform three-dimensional display, a work of pasting a picture or pattern on a two-dimensional plane onto a three-dimensional curved surface or plane is performed. Mapping a two-dimensional texture onto a three-dimensional surface or the like in this way is called texture mapping.

This operation is performed for all pixels on the curved surface.
This can be done by calculating an appropriate value based on one or more corresponding pixel values on the texture. Also,
The material data is data necessary for determining the surface color, such as the environmental light coefficient and diffused light coefficient of the surface. Here, the ambient light refers to light whose illumination cannot be specified due to repeated reflection of light from the lighting many times, and the ambient light coefficient refers to a coefficient indicating a ratio of reflecting ambient light. Also,
The diffused light is light that is scattered in all directions when the illumination is illuminating the surface, and the diffused light coefficient is a coefficient indicating which color is reflected as diffused light and at what ratio.

Although only the movement of the virtual viewpoint is used in the embodiment, the movement of the polygon itself has a relative relationship to the movement of the virtual viewpoint and may be unified. The polygon shape calculation unit 14 performs contouring based on the above information (1). That is, a polygonal shape (contour) is calculated from the information (1), and information (contour information) of the polygonal shape is calculated.
Is output to the luminance calculator 15.

In the luminance calculating section 15, the above information (3),
Coloring is performed based on (5). That is, the luminance values of all the pixels inside the polygon are calculated based on the distance from the light source based on the information (3) and (5).

When texture mapping is performed, the luminance calculation unit 15 determines the pixel values inside the polygon from the correspondence between the unique coordinate system of the texture data and each vertex based on the above information (4) and (5). Is calculated by sampling from the texture data. The polygon image obtained in this way is transmitted to the polygon image memory 1 via the switch 21.
6a or 16b.

The polygon information encoding unit 13 includes a switch 2
0, the information (1), (2),
(3), (4), and (5) are all encoded, and output 2
2 is transmitted. The image transformation unit 17 transforms the image stored in the polygon image memory 16a into an image after the lapse of time α by using the three-dimensional motion vector as the information (2), and generates an image after the transformation. .

The difference signal calculating section 18 calculates a difference between the image data Q from the second polygonal image memory 16b and the image data P from the image deforming section 17 and outputs it as a difference signal.
The difference signal encoding unit 19 encodes the difference signal and outputs
Sent from

Next, the operation of the image encoding apparatus shown in FIG. 1 will be described. The above-mentioned polygon information (1) to (5) is generated from the coordinate conversion unit 11 and sent to the polygon information memory 12. A plurality of frames can be formed using one piece of polygon information (1) to (5). A trigger signal is sent to the switches 20 and 21 at the timing when these pieces of information (1) to (5) are sent to the polygon information memory 12. The switches 20 and 21 are switched to the position shown by the dotted line for one frame period (α period) by the trigger signal, held, and returned to the position shown by the solid line after the elapse of one frame period.

At time t, if the above-mentioned information (1) to (5) are sent from the coordinate conversion unit 11 to the polygon information memory 12 and stored, the switch 20 is connected between time t and time t + α. 21 is switched to the direction shown by the dotted line, and after time t + α, it is switched to the direction shown by the solid line.

At time t, all of the information (1) to (5) stored in the polygon information memory 12 is sent to the polygon information encoding unit 13, where the encoded information is subjected to a predetermined encoding, and the encoded polygon is encoded. Output information as the first frame information.
2 output.

At time t, the information (1) to (5) are stored in the polygon shape calculator 14 and the brightness calculator 1.
Sent to 5. In the polygon shape calculation unit 14, information (1)
, A polygon shape (contour) is calculated, and information on the polygon shape (contour information) is output to the luminance calculation unit 15. In the brightness calculation unit 15, coloring is performed on the contoured image based on the above information (3) and (5). Alternatively, texture mapping is performed based on the information (4) and (5).

Accordingly, the completed image Q0 (FIG. 3) created at time t is stored in the first polygonal image memory 16a via the switch 21.

Next, at time t + α, switch 2
0 and 21 are returned to the positions shown by the solid lines. The polygon shape calculation unit 14 creates a contour at time t based on the information (1), and substitutes the current time t + α into the time function V (t) of the motion vector obtained based on the information (2) to change The amount V (t + α) is calculated, and the contour at time t is moved according to the amount of change to generate a vector-moved image. Then, in the luminance calculation unit 15, information (3), (4),
Coloring is performed according to (5), and the completed image Q1 (FIG. 3) of the second frame is stored in the second polygon image memory 16b.

At time t + α, the image transformation unit 1
7, the time function V (t) of the motion vector at time t
Is substituted for the current time t + α, the change amount V (t + α) is calculated, and the completed image Q0 (FIG. 3) at the time t stored in the first polygonal image memory 16a is moved according to the change amount. An image P1 (FIG. 3) shifted by a vector is generated.

Further, at time t + α, the difference signal calculating section 18 compares the image Q1 and the image P1 on a pixel-by-pixel basis, and calculates a difference signal. The difference signal encoding unit 19 encodes the difference signal, and outputs the encoded difference signal from the output terminal 23 as second frame information.

Subsequently, at time t + 2α, since the switches 20 and 21 are at the positions shown by the solid lines, the polygon shape calculation unit 14 creates a contour at time t based on the information (1) and the information ( The current time t + 2α is substituted into the time function V (t) of the motion vector obtained in 2), the change amount V (t + 2α) is calculated, and the contour at the time t is moved according to the change amount. Generate an image. Then, in the luminance calculation unit 15, information (3),
Coloring is performed by (4) and (5), and the completed image Q2 (FIG. 3) of the third frame is stored in the second polygon image memory 16b.
Is stored.

At time t + 2α, the image transformation unit 17 calculates the time function V of the motion vector at time t.
The current time t + 2α is substituted for (t), and the change amount V (t + 2
α) is calculated, and the completed image Q0 (FIG. 3) at the time t stored in the first polygonal image memory 16a is moved according to the amount of change to generate the vector-moved image P2 (FIG. 3).

Further, at time t + 2α, the difference signal calculating section 18 compares the image Q2 with the image P2 on a pixel-by-pixel basis to calculate a difference signal. The difference signal encoding unit 19 encodes the difference signal, and outputs the encoded difference signal from the output end 23 as third frame information. Hereinafter, the frame information of the fourth and subsequent frames is output from the output terminal 23 in the same manner.

Next, the scene changes, and the next one piece of polygon information (1) to (5) is sent to the polygon information memory 12. Then, the first frame information is output from the output terminal 22, and the subsequent frame information is sequentially output from the output terminal 23.

In this way, it is possible to transmit less information than the amount of information of each frame of the moving image. FIG. 4 shows an embodiment of an image compounding apparatus according to the present invention. In FIG. 4, reference numerals 30 and 31 denote input terminals, 32 denotes a polygonal information compounding unit, 33 denotes a polygonal shape calculating unit, 34 denotes a luminance calculating unit, 35 denotes a polygonal image memory, 36 denotes an image deforming unit,
7 is a difference signal combining unit, 38 is a combined image calculating unit, and 39 is an image display unit.

The input terminals 30, 31 are connected to the output terminals 22,
The signal from 23 is received. The encoded polygon information received at the input terminal 30 at the time t is decrypted by the polygon information decryption unit 32, and the above information (1) to (5) is reproduced.

The polygon shape calculator 33 creates an outline at time t based on the information (1), and the luminance calculator 34 applies coloring to the contoured image based on the information (3) and (5). Done. Alternatively, texture mapping is performed based on the information (4) and (5). The reproduced completed image Q0 (FIG. 3) is displayed on the image display unit 39 and stored in the polygon image memory 35.

Next, at time t + α, the encoded difference signal received at the input terminal 31 is combined by the difference signal combining unit 37, and the difference signal is sent to the composite image calculating unit 38.
At time t + α, the image transformation unit 36 substitutes the current time t + α into the time function V (t) of the motion vector at time t, calculates the amount of change V (t + α), and stores it in the polygon image memory 35. The stored completed image Q0 (FIG. 3) at time t is moved according to the amount of change to generate a vector-moved image P1 (FIG. 3). In the combined image calculation unit 38, the difference signal and the image P1 are combined to generate the image Q1. The generated image Q1 is displayed on the image display unit 39.

Further, at time t + 2α, the image transformation unit 36 sets the time function V of the motion vector at time t.
The current time t + 2α is substituted for (t), and the change amount V (t + 2
α) is calculated, and the completed image Q0 (FIG. 3) at the time t stored in the polygon image memory 35 is moved according to the amount of change to generate a vector-moved image P2 (FIG. 3). In the combined image calculation unit 38, the newly received difference signal and the image P2 are combined to generate the image Q2.
The generated image Q2 is displayed on the image display unit 39.

As described above, the processing capability of the multifunction device is relatively small, and a high-quality image can be reproduced with a small amount of information. Modified Example of Image Encoding Device FIG. 5 shows a modified example of the image encoding device according to the present invention. Compared with the image encoding device shown in FIG. 1, the modification of FIG. Difference signal discriminator 41
Receives the difference signal from the difference signal calculation unit 18 and determines whether or not the magnitude of the difference signal is within a predetermined range. If the magnitude of the difference signal is within a predetermined range, the difference signal is sent to the difference signal encoding unit 19 as it is and encoded. If the magnitude of the difference signal is out of the predetermined range, a clear signal is sent to the coordinate conversion unit 11. When the difference signal is large, inconveniences such as an increase in the amount of information to be transmitted and a failure to faithfully reproduce an image occur.

When the difference signal is increased and a clear signal is output, the coordinate conversion unit 11 outputs a new one-time polygon information.
(1) to (5) are sent to the polygon information memory 12. As a result, the magnitude of the difference signal can be reduced.

[0036]

As described above, according to the present invention,
With a small amount of image information, high-quality images can be transmitted and received and reproduced. That is, according to the present invention, the image encoding apparatus does not send a completed image to all frames (frames), but sends a completed image to a certain representative image, and sends a completed image to a representative image. Since the difference from the completed image is sent, the amount of data transmission per unit time can be reduced. Further, in the image compounding apparatus on the receiving side, a completed image is generated from polygon information only for the representative image without generating a completed image from polygon information for all frames, so that the data processing capability is reduced. be able to.

[Brief description of the drawings]

FIG. 1 is a block diagram of an embodiment of an image encoding device according to the present invention.

FIG. 2 is an explanatory diagram relating to a coordinate form in image generation.

FIG. 3 is an explanatory diagram of the operation of the image encoding device in FIG. 1;

FIG. 4 is a block diagram of a modified example of the image encoding device according to the present invention.

FIG. 5 is a block diagram of an embodiment of an image decoding apparatus according to the present invention.

[Explanation of symbols]

 Reference Signs List 11 ... Coordinate conversion unit 12 ... Polygon information memory 13 ... Polygon information encoding unit 14 ... Polygon shape calculation unit 15 ... Luminance calculation unit 16a ... First polygon image memory 16b ... Second polygon image memory 17 ... Image Deformation unit 18 ... Difference signal calculation unit 19 ... Difference signal encoding unit 20, 21 ... Changeover switch 22, 23 ... Output terminal 30, 31 ... Input terminal 32 ... Polygon information combining unit 33 ... Polygon shape calculation unit 34 ... Brightness calculation unit 35 Polygon image memory 36 Image deformation unit 37 Difference signal compounding unit 38 Composite image calculation unit 39 Image display unit

──────────────────────────────────────────────────続 き Continuation of the front page (51) Int.Cl. ⁶ Identification symbol FI // H03M 7/30 H04N 7/13 Z (72) Inventor Yoshimori Nakase 1006 Odakadoma, Kadoma-shi, Osaka Matsushita Electric Industrial Co., Ltd. (72) Inventor Akira Yoneyama 1006 Kadoma Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd.

Claims (3)

[Claims] 1. A coordinate conversion unit (11) for calculating polygon information comprising a polygon-shaped motion vector and a polygon area in a virtual space; and a polygon information memory (12) for storing the polygon information.
And a polygon information encoding unit (1) encoding the polygon information.
3) and contouring and coloring the image from the polygon information;
An image calculator (14, 15) for generating a completed image; a first polygonal image memory (16a) for storing the completed image at time t; and a second polygonal image memory (16b) for storing the completed image at time t + α. ), And the time stored in the first polygonal image memory (16a)
an image transformation unit (17) that transforms the completed image at t using the motion vector to calculate a predicted image at time t + α; and a completed image stored in the second polygonal image memory; An image coding apparatus comprising: a difference signal calculation unit (18) for comparing and calculating a difference between the two as a difference signal; and a difference signal coding unit (19) for coding the difference signal. 2. A difference judging section (41) for judging whether the difference amount of the difference signal is within a designated range or not, and if the difference amount is within a designated range, sign the difference signal. If the difference is out of the specified range, the coordinate conversion unit newly calculates the polygon-shaped motion vector and the polygon area, and updates the polygon information memory. 2. The image encoding device according to 1. 3. A polygon information compounding unit (32) for compounding a signal obtained by coding polygon information comprising a polygon-shaped motion vector and a polygon area in a virtual space, Image calculation units (33, 34) for contouring and coloring the image from the polygon information thus generated and for generating a completed image
A polygon image memory (35) for storing a completed image at time t; and a completed image at time t stored in the polygon image memory (35), using the motion vector,
Image transformation unit (36) for calculating a predicted image at time t + α
Adding a difference signal representing the difference between the completed image at time t + α and the predicted image at time t + α, and further combining a coded signal with a composite unit (37) and the predicted image and the difference signal. A composite image calculation unit (38) for generating a composite image at time t + α, and an image display unit (39) for displaying a completed image from the image calculation unit at time t and displaying the composite image at time t + α. An image compounding apparatus, characterized in that: