JP2009284298A - Moving image encoding apparatus, moving image decoding apparatus, moving image encoding method and moving image decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make it possible to utilize a plurality of models or to flexibly change kinds of the models in a model-based encoding/decoding process. <P>SOLUTION: A model-based prediction system includes a model data storage section 122 for storing model data of a plurality of models to be used for model-based prediction and a model matching section 123 for deforming a plurality of model data items while using a predetermined parameter and for selecting a model optimal for an input image, and generates a model-based prediction image. The model data include model names, element types, element data, attribute data sets and model parameter initial values, and an encoded stream 106 stores in the model-based prediction image information indicative of the model used for prediction and the parameter used for deforming the model. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a moving image encoding device, a moving image decoding device, a moving image encoding method, and a moving image decoding method that encode and decode a moving image signal with high efficiency.

  As a conventional moving image coding technique, the amount of redundant information is reduced by using the correlation between image pictures in the time direction and the correlation between pixels in the picture, and variable length coding is performed. Thus, an encoding method for compressing the amount of information has been put into practical use. Such an encoding method (referred to herein as a “waveform encoding method”) has an advantage that the encoding target is not limited, but has a limit on the compression rate of the information amount.

  On the other hand, a model-based encoding method using a three-dimensional model to be encoded has been proposed. In this method, since the prediction image to be encoded can be expressed by the parameters of the three-dimensional model, the compression rate can be significantly improved as compared with the waveform encoding method.

  Patent Document 1 describes a hybrid encoding method in which waveform encoding and model-based encoding are combined and these encoding methods are selected according to the encoding target block. Further, Patent Document 2 describes a model-based encoding method configured to extract a target region by generating a two-dimensional template based on a three-dimensional model.

JP-A-3-253190 JP 2004-320799 A

  The model-based encoding method has become more feasible due to recent improvements in computer processing speed, image recognition, analysis, and computer graphics. In other words, it has become technically possible to construct an image close to the encoding target image by analyzing the encoding target image, selecting an appropriate model, and adjusting the model parameters. ing. If an image close to the input image can be generated from the model, the input image can be decoded only by the difference information between the generated image obtained from the model and the input image and the parameter information of the model, so that a large amount of information can be expected to be compressed.

  However, in conventional model-based coding as described in Patent Documents 1 and 2 and hybrid coding using the same, data (model data) relating to a three-dimensional model used by the coding device and the decoding device. Is pre-shared. Therefore, there is a problem that the encoding target that can make use of the advantages of model-based encoding is limited to a single shared model.

  In view of the above-described problems, an object of the present invention is a video encoding / decoding device that can use a plurality of models and flexibly change model types in model-based encoding / decoding. And to realize the method.

  In order to achieve the above object, a video encoding apparatus according to the present invention includes a model-based prediction system for generating a model-based prediction image as a prediction image, and the model-based prediction system includes a plurality of models used for model-based prediction. A model data storage unit that stores model data, a model matching unit that modifies a plurality of model data using predetermined parameters and selects an optimal model for an input image, and projects the deformed model data on a two-dimensional plane Alternatively, a two-dimensional image synthesis unit that synthesizes a two-dimensional model image by cutting along a two-dimensional plane, and a model-based motion prediction unit that generates a model-based motion prediction vector and a model-based prediction image of an input image with reference to the two-dimensional model image And have.

  Here, the model data includes a model name, an element type, element data, an attribute data set, and model parameter initial values, and model operation rules based on parameters are defined in advance.

  The encoded stream to be output includes information on the prediction difference image, information indicating the model used for prediction in the model-based prediction image, parameters used for model deformation, a model-based motion prediction vector, and a two-dimensional plane. Stores information about

  Furthermore, a model list used for model-based prediction is presented, and a device control unit that obtains model data of a necessary model in the list from the outside is provided.

  In addition, a model control unit that converts the hierarchical level of the model data used for model-based prediction according to the resolution of the input image, the model control unit converts the size and number of components of the model data according to the hierarchical level, and changes the attribute value Perform conversion.

  A moving picture decoding apparatus according to the present invention includes a model-based prediction system for generating a model-based prediction image as a prediction image, and the model-based prediction system stores model data of a plurality of models used for model-based prediction. Based on the data storage unit, the information indicating the model used for prediction obtained from the encoded stream and the parameters used for the model deformation, the model deforming unit for deforming the corresponding model data, and 2 acquired from the encoded stream Based on information about the two-dimensional plane, a two-dimensional image synthesis unit that synthesizes a two-dimensional model image by projecting on the two-dimensional plane or cutting along the two-dimensional plane, and a model-based motion obtained from the two-dimensional model image and the encoded stream A model-based motion compensation unit that generates a model-based prediction image using the prediction vector.

  In order to generate a model-based prediction image as a prediction image, the moving image encoding method according to the present invention stores model data of a plurality of models used for model-based prediction, and uses the plurality of model data using predetermined parameters. Select the model that is most suitable for the input image and deform it, project the deformed model data on the 2D plane, or cut the 2D plane and synthesize the 2D model image, refer to the 2D model image and input image Model-based motion prediction vectors and model-based prediction images are generated.

  The moving image decoding method according to the present invention stores model data of a plurality of models used for model-based prediction in order to generate a model-based predicted image as a predicted image, and used the prediction obtained from the encoded stream. Based on the information indicating the model and the parameters used for the deformation of the model, the corresponding model data is deformed and projected on the two-dimensional plane or based on the information about the two-dimensional plane acquired from the encoded stream. The two-dimensional model image is synthesized by cutting, and a model-based prediction image is generated using the two-dimensional model image and the model-based motion prediction vector acquired from the encoded stream.

  According to the present invention, a model effective for improving the encoding efficiency can be appropriately selected and used according to the encoding target, so that the compression rate at the time of encoding can be greatly improved for various input images. effective.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The first embodiment relates to a moving picture coding apparatus according to the present invention, and performs hybrid coding using a combination of waveform coding and model-based coding using a plurality of models. The moving image encoding apparatus according to the present embodiment can be applied not only to a general moving image in which frames are arranged in the time direction but also to a three-dimensional image in an image diagnosis apparatus such as X-ray CT or MRI. This is noted in advance. Model-based encoding is particularly effective when the imaging target is limited, such as a TV conference or a medical image diagnostic apparatus.

FIG. 1 is a block diagram showing an embodiment of a moving picture coding apparatus according to the present invention.
The video encoding apparatus 100 includes a subtractor 102 that generates a prediction difference image 117 between the prediction image 116 stored in the prediction image storage unit 114 and the input image 101, and a conversion that performs orthogonal transformation such as DCT on the prediction difference image 117. Unit 103, quantization unit 104 that quantizes the converted signal, and variable length encoding unit 105 that encodes the quantized signal, and outputs encoded stream 106.

  The moving image encoding apparatus 100 according to the present embodiment has three types of prediction processing systems in order to generate the predicted image 116 described above. The first system is based on inter-screen prediction, and in order to obtain a reference image for the next input image, an inverse quantization unit 107 that inversely quantizes the quantized signal output from the quantization unit 104, and inverse quantization An inverse transform unit 108 that inversely transforms the signal to obtain a prediction difference image, an adder 109 that adds the prediction difference image after the inverse transformation and the prediction image from the prediction image storage unit 114, and block noise is removed from the image after the addition A deblocking processing unit 110 for obtaining a reference image is included. A reference image storage unit 111 that stores the obtained reference image, and a motion prediction unit 112 that performs motion prediction between the reference image and the input image 101 are included. The second system is based on intra-screen prediction, and has an intra-screen prediction unit 118 that performs intra-screen prediction from the input image 101. The third system is based on model-based prediction, and includes a model-based image generation unit 120 and a model-based motion prediction unit 125. Model-based prediction will be described later.

  In the predicted image comparison unit 113, the above three prediction processing systems, that is, the inter-screen prediction image from the motion prediction unit 112, the intra-screen prediction image from the intra-screen prediction unit 118, and the model base from the model base motion prediction unit 125 are used. A prediction image estimated to have the highest prediction efficiency is selected from the prediction images. As an index of prediction efficiency, for example, prediction error energy can be cited, but other than that, a prediction image is taken into consideration such as similarity to a prediction method of neighboring blocks (inter-screen prediction, intra-screen prediction or model-based prediction). (That is, the prediction method) may be selected.

The selected prediction image is subjected to a low-pass filter at the block boundary of the prediction image by the filter processing unit 113 as necessary. This is because block noise tends to be conspicuous if the prediction method is different on both sides of the block boundary. Filter processing may be performed not only when the prediction method is different but also when the model used in model-based prediction is different. The final predicted image 116 subjected to the filter process is stored in the predicted image storage unit 115 and used to generate a predicted difference image 117 from the input image 101.
Note that the information 126 related to the prediction method selected by the predicted image comparison unit 113 is sent to the variable length coding unit 105 and stored in a part of the coded stream 106.

  This embodiment is characterized by the configuration of model-based prediction, which is the third system, and will be described in detail below. The model base image generation unit 120 includes a model control unit 121, a model data storage unit 122, a model matching unit 123, and a two-dimensional image synthesis unit 124. The model control unit 121 controls the operation of each unit. The model data storage unit 122 stores model data used for model-based prediction. The model data to be stored is determined according to the encoding target, and stores all necessary model data. If the necessary model data does not exist, a message indicating that the model data does not exist can be notified and acquired from the outside.

  The model matching unit 123 changes the model data by changing the value of a predetermined model parameter. The two-dimensional image synthesis unit 124 projects the deformed model data on a two-dimensional plane or cuts the two-dimensional plane to synthesize a two-dimensional plane model image. The combined model image is compared with the input image 101 by the model matching unit 123. The model matching unit 123 extracts deformation information by comparison, updates the model parameter value again, and deforms the model data. By repeating such processing, an optimal model parameter for the input image 101 and a two-dimensional model image when the optimal model parameter is applied to the model are generated.

An example of such model parameter extraction is reported in Reference Document 1, for example. In Reference Document 1, it is assumed that one face model is used. However, the model matching unit 123 according to the present embodiment performs the same model parameter extraction process on a plurality of models stored in the model data storage unit 122 to obtain the optimum model. It is possible to select a simple model.
[Reference Document 1] Kaneko, Hatori, Koike: “Detection of shape change and encoding of moving face image based on three-dimensional model”, IEICE Transactions (B), J71-B, pp. 11-27. 1554-1556 (December 1988).

  The model-based motion prediction unit 125 regards the two-dimensional model image obtained as described above as a reference image, and performs motion prediction processing from the input image 101. Regarding this motion prediction algorithm, a known algorithm can be used, and the processing may be shared with the motion prediction unit 112. The model parameter information 127 used for generating the two-dimensional model image is sent from the model base image generation unit 120 to the variable length encoding unit 105 and stored in a part of the encoded stream 106.

  FIG. 2 is a diagram illustrating an example of the structure of model data stored in the model data storage unit 116. Each element of the model data will be described.

  “ID” is a number for identifying the type of model data. “Version” is a number indicating the version of the model data having the same ID, and is used for identifying the updated model data. “Model name” is a character string of a name given to model data. For example, in the case of a human face, a character string indicating the characteristics of model data such as “face (male)”, “face (female)”, “face (adult)”, “face (child)” is desirable.

  The “element type” indicates information related to the types of elements constituting the model data. For example, “triangular patch” or “square patch” is used for modeling a three-dimensional surface, and “voxel” is used for modeling a three-dimensional solid including the interior.

  “Element data” is information relating to individual elements constituting the model data. “Number of element data” represents the number of element data, and there are M elements in the figure. The positions of the M elements on the model are represented by “element coordinate values”. Here, the “element coordinate value” indicates, for example, a set of coordinates of three points constituting each triangular patch surface as an element if the element type is a triangular patch surface. The format of “coordinate values of” may be different.

  The “attribute data set” is a set related to attribute data given to each element. “Number of attribute data” represents the number of sets of attribute data, and there are N attribute data in the figure. Each “attribute data” includes “attribute name”, “attribute type”, and M “attribute values”. The “attribute name” is a character string representing the name of the attribute, and a name representing the characteristics of each attribute such as “color (RGB)”, “brightness”, “reflectance”, and “transparency” is desirable. The “attribute type” is information regarding the type of each attribute value, and is information such as “integer”, “binary number”, “decimal point”. The “attribute value” is an attribute value given to each element, and there are as many values as the number of elements, and this attribute value needs to be associated with the coordinates of each element in the element data.

  The “model parameter initial value” is an initial value of a model parameter for manipulating and transforming model data. “Number of parameters” represents the number of “parameter data”. Each “parameter data” includes “parameter name”, “parameter type”, and “parameter value”.

The “model parameters” may include parameters corresponding to individual models, in addition to parameters common to many models, such as a rotation angle and a scale size. For example, Reference 2 shows an example of creating a face model. Define feature points of each part of the face such as eyes, nose, eyebrows, lips, etc., and move each feature point to move each face. Form. In the above example, “feature point (eye 1)” and “feature point (eye 2)” are given as parameter names, “three-dimensional coordinates” is given as a parameter type, and individual feature point coordinate values are given as numerical values.
[Reference Document 2] Japanese Patent Application Laid-Open No. 2003-44873 When a facial muscle model for changing the facial expression is added to the facial model, length and contraction rate values are given to the individual muscles, and the facial muscle model is determined in advance. It is also possible to perform a deformation operation such as changing the position of the feature point according to the equation of motion. In this way, the initial value of the model parameter corresponding to each model is given, whereby the initial state of the model is defined.

  As in the face example above, the rules for how to deform using model parameters depend on the individual model. For this reason, in the model control unit 121, it is necessary to manage information about which model of the ID the model base image generation unit 120 can handle, including deformation and operation processing.

In addition, the model data may include “Description of model data”, “Size” information of the model used when downloading the model data, and the like.
The model data storage unit 122 stores a plurality of model data as described above.

  FIG. 3 is a diagram illustrating the configuration of the encoded stream 106 output from the moving image encoding apparatus 100 according to the present embodiment.

  (A) shows the overall structure of a general encoded stream 106, which is composed of sequence header information 1061 relating to the entire stream, a picture header 1062 which is information adapted to each picture unit, and each picture data 1063. . Hereinafter, a storage example of information related to the model-based encoding of the present embodiment will be shown.

  (B) shows a sequence header 1061 in which model set information relating to a model used for encoding a stream is stored. The model set information includes the number of models used for encoding and each model information. The model information includes “ID”, “Version”, “URL”, and “hierarchy level” for identifying the model. “ID” and “Version” are the same as the contents of the model data described in FIG. “URL” is an address on the network for acquiring model data. As will be described later, “URL” is used to acquire model data as necessary during decoding. The “hierarchy level” will be described later in the third embodiment.

  (C) shows a picture header 1062, which stores the “number of models” used to generate the above-described model prediction image and model-based prediction parameters when encoding individual picture images. The prediction parameters are “ID” of each model, “picture plane information” indicating a projection plane or a cut plane when a model image of a two-dimensional plane is synthesized from the model, “projection information” indicating whether projection or cutting, and Includes “model parameter values”.

  “Picture plane information” is a coefficient (a, b, c, d) when a projection plane or a cut plane for obtaining a two-dimensional plane model image is expressed as, for example, ax + by + cz + d = 0. “Projection information” is whether the three-dimensional model is projected or cut with respect to the above-described two-dimensional plane, and if it is projected, it is a parallel projection (projection by parallel rays) or a perspective projection (projection by radiation rays), It is information about etc. If the projection is a perspective projection, the coordinates of the position of the light source of the radiation beam are also stored.

  The “model parameter value” is a model parameter value when the model matching unit 123 and the two-dimensional image synthesis unit 124 finally create a model-based predicted image.

  (D) shows picture data 1063, in which encoded data of a picture is stored. In general moving picture coding, a picture is converted or quantized in units of 16 × 16 pixel blocks called macroblocks (indicated as MB in FIG. 3), and a prediction method, motion vector, code Stored data as macroblock data.

  Each macroblock data includes an “MB type” indicating a motion prediction method. “MB type” indicates which prediction method (intra-screen prediction, inter-screen prediction, or model-based prediction) is selected by the predicted image comparison unit 113 when a predicted image of the macroblock is generated.

  If “MB type” indicates that the model-based prediction method is used, “ID” indicating which model is used for the prediction, and “model” indicating the motion vector when the model-based prediction image is generated. “Base motion vector”, “difference image information” obtained by converting and quantizing the difference between the model base prediction image and the input image is stored.

  In the present embodiment, information related to model-based prediction is stored in the encoded stream 106 with the configuration described above and transmitted.

  Next, FIG. 4 is a block diagram showing another embodiment of the moving picture coding apparatus according to the present invention. The moving picture coding apparatus 150 according to the present embodiment includes the configuration of the moving picture coding apparatus 100 of FIG. 1 as a moving picture coding section 100 ′, a user interface section 151, a coding apparatus control section 152, a code A model model database unit 153. The encoding model database unit 153 manages model data that can be used in the moving image encoding apparatus 150. Furthermore, the encoding device control unit 152 connects to an external model data server 170 via the network 160 and acquires necessary model data. With these additional configurations, functions for selecting a model used for encoding and setting model data in the moving image encoding unit 100 ′ are realized.

  In this embodiment, model data is set according to the following procedure. The encoding device control unit 152 accesses the model data server 170 via the network 160 and requests a list of encoding models by a message 161. The model data server 170 returns a model list that can be distributed by the message 162 to the encoding device control unit 152. This model list stores at least the model ID, version, and model name.

  The encoding device control unit 152 checks whether or not each model in the received model list is stored in the encoding model database unit 153. Then, the list of the model list and whether each model has already been stored in the moving image encoding device 150 is notified to the operator via the user interface unit 151.

  FIG. 5 shows an example of a model list notification screen, which displays the model name and whether or not it has been downloaded. While referring to this screen, the operator selects an appropriate model to be used for the input image to be encoded.

  If the selected model does not exist in the moving image encoding apparatus 150, the encoding apparatus control unit 152 requests the model data server 170 for data of the selected model using a message 161. This message 161 stores the ID of the requested model. The model data server 170 transmits the requested model data by a message 162.

  The encoding device control unit 152 stores the received model data in the model database unit 153. Alternatively, the model data is stored in a dynamic memory area managed by the video encoding device 150, and the memory address is set in the video encoding unit 100 '. The memory address of this model data is passed to the model control unit 121 in the moving image encoding unit 100 '. The model control unit 121 copies model data to the model data storage unit 122 with reference to the memory address. Alternatively, the model data may be obtained directly with reference to the memory address.

  As described above, the moving image encoding unit 100 ′ can use the model data specified by the moving image encoding device 150 for internal encoding.

  In the above example, an example is shown in which the operator selects a model via the user interface unit 151. However, depending on the use of the video encoding device 150, the model is automatically selected by providing information on the encoding target. It is also possible to set the encoding device control unit 152 to do so.

  For example, in the case of an application such as an image diagnostic apparatus, the part to be imaged, the age and sex of a patient are often separately managed. Therefore, by setting a rule for associating such information with the model to be used in the encoding device control unit 152 in advance, the encoding device control unit 152 automatically selects the model without the operator selecting the model. Is possible.

  As described above, in the moving picture encoding apparatus according to the present embodiment, a model can be selected according to the encoding target, and this model data can be acquired from the outside during encoding. Therefore, model-based encoding can be widely applied to various encoding targets, and there is an effect of greatly improving the compression rate of moving image encoding.

  Embodiment 2 relates to a moving picture decoding apparatus according to the present invention, and decodes an encoded stream generated by the moving picture encoding apparatus of Embodiment 1. At that time, model data necessary for decoding is acquired in advance.

FIG. 6 is a block diagram showing an embodiment of the moving picture decoding apparatus according to the present invention.
The video decoding apparatus 200 includes a variable length decoding unit 202 that decodes a variable length code of the encoded stream 201, an inverse quantization unit 203 that inversely quantizes the decoded signal, and an inversely quantized signal. An inverse transform unit 204 that performs orthogonal transform, an adder 205 that adds the predicted image 215 stored in the predicted image storage unit 214 and the prediction difference image 216 output from the inverse transform unit 204, and a block for the added image A deblocking processing unit 206 that performs low-pass filter processing at the boundary and outputs an output image 207.

  The moving picture decoding apparatus 200 according to the present embodiment has three types of prediction processing systems in order to generate the predicted image 215 described above. The first system is based on inter-screen prediction. The output image storage unit 208 that sequentially stores the output image 207 in units of macroblocks, and the image data of all macroblocks stored in the output image storage unit 208 refer to the next picture. A reference image storage unit 209 separately stored as an image, and a motion compensation unit 210 that performs motion compensation on the reference image using the motion vector decoded by the variable length decoding unit 202 and obtains an inter-screen prediction image . The second system is based on intra-screen prediction, and includes an intra-screen prediction unit 211 that performs intra-screen prediction using image data of processed macroblocks stored in the output image storage unit 208. The third system is based on model-based prediction, and includes a model-based image generation unit 220 and a model-based motion compensation unit 225. Model-based decoding will be described later.

  The above three prediction processing systems, that is, the inter-screen prediction image from the motion compensation unit 210, the intra-screen prediction image from the intra-screen prediction unit 211, and the model-based prediction image from the model-based motion compensation unit 225 are predicted images. It is passed to the generation unit 212. The prediction image generation unit 212 has a switch function that receives the “MB type” information 226 for each macroblock from the variable length decoding unit 202 and switches the prediction image for each macroblock. That is, if the prediction method indicated by “MB type” is inter-screen prediction, the prediction image from the motion compensation unit 210 is transferred to the filter processing unit 213. If the prediction method is intra prediction, the prediction image from the intra prediction unit 211 is transferred to the filter processing unit 213. If the prediction method is model-based prediction, the prediction image from the model-based motion compensation unit 225 is transferred to the filter processing unit 213.

  The filter processing unit 213 applies a low-pass filter to the predicted image pixel at the macroblock boundary of the predicted image as necessary. This is because block noise tends to be noticeable at macroblock boundaries with different prediction schemes, and is therefore removed. Further, even when the both sides of the boundary are model-based prediction methods, even when the models used for prediction are different, the filtering process may be performed similarly.

  The predicted image 215 after the filter processing is stored in the predicted image storage unit 214 and used for adding to the predicted difference image 216. Since the filter processing unit 213 performs filtering at the macroblock boundary, the predicted image data already stored in the predicted image storage unit 214 may be acquired and changed as necessary.

  This embodiment is characterized by the configuration of model-based prediction, which is the third system, and will be described in detail below. The model base image generation unit 220 includes a model control unit 221, a model data storage unit 222, a model deformation unit 223, and a two-dimensional image synthesis unit 224. The model control unit 221 controls the operation of each unit. The model data storage unit 222 stores model data used for model-based prediction, but stores all necessary model data because the model data to be used is switched according to the decoding target. If the necessary model data does not exist, a message indicating that the model data does not exist can be notified and acquired from the outside.

  The variable length decoding unit 202 analyzes the encoded stream 201 and passes information 228 necessary for decoding to the model control unit 221. First, the sequence header 1061 (see FIG. 3) in the encoded stream 201 is analyzed, and model set information necessary for decoding is sent. The model control unit 221 confirms that the model data necessary for decoding the encoded stream 201 exists in the model data storage unit 222 in advance, and performs the decoding process on each picture.

  Also, the picture header 1062 (see FIG. 3) in the encoded stream 201 is read, and model-based prediction parameters necessary for decoding this picture are sent to the model control unit 221. Of the model-based prediction parameters, picture plane information and projection information are sent to the two-dimensional image synthesis unit 224, and model parameter values are sent to the model transformation unit 223.

  Next, the picture data 1063 (see FIG. 3) in the encoded stream 201 is read, and each macroblock is decoded. When the “MB type” indicating the prediction method is model-based prediction, “ID” and “model-based motion vector” are decoded. “ID” is sent to the model control unit 221 and “model base motion vector” is sent to the model base motion compensation unit 225 (reference numeral 227). Here, “ID” is a number indicating a model used for macroblock prediction. The difference image information is restored to the predicted difference image 216 via the inverse quantization unit 203 and the inverse transform unit 204. Note that when the “MB type” is a method other than model-based prediction (intra-screen prediction or inter-screen prediction), the description is omitted because it is a conventional decoding method.

  The model control unit 221 passes the “ID” to the model deformation unit 223. The model deforming unit 223 acquires model data of the model specified by “ID” from the model data storage unit 222, and deforms the model using the already acquired “model parameter value”. The model transformation rules are specific to the model, and are defined by international standards, industry standards, operational rules, etc., and are executed according to these standards.

  The two-dimensional image composition unit 224 obtains a model image of a two-dimensional plane by projecting or cutting the model deformed in this way on a plane designated by “picture plane information”. The “projection information” used here includes information on whether the projection is performed on the plane, is cut on the plane, or is parallel projection or projection projection. Includes position coordinates.

  The obtained two-dimensional model image is passed to the model-based motion compensation unit 225. The model-based motion compensation unit 225 obtains the pixel value of the 16 × 16 block at the position moved by the “model-based motion vector” from the current macroblock position in the model image, so that the model-based prediction at the macroblock position is obtained. Create an image.

  FIG. 7 is a block diagram showing another embodiment of the moving picture decoding apparatus according to the present invention. The moving picture decoding apparatus 250 according to the present embodiment includes the configuration of the moving picture decoding apparatus 200 in FIG. 6 as a moving picture decoding section 200 ′, a user interface section 251, a decoding apparatus control section 252, and a code. A model model database unit 253. The encoding model database unit 253 manages model data that can be used for decoding processing in the video decoding device 250. Further, the decoding device control unit 252 connects to an external model data server 270 via the network 260 and acquires necessary model data. With these additional configurations, a function of acquiring a model necessary for decoding an encoded stream and supplying model data to the moving picture decoding unit 200 'is realized.

  In this embodiment, model data is supplied by the following procedure. When decoding the encoded stream 201, the decoding device control unit 252 analyzes the sequence header and acquires information on which model data is necessary. If the necessary model does not exist in the encoded model database unit 253, the model data is acquired from the external model data server 270. Specifically, the decoding device control unit 252 accesses the model data server 270 via the network 260 and requests summary information of necessary model data by a message 261. The model data server 270 returns the summary information of the model data requested by the message 262 to the decoding device control unit 252. As this summary information, for example, the model name, its description, model data size and the like can be mentioned.

  When the decryption device control unit 252 acquires the summary information of the model data, the decryption device control unit 252 indicates a list of summary information of the model that needs to be newly acquired for decryption to the operator via the user interface unit 251. Provide a screen that prompts the user to select whether or not to acquire.

  FIG. 8 is an example of a model data summary information screen. The operator refers to such a screen, and indicates whether or not to acquire model data from the outside while confirming model information necessary for a stream to be decoded.

  If the model data is acquired from the outside, the decoding device control unit 252 requests the model data server 270 for necessary model data by using the message 261. The message 261 at this time stores the ID of the requested model. The model data server 270 transmits the requested model data by a message 262.

  The decryption device control unit 252 stores the received model data in the model database unit 253. Alternatively, the model data is stored in a dynamic memory area managed by the video decoding device 250, and the memory address is set in the video decoding unit 200 '. The memory address of this model data is passed to the model control unit 221 in the moving image decoding unit 200 '. The model control unit 221 copies model data to the model data storage unit 222 with reference to the memory address. Alternatively, the model data may be obtained directly with reference to the memory address.

  As described above, the moving picture decoding unit 200 ′ can use the model data set by the moving picture decoding apparatus 250 for internal decoding.

  Note that the timing at which the decoding apparatus control unit 252 acquires model data does not necessarily have to be after the encoded stream 201 is analyzed. That is, if the information of the necessary model data set is known in advance, the model data can be acquired before the encoded stream is acquired. For example, if model set information (corresponding to model set information in a sequence header) used for an encoded stream in a program is stored in advance as program information in terrestrial digital broadcasting, necessary model data is acquired in advance. May be.

  In addition, the model data server 270 may be regularly communicated to constantly update the model data to the latest state, or all model data managed there may be acquired. For example, in an image diagnostic apparatus or the like, since the target to be imaged is limited, it is also possible to store all model data corresponding to the imaging region in the model data server 270. The decoding device 250 and the encoding device 150 that use the model data periodically acquire a list of models from the model data server 270, and if there is added or updated model data, each encoded model database You may download to a part beforehand.

  As described above, the moving picture decoding apparatus according to the present embodiment is suitable for a model-based encoded stream encoded using various models by acquiring in advance model data necessary for decoding. It is possible to perform decryption.

  The third embodiment is characterized in that model data used for moving picture encoding and decoding is used after being converted according to a hierarchical level. Here, the hierarchical level is an index indicating the level of resolution set in the same model data.

  The model control unit 121 in the moving image coding apparatus 100 in FIG. 1 stores in advance the correspondence between the resolution of the input image and the hierarchical level of the model data suitable for this. When the resolution of the encoding target image is low and the hierarchical level of the model data is switched to a low level, the model control unit 121 increases the size (area or volume) of the elements constituting the model data. The number of elements is reduced, and attribute values corresponding to the number of elements are recalculated and stored in the model data storage unit 122. In this way, by reducing the hierarchical level of the model data, the processing amount in the model matching unit 123 and the two-dimensional image composition unit 124 can be reduced. When the hierarchical level of the model data is changed, the information is stored in the encoded stream.

  Similarly, the moving picture decoding apparatus 200 in FIG. 6 acquires layer level information from the encoded stream. The model control unit 221 converts the size and number of the constituent elements of the model data and the attribute value according to the hierarchical level information. Thereby, the processing amount in the model deformation | transformation part 223 and the 2-dimensional image synthetic | combination part 224 is also reduced.

  Note that the user may determine whether to lower the model data hierarchy level, and may instruct from the user interface unit 151 in FIG. 4.

  When switching the model data hierarchy level, the size and number of model data components and the corresponding attribute value conversion should be determined in advance according to the type and hierarchy level of each model data. Is desirable.

  FIG. 9 is a diagram showing an example of conversion of model data, in which the model elements are composed of triangular patch surfaces. (A) is before conversion, and (b) is after conversion. In order to reduce the number of elements by increasing the size of the elements from the state of (a), the vertices constituting the triangular patch surfaces A0, A1, A2,. Then, as shown in (b), elements of new triangular patch surfaces B0 and B1 having the sampled points as vertices are formed.

  When the number of elements and the size of the model data are changed in this way, the attribute value in the new element is calculated according to a predetermined rule from the original element value. For example, the attribute value in the element B0 in FIG. 5B is obtained by calculating the average of the attribute values of the elements A0 to A3 in FIG.

  FIG. 10 is a diagram illustrating another example of conversion of model data, in which the model elements are configured by cubic voxels. Similarly, in this case, the vertices constituting the voxel before conversion (a) are sampled every other point, and a new voxel element having the sampled points as vertices is formed as shown in (b). The attribute value in this new voxel element is obtained by calculating the average of the attribute values for the eight original voxels in (a).

  In the above example, the example in which the size and number of elements are converted by changing the interval between the vertices constituting the elements is shown, but the conversion method is not limited to this, and some rules are defined in advance. Any new element can be obtained uniquely.

  As described above, according to the present embodiment, the model data necessary for video encoding and decoding is converted into the size and number of elements according to the hierarchical level, and the attribute values of individual elements, There is an effect of reducing the processing amount in encoding and decoding.

  The moving image encoding apparatus and decoding apparatus of the present invention are applicable not only to general moving images in which frames are arranged in the time direction but also to three-dimensional images in image diagnostic apparatuses such as X-ray CT and MRI. Is possible. Example 4 relates to such an image diagnostic apparatus. That is, in the image diagnostic apparatus, since the position of the imaging surface (corresponding to the time of each frame in the moving image) is arranged in the three-dimensional space, the frame time information in the general moving image is obtained from each imaging in the image diagnostic apparatus. It can be associated with the spatial position of the surface. Information regarding such association may be added as necessary.

  FIG. 11 is a block diagram showing an embodiment of a medical image diagnostic apparatus to which the present invention is applied. Examples of the medical image diagnostic apparatus include X-ray CT and MRI. Here, X-ray CT will be described as an example.

  The medical image diagnostic apparatus 300 includes an image acquisition unit 320, an imaging protocol storage unit 330, a control unit 340, a user interface unit, in addition to an encoding / decoding processing unit 310 that performs model-based encoding and decoding of a captured image. 350 is comprised. Furthermore, the control unit 340 is connected to an external model data server 370 via the network 360. The image acquisition unit 320 acquires 3D reconstructed image data to be imaged. The shooting protocol storage unit 330 records a shooting protocol including shooting conditions. The control unit 340 controls the entire operation of the medical image diagnostic apparatus 300. The encoding / decoding processing unit 310 performs encoding of the tomographic image generated by the image acquisition unit 320 and decoding of the tomographic image from the already encoded stream. The user interface unit 350 selects the tomographic image generated by the image acquisition unit 320 and the model used for encoding the tomographic image by the encoding / decoding processing unit 310 and is decoded by the encoding / decoding processing unit 310. This is used for displaying tomographic images and inputting imaging protocols.

  The configuration and operation of the image acquisition unit 320 will be described. The image acquisition unit 320 includes a scan control unit 321, an imaging unit 322, an image reconstruction unit 323, and a volume data storage unit 324. The image acquisition unit 320 irradiates the imaging target with an X-ray beam, detects the X-ray beam transmitted through the imaging target, and generates three-dimensional volume data based on the detected X-ray projection data.

  The imaging unit 322 includes a bed on which an imaging target is placed, an X-ray source, an X-ray detector, a data collection unit, and the like. The imaging unit 322 irradiates the imaging target with an X-ray beam and detects X-rays transmitted through the imaging target. . The X-ray detector is composed of X-ray detection elements arranged in an array in two directions orthogonal to each other. The data collection unit includes data collection elements arranged in an array like the X-ray detection elements, and collects data according to a control signal output from the scan control unit 321. This collected data becomes X-ray projection data.

  The imaging unit 322 controls the imaging target by moving the X-ray source around the imaging target while moving the bed on which the imaging target is placed along the body axis of the imaging target under the control of the scan control unit 321. Take a picture with a spiral scan.

  The image reconstruction unit 323 reconstructs three-dimensional image data (volume data) that is three-dimensional distribution data of the X-ray absorption coefficient from the X-ray projection data output from the imaging unit 322. This reconstruction processing is realized by a known method such as a method using Fourier transform or back projection processing. The reconstructed volume data is stored in the volume data storage unit 324.

  The image generation unit 325 uses the volume data stored in the volume data storage unit 324 to generate arbitrary cross-sectional image data. The generated cross-sectional image is displayed on the user interface unit 350.

  The encoding / decoding processing unit 310 includes a moving image encoding unit 311, a moving image decoding unit 312, and an encoding model database 313. The moving image encoding unit 311 is equivalent to the moving image encoding apparatus 100 according to the first embodiment (FIG. 1), and inputs the sequence of tomographic images generated by the image acquisition unit 320, and the image code already described. To generate a compressed encoded stream.

  The moving picture decoding apparatus 312 is equivalent to the moving picture encoding apparatus 200 according to the second embodiment (FIG. 6), and decodes an arbitrary tomographic image included in the compressed encoded stream.

  The encoding model database 313 manages model data that can be used by the moving image encoding unit 311 and model data that can be used by the moving image decoding unit 312. The operation is the same as that of the encoding model databases 153 and 253 of FIGS.

  The user interface unit 350 selects the tomographic image generated by the image acquisition unit 320, the model data used for encoding the tomographic image by the encoding / decoding processing unit 310, and acquires the model from the external model data server 370. Is used to display the tomographic image decoded by the encoding / decoding processing unit 310, input an imaging protocol, and the like. Here, the interface for selecting the model data and determining the model acquisition from the model data server 370 is the same as the example described with reference to FIGS.

  An example of model data used in the medical image diagnostic apparatus of this embodiment will be described. “Model name” is a character string indicating the characteristics of model data. For example, “X-ray CT image of brain (for horizontal section)” “X-ray CT image of brain (for sagittal section)” “abdominal MRI image” (Standard male) "" Abdominal MRI (obese male) "etc. are prepared. In addition, a model to be used for encoding can be selected according to the captured target region, such as “chest X-ray CT image”, “abdominal X-ray CT image”, and “head MRI image”. Thus, it is possible to execute optimal model-based encoding for various imaging targets. In addition, when encoding a captured image of “abdominal X-ray CT”, encoding targets such as “adult male”, “adult female”, “obese male”, and “obese female” are standard abdominal models. By preparing a model that is further subdivided according to the characteristics of the image, a predicted image with higher accuracy can be generated, and further improvement of the compression rate can be expected.

1 is a configuration diagram (Example 1) showing an example of a moving picture encoding apparatus according to the present invention. FIG. The figure which shows an example of the structure of model data. The figure explaining the structure of an encoding stream. The block diagram which shows the other Example of the moving image encoder by this invention. The figure which shows an example of the notification screen of a model list. The block diagram which shows one Example of the moving image decoding apparatus by this invention (Example 2). The block diagram which shows the other Example of the moving image decoding apparatus by this invention. The figure which shows an example of the screen of the outline information of model data. FIG. 10 is a diagram illustrating an example of model data conversion (triangular patch surface) (Example 3); The figure (voxel element) which shows an example of conversion of model data. The block diagram (Example 4) which shows one Example of the medical image diagnostic apparatus to which this invention is applied.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Moving image encoding device 121 ... Model control part 122 ... Model data storage part 123 ... Model matching part 124 ... Two-dimensional image synthetic | combination part 125 ... Model base motion estimation part 150 ... Moving image encoding apparatus 151 ... User interface part 152 ... Coding device control unit 153 ... Coding model database unit 200 ... Video decoding device 221 ... Model control unit 222 ... Model data storage unit 223 ... Model transformation unit 224 ... Two-dimensional image synthesis unit 225 ... Model-based motion compensation unit 250 ... moving picture decoding apparatus 251 ... user interface section 252 ... decoding apparatus control section 253 ... coding model database section.

Claims (13)

In a video encoding device that takes a difference between a prediction image and an input image, orthogonally transforms, quantizes, and variable-length encodes the prediction difference image and outputs an encoded stream.
A model-based prediction system for generating a model-based prediction image as the prediction image is provided,
The model-based prediction system is
A model data storage unit for storing model data of a plurality of models used for model-based prediction;
A model matching unit that transforms the plurality of model data using predetermined parameters and selects an optimal model for the input image; and
A two-dimensional image synthesis unit that synthesizes a two-dimensional model image by projecting the deformed model data on a two-dimensional plane or cutting the two-dimensional plane;
A model-based motion prediction unit that refers to the two-dimensional model image and generates the model-based motion prediction vector of the input image and the model-based prediction image;
A moving picture encoding apparatus comprising: The moving picture encoding apparatus according to claim 1,
The model data includes a model name, element type, element data, attribute data set, and model parameter initial values, and a model operation rule based on the parameters is defined in advance. . The moving picture encoding apparatus according to claim 1 or 2,
The encoded stream to be output includes information indicating the model used for prediction in the model-based prediction image, parameters used for deformation of the model, the model-based motion prediction vector, and information on the prediction difference image. A moving picture coding apparatus storing information related to the two-dimensional plane. The moving picture encoding apparatus according to claim 1 or 2,
A moving picture coding apparatus comprising: a device control unit that presents a model list used for model-based prediction and acquires model data of a necessary model in the list from the outside. The moving picture encoding apparatus according to claim 1 or 2,
A model control unit that converts a hierarchical level of model data used for model-based prediction according to the resolution of the input image;
The moving picture coding apparatus characterized in that the model control unit performs conversion of size and number of constituent elements of model data and conversion of attribute values according to a hierarchical level. In a video decoding device that decodes, dequantizes, and inversely orthogonally transforms a variable length code of an input encoded stream to obtain a prediction difference image, adds the prediction image, and outputs an image.
A model-based prediction system for generating a model-based prediction image as the prediction image is provided,
The model-based prediction system is
A model data storage unit for storing model data of a plurality of models used for model-based prediction;
Based on information indicating the model used for prediction obtained from the encoded stream and parameters used for deformation of the model, a model deformation unit that deforms corresponding model data;
A two-dimensional image synthesis unit that synthesizes a two-dimensional model image by projecting on the two-dimensional plane or cutting on the two-dimensional plane based on information about the two-dimensional plane acquired from the encoded stream;
A model-based motion compensation unit that generates the model-based prediction image using the two-dimensional model image and a model-based motion prediction vector acquired from the encoded stream;
A moving picture decoding apparatus comprising: The moving picture decoding apparatus according to claim 6, wherein
A moving picture decoding apparatus comprising: an apparatus control unit that presents information indicating a model used for prediction acquired from the encoded stream, and acquires model data necessary for decoding from the outside. The moving picture decoding apparatus according to claim 6, wherein
A model control unit that converts a hierarchical level of model data used for the model-based prediction,
The moving picture decoding apparatus characterized in that the model control unit performs conversion of the size and number of constituent elements of model data and conversion of attribute values according to hierarchical level information acquired from the encoded stream. In a video encoding method that takes a difference between a prediction image and an input image, orthogonally transforms, quantizes, and variable-length-encodes the prediction difference image and outputs an encoded stream.
In order to generate a model-based prediction image as the prediction image,
Store model data of multiple models used for model-based prediction,
Deform the plurality of model data using predetermined parameters to select an optimal model for the input image,
Projecting the deformed model data on the 2D plane or cutting it on the 2D plane to synthesize a 2D model image,
A moving picture coding method comprising generating a model-based motion prediction vector of the input image and the model-based prediction image with reference to the two-dimensional model image. The moving image encoding method according to claim 9, wherein
A moving picture coding method characterized by presenting a model list used for model-based coding at the time of coding, and acquiring model data from the outside when a necessary model is not stored in the list. The moving image encoding method according to claim 9, wherein
Moving picture coding characterized in that the hierarchical level of model data used for model-based coding is converted in accordance with the resolution of the input image, and the size and number of model data components and attribute values are converted. Method. In a video decoding method for decoding, dequantizing, and inverse orthogonal transforming a variable length code of an input encoded stream to obtain a prediction difference image, and adding the prediction image to output an image,
In order to generate a model-based prediction image as the prediction image,
Store model data of multiple models used for model-based prediction,
Based on the information indicating the model used for prediction obtained from the encoded stream and the parameters used for deformation of the model, the corresponding model data is modified,
Based on the information about the two-dimensional plane obtained from the encoded stream, a two-dimensional model image is synthesized by projecting on the two-dimensional plane or cutting on the two-dimensional plane,
A moving picture decoding method, wherein the model base predicted image is generated using the two-dimensional model image and a model base motion prediction vector acquired from the encoded stream. The moving picture decoding method according to claim 12,
Presenting information indicating the model used for prediction obtained from the encoded stream when decoding,
A moving picture decoding method, wherein model data necessary for decoding is not stored when the model data is not stored.

Images