JP5524594B2 - Image decoding apparatus and image decoding method - Google Patents

Description

  The present invention relates to an image decoding apparatus and an image decoding method, and in particular, a first encoded signal obtained by encoding a first image signal of a first viewpoint and a second encoded image of a second image signal of a second viewpoint. The present invention relates to an image decoding apparatus for decoding an encoded signal.

  In recent years, as a high-definition image system, there is an increasing expectation for a stereoscopic image system centering on digital broadcasting and package media. In general, it is known that a human can perceive depth by binocular parallax. In the stereoscopic image system, a stereoscopic effect is generated by using this and projecting two images having an image shift corresponding to the parallax to the left eye and the right eye independently.

  Various methods have been proposed so far for methods for giving images with parallax to the left eye and the right eye independently. For example, a two-dimensional image for the left eye and a two-dimensional image for the right eye are alternately displayed on the same screen. Further, in synchronization with the display timing, the left-eye shutter and the right-eye shutter are switched using glasses with a built-in liquid crystal shutter or the like. Thereby, there is a method of giving a viewer a parallax image. In addition, in order to give a more realistic feeling, it is necessary to give the same number of display frames as the two-dimensional image system to each of the left eye and the right eye.

  That is, in such a stereoscopic image system, it is necessary to simultaneously decode a plurality of bit streams for left eye and right eye. That is, the stereoscopic image system requires a larger image decoding performance than the two-dimensional image system, which increases the cost.

  As a method of simultaneously decoding a plurality of bitstreams while reducing such an increase in cost, Patent Document 1 discloses a technique for decoding a plurality of bitstreams in parallel using a plurality of image decoding units. Yes.

  FIG. 30 is a block diagram illustrating a configuration of an image decoding device 500 described in Patent Document 1.

  An image decoding apparatus 500 illustrated in FIG. 30 includes an input unit 502, a header analysis unit 503, and a decoding control unit 504. Furthermore, the image decoding apparatus 500 includes a plurality of arithmetic decoding units 505, 506, and 507, a plurality of storage units 508, 509, and 510, and a plurality of image decoding units 511, 512, and 513, following the decoding control unit 504. Are provided in parallel. The image decoding apparatus 500 receives a plurality of bit streams A, B, and C.

  Upon receiving the analysis result from the header analysis unit 503, the decoding control unit 504 calculates the arithmetic decoding processing start timing in the arithmetic decoding units 505, 506, and 507, the start position of the arithmetic decoding processing in the bitstream, and the like. Notification is made to the decoding units 505, 506, and 507.

  Arithmetic decoding units 505, 506, and 507 are individually provided corresponding to the input bit streams A, B, and C, respectively. The arithmetic decoding units 505, 506, and 507 perform arithmetic decoding processes of the bit streams A, B, and C in parallel, and output the generated arithmetic decoding data to the storage units 508, 509, and 510, respectively.

  The image decoding units 511, 512, and 513 simultaneously reproduce a plurality of data streams by decoding the arithmetic decoding data stored in the storage units 508, 509, and 510 into actual image data.

  Patent Document 2 discloses a method of simultaneously decoding a plurality of bit streams by time division processing using a single image decoding unit.

  FIG. 31 is a block diagram illustrating a configuration of an image decoding device 600 described in Patent Document 2.

  31 includes a plurality of VBV buffers 601, a variable length decoder 602, an inverse quantizer 603, an IDCT device 604, an adder 605, an image memory 606, and a motion compensation predictor. 607, a demultiplexer 611, signal switchers 612, 613, and 614, a decoding controller 621, a display controller 622, and a switch controller 623.

  In the image decoding device 600, a variable length decoder 602, an inverse quantizer 603, an IDCT device 604, an adder 605, and a motion compensation predictor 607 are each configured by a single resource. Also, the image decoding apparatus 600 inputs a plurality of bit streams to these circuits in a time division manner and performs a decoding process.

  Specifically, a transport stream in which image bit streams of a plurality of channels are multiplexed is input to the preceding demultiplexer 611. The demultiplexer 611 separates the transport stream into bit streams for each channel. The separated bit stream of each channel is accumulated in the corresponding VBV buffer 601 (VBV buffer 1 to VBV buffer N).

  After the demultiplexing process, it is necessary to simultaneously decode N bit streams stored in the VBV buffer 1 to the VBV buffer N. On the other hand, the bit stream in the VBV buffer 601 corresponding to a specific channel is selected by the signal switcher 612 based on the switch switch signal from the switch controller 623. This selected bit stream is input to the variable length decoder 602. Thereafter, this bit stream is decoded by a variable length decoder 602, an inverse quantizer 603, an IDCT unit 604, and a motion compensation predictor 607. The decoded image data is input from the adder 605 to the signal switch 613. The signal switch 613 stores this image data in the memory of the corresponding channel among the plurality of image memories 606 (image memory 1 to image memory N) based on the switch switching signal output from the switch controller 623. To do.

  The display controller 622 performs control so that the image data of each channel in the image memory N is simultaneously read from the image memory 1 and displayed at a predetermined display timing. As a result, the display controller 622 synchronizes the channels and simultaneously reproduces a plurality of bit streams.

JP 2007-150569 A JP 2000-165858 A

  As described above, in the stereoscopic image system, the left-eye image and the right-eye image need to be individually transmitted or recorded as independent images in order to give the viewer a sense of reality. Therefore, the stereoscopic image system requires about twice the amount of information as compared to a single two-dimensional image system. This is because a transmission system requires an enormous transfer bandwidth, and a storage system requires an enormous storage capacity in addition to this. As a result, the cost of the system increases.

  In view of this, in order to reduce the amount of data for realizing a stereoscopic image system, an image encoding method is also devised. H. The H.264-MVC (Multi View Coding) standard takes advantage of the fact that there is often a correlation between videos of each viewpoint. Based on the H.264 standard, a prediction structure between viewpoints as shown in FIG. 32 is provided. Thereby, compression efficiency can be improved by removing redundancy.

  Each viewpoint is given a viewpoint ID of view_id, is managed in the form of a view, and is distinguished. This view includes a base view to be referenced from other views and a non-base view that refers to the base view. That is, at the time of inter-viewpoint prediction, encoding efficiency is improved by encoding a non-base view using a base view image as a predicted image.

  However, in a system in which code data encoded using such inter-view prediction is provided as an individual bit stream for each view, in the prior art, when decoding these multiple bit streams, There is a problem related to specifying an image to be referred to by the base view.

  Specifically, H.C. In the 264-MVC standard, only non-base view images that are decoded at the same time (same access unit) are allowed to be referenced. Therefore, in the syntax information in the bitstream, only the view_id of the view (base view) to be referred to is specified for each non-base view.

  That is, when decoding a non-base view, the view (base view) to be referred to is known by view_id, and which is the reference target image from among the frame memories in which the decoded images of the base view are stored. It is necessary to know (which is the image of the same access unit).

  However, if the configuration is such that the decoding processing of each channel image is performed independently without considering the decoding information of other channels as in the prior art, the image to be referred to by each channel can be specified. Therefore, there arises a problem that the decoding process cannot be performed correctly.

  The present invention has been made to solve this problem, and an object of the present invention is to decode a plurality of bit streams encoded using inter-view prediction and to specify an image to be referred to by each channel. An image decoding apparatus is provided.

  In order to achieve the above object, an image decoding apparatus according to an aspect of the present invention includes a first encoded signal obtained by encoding a first image signal of a first viewpoint, and a second viewpoint different from the first viewpoint. An image decoding apparatus for decoding a second encoded signal obtained by encoding a second image signal, wherein the first encoded signal and the second encoded signal are decoded with reference to a reference image An image decoding unit configured to generate a first decoded image signal and a second decoded image signal; and a frame memory for storing the first decoded image signal and the second decoded image signal as the reference image. The unit refers to the first decoded image signal as the reference image, decodes the first encoded signal, and refers to the first decoded image signal and the second decoded image signal as the reference image. 2 decoding the encoded signal, and The apparatus further specifies first management information for specifying an area of the frame memory in which the first decoded image signal is stored and an area of the frame memory in which the second decoded image signal is stored. A management information storage unit that stores the second management information; and the reference image that is referred to when the first encoded signal is decoded is stored in the image decoding unit with reference to the first management information. The reference image to be referred to when the second encoded signal is decoded is stored in the image decoding unit with reference to the first management information and the second management information. And a controller for notifying the area of the frame memory.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention can specify the first decoded image signal referred to when decoding the second encoded signal, using the first management information. As described above, the image decoding apparatus according to an aspect of the present invention can decode a plurality of bit streams encoded using inter-view prediction, and can specify an image to be referred to by each channel.

  The first management information is included in the first encoded signal for each of the first decoded image signals, and includes first syntax information for specifying the first decoded image signal, and the first syntax information. A first syntax information list including a first identification number associated with the tax information, and a first relationship indicating a correspondence relationship between the first identification number and the area of the frame memory for each first decoded image signal. The control unit obtains the first identification number corresponding to the first decoded image signal by referring to the first syntax information list, and refers to the first mapping list. Thus, the area of the frame memory corresponding to the first identification number is acquired, and the acquired area of the frame memory is defined as the area of the frame memory in which the reference image is stored. And the second management information is included in the second encoded signal for each second decoded image signal, and the second thin information for specifying the second decoded image signal is sent to the image decoding unit. A second syntax information list including tax information and a second identification number associated with the second syntax information, and the second identification number and the area of the frame memory for each second decoded image signal. A second mapping list indicating a correspondence relationship with the second mapping list, the control unit obtains the second identification number corresponding to the second decoded image signal by referring to the second syntax information list, By referring to the second mapping list, an area of the frame memory corresponding to the second identification number is acquired, and the acquired area of the frame memory is stored before the reference image is stored. As an area of the frame memory, it may notify the image decoding units.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention uses the first syntax information list and the first mapping list to refer to the first decoded image that is referred to when the second encoded signal is decoded. The signal can be specified.

  Further, the first syntax information and the second syntax information may be H.264. It may be syntax information necessary when updating a decoded picture buffer (DPB) defined in the H.264 standard.

  In addition, the first syntax information list includes the first syntax information and the first identification number for each first decoded image signal referred to when the image decoding unit decodes the first encoded signal. And the first syntax information and the first information for each first decoded image signal referred to when the image decoding unit decodes the second encoded signal. An inter-view reference syntax information list including an identification number, and when the image decoding unit decodes the first encoded signal, the control unit displays the intra-view reference syntax information list. By referencing, the first identification number corresponding to the first decoded image signal is obtained, and by referring to the first mapping list, the area of the frame memory corresponding to the first identification number is acquired. When the image decoding unit decodes the second encoded signal, the image decoding unit notifies the image decoding unit of the acquired frame memory region as the frame memory region in which the reference image is stored. The first identification number corresponding to the first decoded image signal is obtained by referring to the inter-view reference syntax information list, and the first mapping list is referred to to obtain the first identification number. The frame memory area corresponding to the identification number may be acquired, and the acquired frame memory area may be notified to the image decoding unit as the frame memory area in which the reference image is stored.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention includes an intra-view reference syntax information list used for reference within a viewpoint and an inter-view reference syntax information list used for reference between viewpoints. Can be individually managed and controlled for references within viewpoints and references between viewpoints. Thereby, the image decoding apparatus according to an aspect of the present invention can suitably manage and control reference images between viewpoints.

  In addition, the first mapping list includes a correspondence between the first identification number and the area of the frame memory for each first decoded image signal referred to when the image decoding unit decodes the first encoded signal. A mapping list for in-view reference indicating a relationship, and for each first decoded image signal referred to when decoding the second encoded signal by the image decoding unit, the first identification number and the area of the frame memory And the control section refers to the intra-view reference syntax information list when the image decoding section decodes the first encoded signal. By acquiring the first identification number corresponding to the first decoded image signal and referring to the in-view reference mapping list, the frame corresponding to the first identification number is obtained. The memory area is acquired, the acquired area of the frame memory is notified to the image decoding section as the area of the frame memory in which the reference image is stored, and the image decoding section transmits the second encoded signal. When decoding, the first identification number corresponding to the first decoded image signal is obtained by referring to the inter-view reference syntax information list, and the inter-view reference mapping list is referred to. To acquire the area of the frame memory corresponding to the first identification number, and notify the image decoding unit of the acquired area of the frame memory as the area of the frame memory in which the reference image is stored. Also good.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention separately performs the intra-view reference mapping list used for intra-view reference and the inter-view reference mapping list used for inter-view reference. By providing, the reference within a viewpoint and the reference between viewpoints can be managed and controlled separately. Thereby, the image decoding apparatus according to an aspect of the present invention can suitably manage and control reference images between viewpoints.

  The image decoding apparatus further includes an image display unit that outputs the first decoded image signal and the second decoded image signal stored in the frame memory to the outside, and the first mapping list further includes: For each first decoded image signal, first image display information indicating whether or not the first decoded image signal is output to the outside by the image display unit, and the first decoded image signal is used as the reference image. The first reference information indicating whether or not the first decoded image signal is output to the outside by the first image display information, and the first reference information Indicates that the first decoded image signal is not used as the reference image, the image decoding unit newly adds the first decoded image signal to the area of the frame memory in which the first decoded image signal is stored. The issue is first decoded image signal may be stored.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention can suitably manage the free area of the frame memory using the first mapping list.

  The image decoding unit includes: a first decoding unit that generates a first decoded image signal by decoding the first encoded signal; and a second decoded image signal that is generated by decoding the second encoded signal. A second decoding unit that generates the control unit, wherein the control unit uses the first decoding unit and the second decoding unit to determine predetermined decoding processing units of the first encoded signal and the second encoded signal. After both of the decoding processes are completed, the first decoding unit and the second decoding unit may start the decoding process for the next decoding process unit.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention can synchronize the decoding process of the first encoded signal and the decoding process of the second encoded signal.

  The image decoding apparatus further includes a flag storage unit for storing a first completion flag and a second completion flag, and the control unit controls the first decoding unit and the first decoding unit. A first control unit that stores the first completion flag in the flag storage unit when the decoding process of the first encoded signal unit of the first encoded signal is completed, and the second decoding unit, A second control unit that stores the second completion flag in the flag storage unit when the decoding processing unit of the second encoded signal by the second decoding unit is completed; When the decoding processing unit of the first encoded signal by the first decoding unit is completed by the first decoding unit and the second completion flag is stored in the flag storage unit, the control unit performs the next decoding The first code of the processing unit The first decoding unit starts signal decoding processing, and the second control unit completes the decoding processing unit decoding processing of the second encoded signal by the second decoding unit, and stores the flag When the first completion flag is stored in the unit, the second decoding unit may start decoding processing of the second encoded signal in the next decoding processing unit.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention controls the decoding process of the first encoded signal and the decoding process of the second encoded signal separately, thereby controlling the complexity of the decoding process. Can be suppressed.

  The image decoding unit includes a selection unit that selects one of the first encoded signal and the second encoded signal, and the first encoded signal or the second encoded signal selected by the selection unit. The control unit causes the selection unit to select one of the first encoded signal and the second encoded signal, and the one of the signals is predetermined. After the decoding process of the decoding process unit is completed, the selection unit may cause the other signal of the first encoded signal and the second encoded signal to be selected.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention can synchronize the decoding process of the first encoded signal and the decoding process of the second encoded signal.

  The image decoding apparatus further includes a flag storage unit for storing a first completion flag and a second completion flag, and the control unit controls the first decoding unit and the first decoding unit. A first control unit that stores the first completion flag in the flag storage unit when the decoding process of the first encoded signal unit of the first encoded signal is completed, and the second decoding unit, A second control unit that stores the second completion flag in the flag storage unit when the decoding processing unit of the second encoded signal by the second decoding unit is completed; When the second completion flag is stored in the flag storage unit, the control unit causes the decoding unit to start the decoding process of the first encoded signal of the next decoding processing unit, and the second control unit , The flag storage unit If the completion flag is stored, may initiate decoding of the second coded signal of the next decoding processing units to the decoder.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention controls the decoding process of the first encoded signal and the decoding process of the second encoded signal separately, thereby controlling the complexity of the decoding process. Can be suppressed.

  The image decoding apparatus further generates one third encoded signal by alternately arranging the first encoded signal and the second encoded signal for each predetermined encoding unit. The image decoding unit may include a combining unit, and the image decoding unit may alternately decode the first encoded signal and the second encoded signal included in the third encoded signal for each encoding unit.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention can synchronize the decoding process of the first encoded signal and the decoding process of the second encoded signal.

  The encoding unit may be any one of a frame unit, a field unit, and a slice unit.

  In addition, the combining unit may arrange the first encoded signal of the encoding unit among the first encoded signal and the second encoded signal corresponding to the encoding unit, and then the first encoded signal. The third encoded signal may be generated by arranging the second encoded signal of the encoding unit corresponding to the encoded signal.

  According to this configuration, the image decoding apparatus according to an aspect of the present invention can always decode the first encoded signal referred to in the decoding process of the second encoded signal first.

  The decoding processing unit may be any one of a frame unit, a field unit, a header decoding processing unit, and a slice unit.

  Note that the present invention can be realized not only as such an image decoding apparatus, but also as an image decoding method that uses characteristic means included in the image decoding apparatus as a step, or to perform such characteristic steps in a computer. It can also be realized as a program to be executed. Needless to say, such a program can be distributed via a recording medium such as a CD-ROM and a transmission medium such as the Internet.

  Furthermore, the present invention can be realized as a semiconductor integrated circuit (LSI) that realizes part or all of the functions of such an image decoding device, or can be realized as an image reproduction device including such an image decoding device, It can be realized as an image display system including such an image decoding device.

  As described above, the present invention can provide an image decoding apparatus capable of decoding a plurality of bit streams encoded using inter-view prediction and identifying an image to be referred to by each channel.

It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram of the image decoding part which concerns on Embodiment 1 of this invention. It is a figure which shows the reference picture number which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the syntax information which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the DPB list which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the frame memory mapping list | wrist which concerns on Embodiment 1 of this invention. 6 is a diagram illustrating a correspondence relationship between a reference picture number and a base address according to Embodiment 1. FIG. It is a figure which shows the process of the control part which concerns on Embodiment 1 of this invention. It is a figure which shows the image display timing which concerns on Embodiment 1 of this invention. It is a figure which shows the stream structure which concerns on Embodiment 1 of this invention. It is a flowchart of the decoding process by the image decoding apparatus which concerns on Embodiment 1 of this invention. It is a flowchart of the decoding process by the image decoding apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the synchronous relationship of the L side decoding process which concerns on Embodiment 1 of this invention, and an R side decoding process. It is a figure which shows the operation example of the image decoding apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows the image display timing which concerns on Embodiment 2 of this invention. It is a figure which shows the synchronous relationship of the L side decoding process and R side decoding process in Embodiment 2 of this invention. It is a figure which shows the operation example of the image decoding apparatus which concerns on Embodiment 2 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 3 of this invention. It is a figure which shows the structure of the frame memory mapping list | wrist which concerns on Embodiment 3 of this invention. It is a figure which shows the operation example of the image decoding apparatus which concerns on Embodiment 3 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 4 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 5 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 6 of this invention. It is a figure which shows the process of the control part which concerns on Embodiment 6 of this invention. It is a flowchart of the decoding process by the image decoding apparatus which concerns on Embodiment 6 of this invention. It is a flowchart of the decoding process by the image decoding apparatus which concerns on Embodiment 6 of this invention. It is a block diagram which shows the structure of the image decoding apparatus which concerns on Embodiment 7 of this invention. It is a block diagram which shows the structure of the image decoding apparatus of a prior art. It is a block diagram which shows the structure of the image decoding apparatus of a prior art. H. It is a figure which shows the reference relationship in the 264-MVC standard.

  Embodiments of an image decoding apparatus according to the present invention will be described below in detail with reference to the drawings.

(Embodiment 1)
The image decoding apparatus according to Embodiment 1 of the present invention uses the first management information for specifying the area in which the first decoded image signal obtained by decoding the first encoded bit stream is stored, and uses the second bit stream. An area in which a reference image to be referred to when decoding is stored is specified. Thereby, the image decoding apparatus according to Embodiment 1 of the present invention can decode a plurality of bit streams encoded using inter-view prediction, and can specify an image to be referred to by each channel.

  First, the configuration of the image decoding apparatus 100 according to the embodiment of the present invention will be described.

  FIG. 1 is a block diagram showing a configuration of image decoding apparatus 100 according to Embodiment 1 of the present invention.

  The image decoding apparatus 100 illustrated in FIG. 1 generates an output image signal 112 by decoding the encoded bit stream 110 and the encoded bit stream 111.

  The encoded bit streams 110 and 111 are stream data including data obtained by encoding images having parallax with each other. In other words, the encoded bit stream 110 is a first encoded signal obtained by encoding the first image signal captured from the first viewpoint. The encoded bit stream 111 is a second encoded signal obtained by encoding a second image signal shot at a second viewpoint different from the first viewpoint.

  In the present embodiment, H.264 is used. A decoding process of a bitstream encoded according to the H.264 standard will be described as an example. For example, the encoded bitstreams 110 and 111 are H.264. It is encoded according to the 264-MVC standard. The encoded bit stream 110 is a left-eye bit stream (L side). The encoded bit stream 111 is a right-eye bit stream (R side). The encoded bit streams 110 (L side) and 111 (R side) are input to the image decoding apparatus 100 as two independent encoded bit streams.

  Further, it is assumed that the L-side encoded bit stream 110 is a base view and the R-side encoded bit stream is a non-base view.

  The image decoding apparatus 100 includes an image decoding unit 120, a control unit 130, a management information storage unit 140, a frame memory management unit 150, and an image display unit 160, and a frame memory 170.

  The image decoding unit 120 decodes the encoded bit stream 110 and the encoded bit stream 111 with reference to a reference image, thereby decoding a first decoded image signal (hereinafter referred to as an L-side decoded picture) and a second decoded image signal (hereinafter referred to as a decoded image signal). , R side decoded picture). Note that the L-side decoded picture and the R-side decoded picture are also simply referred to as “decoded pictures” when they are not particularly distinguished.

  The frame memory 170 stores the decoded image data (L-side decoded picture and R-side decoded picture) generated by the image decoding unit 120.

  Also, since the encoded bitstream 110 is a base view, the image decoding unit 120 decodes the encoded bitstream 110 with reference to only the L-side decoded picture as a reference image. Since the encoded bitstream 111 is a non-base view, the image decoding unit 120 decodes the encoded bitstream 111 with reference to the L-side decoded picture and the R-side decoded picture as reference images.

  In the following description, a non-base view picture refers to a base view picture as an inter-view reference. A reference to a picture in the same view (base view or non-base view) is referred to as an in-view reference.

  The control unit 130 controls the decoding processing of the encoded bit streams 110 and 111 by the image decoding unit 120.

  The management information storage unit 140 stores picture management information used for decoding processing by the image decoding unit 120. This picture management information is information for specifying an area (base address) of the frame memory 170 in which the L-side decoded picture and the R-side decoded picture are stored. The picture management information includes L-side management information for specifying the base address where the L-side decoded picture is stored, and R-side management information for specifying the base address where the R-side decoded picture is stored. Including.

  In addition, the control unit 130 refers to the L-side management information and notifies the image decoding unit 120 of a base address in which a reference image to be referred to when decoding the encoded bitstream 110 is stored. In addition, the control unit 130 refers to the L-side management information and the R-side management information, and notifies the image decoding unit 120 of a base address where a reference image to be referred to when decoding the encoded bitstream 111 is stored. .

  The frame memory management unit 150 manages the frame memory 170.

  The image display unit 160 generates an output image signal 112 including the L-side decoded picture and the R-side decoded picture stored in the frame memory 170, and outputs the generated output image signal 112 to the outside.

  FIG. 2 is a block diagram illustrating a detailed configuration of the image decoding apparatus 100.

  As shown in FIG. 2, the image decoding unit 120 includes a first decoding unit 121 and a second decoding unit 122.

  The first decoding unit 121 generates an L-side decoded picture by decoding the encoded bitstream 110 in real time.

  The second decoding unit 122 generates an R-side decoded picture by decoding the encoded bitstream 111 in real time.

  The management information storage unit 140 includes DPB (decoded picture buffer) lists 141a and 141b, frame memory mapping lists 142a and 142b, an inter-view reference syntax information list 143, and an inter-view reference frame memory as picture management information. The mapping list 144 is stored. Here, the DPB list 141a, the frame memory mapping list 142a, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 include an L for specifying a base address in which the L-side decoded picture is stored. Side management information. The DPB list 141b and the frame memory mapping list 142b are R-side management information for specifying a base address in which the R-side decoded picture is stored.

  The DPB list 141a includes syntax information for in-view reference of the L-side decoded picture generated from the encoded bitstream 110. Further, the DPB list 141 b includes syntax information for in-view reference of the L-side decoded picture generated from the encoded bitstream 111. Here, the syntax information includes parameters (variables) defined by the standard.

  The inter-view reference syntax information list 143 includes inter-view reference syntax information.

  The frame memory mapping lists 142a and 142b are used for managing the base address of the frame memory 170.

  The inter-view reference frame memory mapping list 144 is used to manage the base address of the inter-view reference picture.

  The frame memory management unit 150 manages the base address of the frame memory 170 by updating the frame memory mapping lists 142a and 142b based on information obtained from the control unit 130 and the image display unit 160.

  The control unit 130 includes a first control unit 131 and a second control unit.

  The first control unit 131 is in charge of the decoding process for the L-side encoded bitstream 110. In addition, the second control unit 132 is responsible for the decoding process of the R-side encoded bitstream 111.

  The first control unit 131 stores the DPB list 141a, the inter-view reference syntax information list 143, the frame memory mapping list 142a, and the inter-view reference frame memory mapping list 144 stored in the management information storage unit 140. Are used to control the decoding process of the first decoding unit 121.

  The second controller 132 stores the DPB list 141b, the inter-view reference syntax information list 143, the frame memory mapping list 142b, and the inter-view reference frame memory mapping list 144 stored in the management information storage 140. Are used to control the decoding process of the second decoding unit 122.

  That is, the inter-view reference syntax information list 143 and the inter-view reference frame memory mapping list 144 are accessible from both the first control unit 131 and the second control unit 132.

  The image decoding apparatus 100 further includes a flag storage unit 180.

  The flag storage unit 180 stores a first completion flag and a second completion flag used for the first control unit 131 and the second control unit 132.

  Also, the first control unit 131 stores the first completion flag in the flag storage unit 180 when the decoding process for one picture is completed. The second control unit 132 stores the second completion flag in the flag storage unit 180 when the decoding process for one picture is completed. The flag storage unit 180 is accessible from both the first control unit 131 and the second control unit 132. Therefore, the first control unit 131 and the second control unit 132 can confirm that the mutual decoding process has been completed via the flag storage unit 180.

  The image display unit 160 acquires, from the frame memory management unit 150, the base address where the display target picture is stored among the decoded pictures stored in the frame memory 170. The image display unit 160 reads the display target picture stored at the acquired base address from the frame memory 170, and displays and outputs the read display target picture. Further, the image display unit 160 notifies the frame memory management unit 150 that the display is completed.

  Hereinafter, configurations of the first decoding unit 121 and the second decoding unit 122 will be described.

  FIG. 3 is a block diagram showing a configuration of the first decoding unit 121. The configuration of the second decoding unit 122 is the same as that of the second decoding unit 122.

  As shown in FIG. 3, the first decoding unit 121 includes a header analysis unit 123, an entropy decoding unit 124, an inverse quantization unit 125, an inverse orthogonal transform unit 126, an intra prediction unit 127, and a motion compensation unit 128. And a deblocking filter processing unit 129.

  The header analysis unit 123 performs header analysis such as sequence layer / picture layer / slice layer. In addition, the header analysis unit 123 acquires the syntax information 113 of the decoding target image included in the encoded bitstream 110.

  Further, the first decoding unit 121 has a decoding function below the macroblock layer. Specifically, the entropy decoding unit 124, the inverse quantization unit 125, the inverse orthogonal transform unit 126, the intra prediction unit 127, the motion compensation unit 128, and the deblocking filter processing unit 129 are used. A series of decoding processing conforming to the H.264 standard is performed.

  The motion compensation unit 128 performs reference by reading a previously decoded image from the frame memory 170, and stores the decoded image in the frame memory 170.

  Further, when reading from and writing to the frame memory 170, the first control unit 131 acquires the base address from the frame memory management unit 150 and sets it in the first decoding unit 121. Further, the second control unit 132 acquires the base address from the frame memory management unit 150 and sets it in the second decoding unit 122.

  In order to facilitate the description of the specific operation of the image decoding apparatus 100 configured as described above, the following H.264 will be described. The correspondence relationship between the terms defined in the H.264 standard and the components of this embodiment will be described.

  First, H. A relationship between a decoded picture buffer (DPB) defined in the H.264 standard and DPB lists 141a and 141b which are components of the present embodiment will be described.

  H. In the H.264 standard, a plurality of decoded pictures can be used as reference pictures. H. In the H.264 standard, a motion compensation process is performed by selecting a different picture from a plurality of reference pictures for each motion compensation block. In order to designate this reference picture, an identification number assigned to each decoded picture is referred to as a reference picture number (ref_idx). H. In the H.264 standard, a reference picture number (ref_idx) is assigned to each motion compensation block at the time of encoding, and a reference picture is specified using the reference picture number (ref_idx) at the time of decoding.

  Here, when the number of reference pictures is increased, the coding efficiency is increased, while many decoded pictures need to be secured in the memory. In addition to the plurality of reference pictures, a non-reference picture that is a decoded picture that is not referred to at the time of motion compensation needs to be secured in the memory until the display time. As a result, a very large memory capacity is required. Also, the management of these pictures is complicated.

  Therefore, H.H. In the H.264 standard, a memory called a decoded picture buffer (DPB) is conceptually prepared, and a decoded picture is stored in the DPB. In addition, pictures are managed in a unified manner by a method such as sliding window or MMCO (Memory Management Control Operation). As a result, H.C. In the H.264 standard, the memory can be used efficiently.

  In addition, the sliding window or MMCO performs processing using the syntax information 113 of the picture.

  FIG. 4 is a diagram illustrating a reference relationship between pictures.

  As described above, the reference picture number (ref_idx) is assigned to designate the reference picture. However, as shown in FIG. 4, the allocation of the reference picture corresponding to the reference picture number is changed for each decoding target picture. Can do. A list called a reference picture list (RPL) is used for associating a reference picture reserved in the DPB with a reference picture number.

  This RPL is created based on the syntax information 113 for each picture.

  In the present embodiment, such an H.264. In order to support the flexible reference structure of the H.264 standard, the DPB lists 141a and 141b are stored in the management information storage unit 140 as described above, and the information of the DPB lists 141a and 141b is used. H.264 specifies and manages a picture to be stored in a decoded picture buffer (DPB) defined by the H.264 standard.

  FIG. 5 is a diagram illustrating a configuration of the syntax information 113 of each picture. This syntax information 113 is used to specify a decoded picture used as a reference picture.

As shown in FIG. 5, for example, the syntax information 113 includes a short term / long term, nal_ref_idc, a field_pic_flag, and in
ter_view_flag, frame_num, and view_id are included. Note that the syntax information 113 may include only part of the information, or may include information other than these.

  The short trem / long term indicates whether the picture is a short term reference picture or a long term reference picture. The short term reference picture is a picture that can be managed by the sliding window, and the long term reference picture is a picture that exists at a position separated in time that cannot be managed by the sliding window.

  nal_ref_idc indicates whether or not the picture is referenced in the view.

  field_pic_flag indicates whether the picture method is a field (interlace) or a frame (progressive).

  inter_view_flag indicates whether or not the picture is referenced between views.

  frame_num indicates the number of the picture.

  view_id is an identifier of a view including the picture. For example, “0” is assigned to a picture included in the base view, and “1, 2,...” Is assigned to a picture included in the non-base view.

  FIG. 6 is a diagram showing the configuration of the DPB list 141a (141b).

  The DPB lists 141a and 141b include decoded picture syntax information 113 used as reference images for intra-view references.

  As shown in FIG. 6, each decoded picture includes syntax information 202 and an index 201 that is an identification number associated with the syntax information 202. This syntax information 202 corresponds to the syntax information 113 shown in FIG.

  The number of syntax information 202 included in the DPB lists 141a and 141b is determined by the number of reference pictures that can be referred to in each view by each picture.

  The DPB lists 141a and 141b include not only the decoding process of the macroblock layer but also the H.264 format. Of the decoded picture buffer (DPB) defined in the H.264 standard, and It is also used when creating a reference picture list (RPL) defined in the H.264 standard.

  Moreover, the inter-view reference syntax information list 143 includes decoded picture syntax information 113 used as a reference image for inter-view reference. For example, the structure of the inter-view reference syntax information list 143 is the same as that shown in FIG.

  Further, the number of syntax information 202 included in the inter-view reference syntax information list 143 is determined by the number of reference pictures that can be referred to by each picture.

  Next, H.I. The relationship between the decoded picture buffer (DPB) defined in the H.264 standard and the frame memory mapping lists 142a and 142b that are components of the present embodiment will be described.

  H. In the H.264 standard, a picture to be referenced is always stored in a decoded picture buffer (DPB). The contents of the decoded picture buffer are updated every time the decoding process for one picture is completed.

  The procedure for updating the decoded picture buffer (DPB) is also described in H.264. The H.264 standard defines the following procedure.

(Step 1) The picture is decoded.
(Step 2) The decoded picture buffer (DPB) is updated according to a predetermined rule (Sliding Window or MMCO) to determine a release target picture.
(Step 3) Instead of the release target picture in (Step 2), the picture decoded in (Step 1) is stored in the decoded picture buffer (DPB).

  As described above, according to the standard, a picture that has been decoded in (Step 1) is temporarily secured in a temporary buffer until a release target picture is determined in (Step 2). In addition, the picture for which decoding has been completed is actually stored in the decoded picture buffer (DPB) in (Step 3). Because of this procedure, an extra temporary buffer that can store the picture being decoded must be secured separately from the memory capacity corresponding to the decoded picture buffer (DPB).

  In this embodiment, the frame memory 170 is also used as this temporary buffer. That is, the picture to be decoded (picture being decoded) is stored in the frame memory 170.

  As described above, the frame memory 170 has H.264. The picture defined in the H.264 standard (the picture in the DPB) and the picture depending on the implementation form (the picture being decoded) are mixed. In addition, the frame memory area in which the picture that has been displayed is stored is released for storing the next picture. Therefore, these need to be managed together.

  In the present embodiment, the frame memory mapping lists 142a and 142b are stored in the management information storage unit 140 as described above in order to flexibly cope with such management. Further, for each frame memory area in the frame memory 170, state management is performed such as whether or not it is designated as a reference picture in the DPB lists 141a and 141b and whether or not it is designated as an image display target. Thus, the frame memory area to be released for storing the next picture is specified.

  FIG. 7 is a diagram showing the configuration of the frame memory mapping list 142a. For example, the frame memory mapping list 142a has a table format as shown in FIG. The configuration of the frame memory mapping list 142b is the same as that in FIG.

  As shown in FIG. 7, the frame memory mapping list 142a includes a plurality of entries associated with each decoded picture. In the frame memory mapping list 142a, each entry includes a decoded picture that is used as a reference picture for intra-view reference, a decoded picture that is used as a reference picture for inter-view reference, and a decoding picture that is not used as a reference picture. Corresponds to one of the pictures. In the frame memory mapping list 142b, each entry corresponds to a decoded picture that is used as a reference picture for intra-view reference, and a decoded or decoded picture that is not used as a reference picture.

  Each entry includes a frame memory address 203, a DPB list index 204, an inter-view reference syntax information index 205, an image display status 206, and state information 207.

  The frame memory address 203 indicates the base address of the frame memory 170 in which the decoded picture is stored.

  The DPB list index 204 indicates the index 201 of the DPB list 141a corresponding to the decoded picture. The DPB list index 204 indicates whether or not the decoded picture is used as a reference picture for intra-viewpoint reference. Specifically, when the index 201 of the DPB list 141a is set in the DPB list index 204, it indicates that the decoded picture is used as a reference picture for intra-viewpoint reference. Further, when the DPB list index 204 is not set to the index 201 of the DPB list 141a (in the case of “None” in FIG. 7), it indicates that the decoded picture is not used as a reference picture for intra-viewpoint reference. It is.

  The inter-view reference syntax information index 205 indicates the index 201 of the inter-view reference syntax information list 143 corresponding to the decoded picture. The inter-view reference syntax information index 205 indicates whether or not the decoded picture is used as a reference picture for inter-view reference. Specifically, when the index 201 of the inter-view reference syntax information list 143 is set in the inter-view reference syntax information index 205, the decoded picture is used as a reference picture for inter-view reference. Is shown. When the index 201 of the inter-view reference syntax information list 143 is not set in the inter-view reference syntax information index 205 (in the case of “None” in FIG. 7), the decoded picture is within the viewpoint. It is indicated that it is not used as a reference picture for reference.

  The image display status 206 indicates whether the decoded picture has been displayed (output has been output as the output image signal 112) or has not been displayed (not displayed).

  The status information 207 indicates whether the entry is in use or unused. Specifically, when the DPB list index 204 and the inter-view reference syntax information index 205 are both “none” and the image display status 206 is “0” displayed, the status information 207 is “unused”. In other cases, the status information 207 is “in use”.

  In addition, the control unit 130 notifies the image decoding unit 120 to store the decoded picture newly decoded by the image decoding unit 120 at the base address whose state information 207 is “unused”.

  In the example shown in FIG. 7, the entry whose frame memory address 203 is “0” is used as a decoded picture buffer (DPB), and the corresponding index 201 assigned to the DPB list 141a is assigned. Further, the entry with the frame memory address 203 of “2” stores the inter-view reference picture, and the corresponding index 201 assigned to the inter-view reference syntax information list 143 is assigned. Also, the entry whose frame memory address 203 is “3” cannot be used because it is used as a temporary buffer (a picture not displayed is stored). Therefore, in this example, it can be specified that an entry whose frame memory address 203 is not used as a decoded picture buffer (DPB) and is not used as a temporary buffer is “1”.

  H. With respect to the reference picture list (RPL) defined in the H.264 standard, in the present embodiment, syntax information 113 of each reference picture obtained from the encoded bitstream 110 during decoding of the macroblock layer, Based on the contents of the DPB list 141a or 141b given from the first control unit 131 or the second control unit 132, the first decoding unit 121 or the second decoding unit 122 creates the first decoding unit 121 or the second decoding unit 122. Hold inside.

  The configuration of the inter-view reference frame memory mapping list 144 is the same as that in FIG. In the inter-view reference frame memory mapping list 144, each entry corresponds to a decoded picture used as a reference picture for inter-view reference.

  Further, since the inter-view reference frame memory mapping list 144 is referenced only at the time of inter-view reference, it may include only the frame memory address 203 and the inter-view reference syntax information index 205.

  When the R-side decoded picture is not used as a reference image for inter-view reference, the frame memory mapping list 142b may not include the inter-view reference syntax information index 205.

  As described above, H.P. By associating the decoded picture buffer (DPB) and reference picture list (RPL) 114 defined in the H.264 standard with each component of the present embodiment, as shown in FIG. The base address of the frame memory 170 can be specified from the reference picture number (ref_idx) obtained from the normalized bit stream 110.

  Specifically, the first control unit 131 acquires a reference picture number (ref_idx) obtained from the encoded bitstream 110 for each motion compensation block. Next, the first control unit 131 refers to the RPL 114 and identifies a picture number corresponding to the reference picture number (ref_idx). Next, the first control unit 131 specifies the index 201 of the syntax information 202 corresponding to the specified picture number with reference to the DPB list 141a. Next, the first controller 131 refers to the frame memory mapping list 142a and identifies the base address (frame memory address 203) corresponding to the identified index 201.

  In addition, the second control unit 132 acquires a reference picture number (ref_idx) obtained from the encoded bitstream 110 for each motion compensation block. Next, the second control unit 132 refers to the RPL 114 and identifies the picture number corresponding to the reference picture number (ref_idx). Next, the second control unit 132 refers to the DPB list 141b and the inter-view reference syntax information list 143, and identifies the index 201 of the syntax information 202 corresponding to the identified picture number. Next, the second control unit 132 identifies the base address (frame memory address 203) corresponding to the identified index 201 with reference to the frame memory mapping list 142b and the inter-view reference frame memory mapping list 144.

  H. 264-MVC is a H.264 having a flexible structure as described above. H.264 is extended as a multi-view coding method. That is, H.H. 264-MVC is a technology that realizes high information compression by encoding each viewpoint as a view and performing inter-view reference using the correlation between viewpoints.

  Each view is identified by a view ID (view_id). A view that does not refer to another view is referred to as a base view, and a view that refers to another view is referred to as a non-base view. A feature of each non-base view picture is that it can refer to pictures in different views at the same time in addition to pictures in the same view.

  In the present embodiment, the L-side encoded bit stream 110 is a base view, and the R-side encoded bit stream 111 is a non-base view. That is, when decoding the R-side encoded bitstream 111, the decoded picture of the L-side encoded bitstream 110 can be referred between views.

  The inter-view reference syntax information list 143 stores the reference picture syntax information 113 of the L-side encoded bitstream 111 that is the base view.

  When decoding the R-side encoded bitstream 111 that is a non-base view, the second control unit 132, in addition to the DPB list 141b including the intra-view reference syntax information 113, includes inter-view reference syntax information 113. The inter-view reference syntax information list 143 including is acquired. Based on these, the second decoding unit 122 creates an RPL including an inter-view reference picture.

  FIG. 9 is a diagram illustrating processing of the first control unit 131 and the second control unit 132.

  Specifically, the first control unit 131 uses the inter-view reference syntax information that can refer to the information stored in the index number “1” of the DPB list 141a on the L side as in the example of FIG. It is also stored in the list 143.

  Accordingly, the frame memory management unit 150 determines whether each frame memory area in the frame memory 170 is designated as a reference picture in the DPB lists 141a and 141b, and whether it is designated as an image display target. Manage state. Further, the frame memory management unit 150 manages a state such as whether or not it is designated as a reference picture used for inter-view reference in the inter-view reference syntax information list 143. As a result, the frame memory management unit 150 can specify the area of the frame memory 170 to be released for storing the next picture.

  In addition, since the inter-view reference can refer only to pictures in the same access unit, the inter-view reference syntax information list 143 and the inter-view reference frame memory mapping list 144 are at least for one picture in the present embodiment. It only needs to be able to store information. However, in the present embodiment, since the first decoding unit 121 and the second decoding unit 122 operate simultaneously, the inter-view reference thin is used for writing from the first decoding unit 121 and for reading from the second decoding unit 122. An area for two pictures is secured in each of the tax information list 143 and the inter-view reference frame memory mapping list 144.

  As described above. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100 according to the first embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  In the following, the number of pictures required for reference per stream is three. Further, as described above, one image is required for storing the picture being decoded.

  Further, it is assumed that the image display is displayed at the timing as shown in FIG. 10 after the decoding of the L side picture and the R side picture belonging to the same access unit. Specifically, in the period T01, the decoding process of the L-side picture L1 is performed. Next, in a period T02, decoding processing of the L-side picture L2 and the R-side picture R1 is performed. Next, in a period T03, the picture L1 and the picture R1 are displayed. Note that the picture L1 and the picture R1 are pictures included in the same access unit.

  In this way, one storage area is provided for display in order to start decoding of the next picture before display.

  Therefore, it is assumed that the frame memory 170 has an area capable of storing a total of 10 pictures (5 on the L side and 5 on the R side).

  Also, the input encoded bit streams 110 and 111 are H.264 as described above. Multi-view coding is performed by H.264-MVC. Also, as shown in FIG. 11, it is assumed that the L-side encoded bit stream 110 is encoded in the order of I / P1 / P2 / B1 / P3 / B2. Similarly, it is assumed that the R-side encoded bit stream 111 is encoded in the order of I / P1 / P2 / B1 / P3 / B2. Note that “I” shown in FIG. 11 indicates that the picture is an I picture. “P1, P2, P3” indicates that the picture is a P picture. “B” indicates that the picture is a B picture. An I picture is a picture that has been subjected to intra-frame coding without reference within the view. The P picture and the B picture are pictures encoded by performing intra-view reference. In addition, the I picture and the P picture are pictures that are referenced in the view from other pictures. In this embodiment, the B picture is a picture that is not referenced in the view from other pictures (nal_ref_idc = 0).

  Further, it is assumed that the L-side encoded bit stream 110 is a base view and the R-side encoded bit stream is a non-base view.

  First, the image decoding apparatus 100 starts a decoding process from the L-side encoded bit stream 110. In addition, the first control unit 131 and the second control unit 132 synchronize the decoding processing of the encoded bit streams 110 and 111 by simultaneously starting each picture and waiting for completion as follows.

  First, the decoding process for the picture on the L side will be described.

  The first control unit 131 performs the picture decoding process on the L-side encoded bitstream 110 by controlling the first decoding unit 121 in the following processing steps.

  FIG. 12 is a flowchart of the decoding process on the L side by the image decoding apparatus 100.

  First, the first control unit 131 clears the first completion flag stored in the flag storage unit 180 (S101).

  Next, the first control unit 131 acquires from the frame memory management unit 150 the base address of the frame memory 170 that stores the decoding result of the decoding target picture (S102).

  Next, the first control unit 131 acquires the DPB list 141a on the L side from the management information storage unit 140 (S103).

  Next, the first control unit 131 acquires the frame memory mapping list 142a. Also, the first control unit 131 refers to the acquired DPB list 141a and the frame memory mapping list 142a, and acquires the base address of the reference picture for reference within the view (S104).

  Next, the first control unit 131 sets the decoding target picture and the base address of the reference picture obtained in steps S102 to S104 and the DPB list 141a in the first decoding unit 121, and activates the first decoding unit 121. (S105).

  Next, the first decoding unit 121 acquires a reference picture from the base address of the reference picture set in step S105, and generates a decoded picture by decoding the encoded bitstream 110 using the acquired reference picture. . In addition, the first decoding unit 121 stores the generated decoded picture at the base address of the decoding target picture set in step S105. Further, the first decoding unit 121 notifies the first control unit 131 that the decoding process for the decoding target picture has been completed (S106).

  Next, the first control unit 131 obtains the syntax information 113 of the picture that has been decoded in step S106 from the first decoding unit 121 (S107).

  Next, when the picture for which decoding processing has been completed in step S105 is used as a reference picture when decoding subsequent pictures, the first control unit 131 uses the syntax information 113 acquired in step S107 to manage information The DPB list 141a in the storage unit 140 is updated (S108). Specifically, the first control unit 131 adds the syntax information 113 acquired in step S107 to the DPB list 141a.

  Next, when there is a picture out of the DPB list 141a due to the update of the DPB list 141a in step S108, the first control unit 131 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141a. . When the index 201 is notified from the first controller 131, the frame memory manager 150 updates the frame memory mapping list 142a (S109). Specifically, the frame memory management unit 150 changes the DPB list index 204 corresponding to the notified index 201 to “none”. In addition, the frame memory management unit 150 adds information on the picture for which decoding processing has been completed to the frame memory mapping list 142a.

  Next, the first control unit 131 stores the syntax information 113 obtained in step S107 in the inter-view reference syntax information list 143 as inter-view reference syntax information (S110).

  Next, the first control unit 131 stores the information of the picture for which decoding processing has been completed in the inter-view reference frame memory mapping list 144 (S111).

  Next, the first control unit 131 temporarily stops the decoding process on the L side and sets the first completion flag in the flag storage unit 180 (S112).

  Next, the decoding process for the picture on the R side will be described.

  The second control unit 132 performs the decoding process on the R-side encoded bitstream 111 by controlling the second decoding unit 122 in the following processing steps.

  FIG. 13 is a flowchart of the decoding process on the R side by the image decoding apparatus 100.

  First, the second control unit 132 clears the second completion flag stored in the flag storage unit 180 (S201).

  Next, the second control unit 132 obtains the syntax information of the L-side decoded picture that can be referred to in the decoding process of the R-side encoded bitstream 111 from the inter-view reference syntax information list 143, thereby A reference picture for reference is specified (S202).

  Next, the second control unit 132 refers to the inter-view reference frame memory mapping list 144 and acquires a base address corresponding to the syntax information acquired in step S202. That is, the second control unit 132 acquires the base address of the L-side decoded picture that can be referred to in the decoding process of the R-side encoded bitstream 111 (S203).

  Next, the second control unit 132 acquires the base address of the frame memory 170 that stores the decoding result of the decoding target picture (R-side picture) from the frame memory management unit 150 (S204).

  Next, the second control unit 132 acquires the DPB list 141b on the R side from the management information storage unit 140 (S205).

  Next, the second control unit 132 acquires the frame memory mapping list 142b. Further, the second control unit 132 refers to the acquired DPB list 141b and the frame memory mapping list 142b, and acquires the base address of the reference picture for reference within the view (S206).

  Next, the second control unit 132 determines the decoding target picture obtained in steps S203, S204, S205, and S206, the base address of the reference picture for intra-view reference and the reference picture for inter-view reference, and the decoding target picture. The DPB list 141b is set in the second decryption unit 122, and the second decryption unit 122 is activated (S207).

  The second decoding unit 122 acquires a reference picture from the base address for intra-view reference or inter-view reference set in step S207, and decodes the encoded bitstream 111 using the acquired reference picture. Generate a picture. Further, the second decoding unit 122 stores the generated decoded picture at the base address of the decoding target picture set in step S207. Further, the second decoding unit 122 notifies the second control unit 132 that the decoding process of the picture has been completed (S208).

  Next, the second control unit 132 acquires the syntax information 113 of the picture that has been decoded in step S208 from the second decoding unit 122 (S209).

  Next, the second control unit 132 updates the DPB list 141b when the picture that has been decoded in step S208 is used as a reference picture at the time of decoding the subsequent picture (S210). Specifically, the second control unit 132 adds the syntax information 113 acquired in step S209 to the DPB list 141b.

  Next, the second control unit 132 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141b when there is a picture that is out of the DPB list due to the update of the DPB list 141b in step S210. When the index 201 is notified from the second control unit 132, the frame memory management unit 150 updates the frame memory mapping list 142b (S211). Specifically, the frame memory management unit 150 changes the DPB list index 204 corresponding to the notified index 201 to “none”. In addition, the frame memory management unit 150 adds information on the picture for which decoding processing has been completed to the frame memory mapping list 142b.

  Next, the second control unit 132 temporarily stops the decoding process on the R side and sets a second completion flag in the flag storage unit 180 (S212).

  As described above, the first control unit 131 and the second control unit 132 are simultaneously activated for each picture and wait for completion, thereby synchronizing the decoding processing of the encoded bit streams 110 and 111 as shown in FIG. be able to.

  For example, at time t01 shown in FIG. 14, the L-side decoding process and the R-side decoding process are started simultaneously. That is, the processing shown in FIGS. 12 and 13 is performed. Also, the decoding process on the R side ends at time t02. At time t02, since the decoding process on the L side has not been completed, the first completion flag is not set. Therefore, the second control unit 132 does not start the decoding process for the next picture until the decoding process on the L side is completed.

  At time t03, the decoding process on the L side is completed. Therefore, at time t03, since both the L-side and R-side decoding processes have been completed, the image decoding unit 120 starts the decoding process for the next picture at the same time.

  Also, at time t04, the decoding process on the L side is completed, but the decoding process on the R side is not completed. Therefore, the first control unit 131 decodes the next picture until the decoding process on the R side is completed. Do not start.

  At time t05, the decoding process on the R side is completed. Therefore, at time t05, since both the L-side and R-side decoding processes are completed, the image decoding unit 120 starts decoding the next picture at the same time.

  As described above, after the first decoding unit 121 and the second decoding unit 122 complete the decoding processing of the predetermined decoding processing units (picture units) of the encoded bit streams 110 and 111 together, The first decoding unit 121 and the second decoding unit 122 are caused to start decoding processing for the next decoding processing unit.

  Specifically, the first control unit 131 has completed the decoding processing unit of the encoded bitstream 110 by the first decoding unit 121, and the second completion flag is stored in the flag storage unit 180. In this case, the first decoding unit 121 is caused to start decoding the encoded bitstream 110 of the next decoding processing unit.

  In addition, the second control unit 132, when the decoding processing unit of the encoded bitstream 111 by the second decoding unit 122 is completed and the first completion flag is stored in the flag storage unit 180, The second decoding unit 122 is caused to start the decoding process of the encoded bit stream 111 of the decoding process unit.

  Note that here, the decoding process of the next picture on the L side is waited until the decoding process on the R side is completed, and the decoding process of the next picture on the R side is waited until the decoding process on the L side is completed. However, when the decoding process on the L side is completed before the decoding process on the R side, the decoding process for the next picture on the L side may be performed without waiting for the completion of the decoding process on the R side. This is because the R side uses the L side decoded picture as the reference picture, but the L side does not use the R side decoded picture as the reference picture, so that the reference picture is not decoded even if the decoding process is performed in advance. The problem does not occur.

  However, when the decoding process on the L side is greatly preceded, a large storage area for storing the decoded picture on the L side is required for display. Alternatively, it is conceivable to control the availability of the frame memory including the completion of use on the R side and perform the control to precede the decoding process on the L side as far as possible, but the control becomes complicated. Therefore, as shown in FIG. 14, the decoding process of the next picture on the L side is made to wait until the decoding process on the R side is completed, and the decoding of the next picture on the R side is performed until the decoding process on the L side is completed. It is preferable to wait for processing.

  Here, in the conventional image decoding apparatus, there is a synchronization problem regarding the decoding processing of a plurality of bit streams.

  Specifically, H.C. In the process of decoding the image data compressed by the encoding method using inter-view prediction such as 264-MVC, the non-base view image is decoded by performing motion compensation on the decoded image of the base view at the same display time. Sometimes the reference image is used. Therefore, there is a dependency relationship between the decoding process timing of the base view and the decoding process timing of the non-base view. In the case of three or more viewpoints, in decoding a non-base view image, a non-base view or base view decoded image having the same display time may be used as a reference image at the time of motion compensation.

  Therefore, since it is necessary to decode a plurality of bit streams while synchronizing the decoding processing of each channel for each predetermined unit, the decoding timings of the images of each channel are different from each other as in the conventional image decoding device. In the case of a configuration in which the playback is performed without considering the decoding status of the other channels and each channel is synchronized for display, the reference between the channels at the time of decoding is not performed well, and the decoding process can be performed correctly. The problem of not occurring.

  On the other hand, the image decoding apparatus 100 according to Embodiment 1 of the present invention activates the first control unit 131 and the second control unit 132 at the same time and waits for the completion of the encoding bitstream 110 and 111 decoding processes can be synchronized.

  In addition, the image decoding apparatus 100 according to Embodiment 1 of the present invention performs L-side encoding via the inter-view reference syntax information list 143 that can be commonly accessed during decoding of the R-side encoded bitstream 111. It is possible to know the syntax management information of the reference picture of the bitstream. As a result, the image decoding apparatus 100 can decode the encoded bitstream encoded using inter-view prediction.

  Next, the relationship of the DPB lists 141a and 141b, the frame memory mapping lists 142a and 142b, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time is shown in FIG. Will be described.

(Period T10)
(Decoding process of L-side encoded bit stream 110)
The decoded I picture (LI) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “0” of the DPB list 141a. Also, the base address is registered in the index number “0” of the frame memory mapping list 142a.

  Also, syntax information 113 of I picture (LI) is registered in the inter-view reference syntax information list 143. Further, the base address of the I picture (LI) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Decoding process of R side encoded bit stream 111)
Since an inter-view reference may occur, processing of the R-side encoded bitstream is not performed in the period T10, and the process starts from the period T11.

(Period T11)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P1) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “1” of the DPB list 141a. Also, the base address is registered in the index number “1” of the frame memory mapping list 142a. Also, the syntax information 113 of the P picture (P11) is registered in the index number “1” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-Pl) is registered in the index number “1” of the inter-view reference frame memory mapping list 144.

(Decoding process of R side encoded bit stream 111)
The decoded I picture (RI) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. Also, the base address is registered in the index number “0” of the frame memory mapping list 142b.

(Period T12)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P2) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “2” of the DPB list 141b. Also, the base address is registered in the index number “2” of the frame memory mapping list 142b. Also, the syntax information 113 of the P picture (L-P2) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P2) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Decoding process of R-side encoded bitstream 111)
The decoded P picture (R-P1) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “1” of the DPB list 141b. Also, the base address is registered in the index number “1” of the frame memory mapping list 142b.

(Period T13)
(Decoding process of L side encoded bit stream 110)
The decoded B picture (L-B1) is stored in the frame memory 170, and the base address is registered in the index number “3” of the frame memory mapping list 142a. However, since the B picture (L-B1) is not referred to when the subsequent picture is decoded, the syntax information 113 is not registered in the DPB list 141a.

  Also, the B picture (L-B1) is registered in the index number “1” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-B1) is registered in the index number “1” of the inter-view reference frame memory mapping list 144.

(Decoding process of R-side encoded bitstream 111)
The decoded P picture (R-P2) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “2” of the DPB list 141b. Also, the base address is registered in the index number “2” of the frame memory mapping list 142b.

(Image display of L side encoded bit stream 110)
Although the I picture (LI) is displayed as an image, the I picture (L-1) can be used as a reference picture when the subsequent picture is decoded, and is registered in the index number “0” of the DPB list 141a. The syntax information 113 is not deleted. Similarly, the base address registered in the index number “0” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
Although the P picture (R-P1) is displayed as an image, the P picture (R-P1) can be used as a reference picture when the subsequent picture is decoded, and is registered in the index number “0” of the DPB list 141b. The syntax information 113 is not deleted. Similarly, the base address registered in the index number “0” of the frame memory mapping list 142b is not deleted.

(Period T14)
(Decoding process of L side encoded bit stream 110)
The decoded P picture (L-P3) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. The base address is registered in the index number “4” of the frame memory mapping list 142b. At this time, the syntax information 113 of the I picture (LI) registered in the index number “0” of the DPB list 141a and the I picture registered in the index number “0” of the frame memory mapping list 142a ( LI) base address is deleted. Also, the P picture (L-P3) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P3) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Decoding process of R-side encoded bitstream 111)
The decoded B picture (R-B1) is stored in the frame memory 170, and the base address is registered in the index number “3” of the frame memory mapping list 142b. Further, since the B picture (R-B1) is not referred to when the subsequent picture is decoded, the syntax information 113 is not registered in the DPB list 141b.

(Image display of L side encoded bit stream 110)
Although the P picture (L-P1) is displayed as an image, the syntax information registered in the index number “1” of the DPB list 141a is not deleted because it can become a reference picture when the subsequent picture is decoded. Similarly, the base address registered in the index number “1” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
Although the P picture (R-P1) is displayed as an image, the syntax information registered in the index number “1” of the DPB list 141b is not deleted because it can become a reference picture when the subsequent picture is decoded. Similarly, the base address registered in the index number “1” of the frame memory mapping list 142b is not deleted.

(Period T15)
(Decoding process of L side encoded bit stream 110)
The decoded B picture (L-B2) is stored in the frame memory 170, and the base address is registered in the index number “0” of the frame memory mapping list 142a. Further, since the B picture (L-B2) is not referred to when the subsequent picture is decoded, the syntax information 113 is not registered in the DPB list 141a.

  Also, the B picture (L-B2) is registered in the index number “1” of the inter-view reference syntax information list 143. Also, the base address of the B picture (L-B2) is registered in the index number “1” of the inter-view reference frame memory mapping list 144.

(Decoding process of R-side encoded bitstream 111)
The decoded P picture (R-P3) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. The base address is registered in the index number “4” of the frame memory mapping list 142b. At this time, the syntax information 113 of the I picture (LI) registered in the index number “0” of the DPB list 141b and the I picture registered in the index number “0” of the frame memory mapping list 142b ( LI) base address is deleted.

(Image display of L side encoded bit stream 110)
Since the B picture (L-B1) is displayed and the B picture (L-B1) is not used for inter-view reference, the base registered from the index number “3” of the frame memory mapping list 142a is used. The address is deleted.

(Image display of R side encoded bit stream 111)
Since the B picture (R-B1) is displayed and the B picture (R-B1) is not used for inter-view reference, the base registered from the index number “3” in the frame memory mapping list 142b is used. The address is deleted.

(Period T16)
(Decoding process of L side encoded bit stream 110)
The decoded P picture (L-P4) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “1” of the DPB list 141b. Also, the base address is registered in the index number “3” of the frame memory mapping list 142b. At this time, the syntax information 113 of the P picture (L-P1) registered in the index number “1” of the DPB list 141a and the I picture registered in the index number “1” of the frame memory mapping list 142a ( The base address of L-P1) is deleted. Also, the P picture (L-P4) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P3) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Decoding process of R-side encoded bitstream 111)
The decoded B picture (R-B2) is stored in the frame memory 170, and the base address is registered in the index number “3” of the frame memory mapping list 142b. Further, since the B picture (R-B2) is not referred to when the subsequent picture is decoded, the syntax information 113 is not registered in the DPB list 141b.

(Image display of L side encoded bit stream 110)
Although the P picture (L-P2) is displayed as an image, the syntax information 113 registered in the index number “2” of the DPB list 141a is not deleted because it can become a reference picture when the subsequent picture is decoded. Similarly, the base address registered in the index number “2” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
Although the P picture (R-P2) is displayed as an image, the syntax information registered in the index number “2” of the DPB list 141b is not deleted because it can become a reference picture when the subsequent picture is decoded. Similarly, the base address registered in the index number “2” of the frame memory mapping list 142b is not deleted.

  As described above, the image decoding apparatus 100 according to Embodiment 1 of the present invention activates the first control unit 131 and the second control unit 132 at the same time, and waits for completion to thereby encode the bit stream 110. And 111 can be synchronized.

  Note that the image decoding apparatus 100 according to Embodiment 1 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the first control unit 131 and the second control unit 132 are synchronized by waiting in picture units (frame units). However, in the case of a field structure or the like, two field picture units are used. It is also possible to synchronize with. In addition, the first control unit 131 and the second control unit 132 are H.264. Synchronization may be performed in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 131 and the second control unit 132 described in the present embodiment may actually be equipped with two processors or may be two logical threads.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  Further, in the present embodiment, the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b are individually managed by the R side encoded bit stream 111 and the L side encoded bit stream 110. At least one of the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b may be managed by one list, and a unified index may be assigned.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

(Embodiment 2)
FIG. 16 is a diagram showing a configuration of an image decoding device 100A according to Embodiment 2 of the present invention.

  The configuration of image decoding apparatus 100A according to Embodiment 2 of the present invention will be described below. Elements similar to those in FIG. 2 are denoted by the same reference numerals, and redundant description is omitted.

  Specifically, in the image decoding device 100A illustrated in FIG. 16, the configuration of the image decoding unit 220 is different from the configuration of the image decoding unit 120 illustrated in FIG.

  The image decoding unit 220 includes a selection unit 221 and a decoding unit 222.

  The selection unit 221 selects one of the encoded bit streams 110 and 111 and outputs the selected encoded bit stream to the decoding unit 222.

  The decoding unit 222 decodes the encoded bit stream output from the selection unit 221. Note that the internal configuration of the decoding unit 222 is the same as that of the first decoding unit 121 and the second decoding unit 122 described in Embodiment 1.

  The encoded bit streams 110 and 111 input to the image decoding apparatus according to the present embodiment are H.264 and H.264. It is encoded using 264-MVC. H. The description of the characteristics of the 264-MVC encoding and the relationship with the configuration of the image decoding apparatus of the present embodiment is as already described in the first embodiment.

  As described above. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100A according to the second embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  In the present embodiment, the number of pictures required for reference per stream is three. Further, as described above, one image is required for storing the image being decoded.

  Further, it is assumed that the image display is displayed at the timing as shown in FIG. 17 after the decoding of the L side picture and the R side picture belonging to the same access unit. Specifically, in the period T21, the decoding process for the L-side picture L1 is performed. Next, in the period T22, a decoding process for the R-side picture R1 is performed. Next, in the period T23, the decoding process of the picture L1 on the L side is performed. In the period T24, the picture L1 and the picture R1 are displayed. Note that the picture L1 and the picture R1 are pictures included in the same access unit.

  In this way, one storage area is provided for display in order to start decoding of the next picture during display.

  Therefore, it is assumed that the frame memory 170 has an area capable of storing a total of 10 pictures (5 on the L side and 5 on the R side).

  In the present embodiment, since the decoding unit 222 alternately performs the decoding process of the encoded bit stream 110 and the encoded bit stream 111, the inter-view reference syntax information list 143 and the inter-view reference frame memory mapping list 144 are performed. The area for one picture is secured.

  The configuration of the input encoded bit streams 110 and 111 is the same as that of the first embodiment.

  First, the image decoding device 100A according to the present embodiment starts the decoding process from the L-side encoded bitstream 110. Further, the first control unit 131 and the second control unit 132 synchronize the decoding processing of the encoded bit streams 110 and 111 by alternately operating as follows.

  First, the decoding process for the picture on the L side will be described.

  The first control unit 131 controls the decoding unit 222 and performs the picture decoding process of the L-side encoded bitstream 110 in the following processing steps. Note that the processing procedure by the first control unit 131 is the same as the processing shown in FIG. 12, and therefore will be described with reference to FIG.

  First, the first control unit 131 clears the first completion flag stored in the flag storage unit 180 (S101).

  Next, the first control unit 131 acquires from the frame memory management unit 150 the base address of the frame memory 170 that stores the decoding result of the decoding target picture (S102).

  Next, the first control unit 131 acquires the DPB list 141a on the L side from the management information storage unit 140 (S103).

  Next, the first control unit 131 acquires the frame memory mapping list 142a. Also, the first control unit 131 refers to the acquired DPB list 141a and the frame memory mapping list 142a, and acquires the base address of the reference picture for reference within the view (S104).

  Next, the first control unit 131 sets the decoding target picture and the base address of the reference picture obtained in steps S102 to S104, and the DPB list 141a in the first decoding unit 121. In addition, the first control unit 131 sets the selection unit 221 so as to select the L-side encoded bitstream 110. Further, the first control unit 131 activates the decryption unit 222 (S105).

  Next, the decoding unit 222 acquires a reference picture from the base address of the reference picture set in step S105, and generates a decoded picture by decoding the encoded bitstream 110 using the acquired reference picture. In addition, the decoding unit 222 stores the generated decoded picture at the base address of the decoding target picture set in step S105. In addition, the decoding unit 222 notifies the first control unit 131 that the decoding process for the decoding target picture has been completed (S106).

  Next, the first control unit 131 acquires the syntax information 113 of the picture that has been decoded in step S106 from the decoding unit 222 (S107).

  Next, when the picture for which decoding processing has been completed in step S105 is used as a reference picture when decoding subsequent pictures, the first control unit 131 uses the syntax information 113 acquired in step S107 to manage information The DPB list 141a in the storage unit 140 is updated (S108).

  Next, when there is a picture out of the DPB list 141a due to the update of the DPB list 141a in step S108, the first control unit 131 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141a. . When the index 201 is notified from the first controller 131, the frame memory manager 150 updates the frame memory mapping list 142a (S109).

  Next, the first control unit 131 stores the syntax information 113 obtained in step S107 in the inter-view reference syntax information list 143 as inter-view reference syntax information (S110).

  Next, the first control unit 131 stores the base address obtained in step S102 in the inter-view reference frame memory mapping list 144 (S111).

  Next, the first control unit 131 temporarily stops the decoding process on the L side and sets the first completion flag in the flag storage unit 180 (S112).

  Next, the decoding process for the picture on the R side will be described.

  After the series of decoding processing of the L-side encoded bitstream 110 is completed in Steps S101 to S112, the second control unit 132 controls the decoding unit 222 in the following processing steps to perform the R-side encoded bitstream. 111 decoding processing is performed.

  The processing procedure by the second control unit 132 is the same as the processing shown in FIG. 13, and will be described with reference to FIG.

  First, the second control unit 132 clears the second completion flag stored in the flag storage unit 180 (S201).

  Next, the second control unit 132 uses the inter-view reference syntax information list in the management information storage unit 140 for the syntax information 113 of the L-side decoded picture that can be referred to in the decoding process of the R-side encoded bitstream 111. 143, the inter-view reference picture is specified (S202).

  Next, the second control unit 132 obtains the base address of the L-side decoded picture that can be referenced in the decoding process of the R-side encoded bitstream 111 from the inter-view reference frame memory mapping list 144 in the management information storage unit 140. Obtain (S203).

  Next, the second control unit 132 acquires the base address of the frame memory 170 that stores the decoding result of the decoding target picture (R-side picture) from the frame memory management unit 150 (S204).

  Next, the second control unit 132 acquires the DPB list 141b on the R side from the management information storage unit 140 (S205).

  Next, the second control unit 132 acquires the frame memory mapping list 142b. Further, the second control unit 132 refers to the acquired DPB list 141b and the frame memory mapping list 142b, and acquires the base address of the reference picture for reference within the view (S206).

  Next, the second control unit 132 determines the decoding target picture obtained in steps S203, S204, S205, and S206, the base address of the reference picture for intra-view reference and the reference picture for inter-view reference, and the decoding target picture. The DPB list 141b is set in the decryption unit 222. In addition, the second control unit 132 sets the selection unit 221 so as to select the R-side encoded bitstream 111. Further, the second control unit 132 activates the decoding unit 222 (S207).

  Next, the decoding unit 222 acquires a reference picture from the base address for intra-view reference or inter-view reference set in step S207, and decodes the encoded bitstream 111 using the acquired reference picture. A decoded picture is generated. Also, the decoding unit 222 stores the generated decoded picture at the base address of the decoding target picture set in step S207. In addition, the decoding unit 222 notifies the second control unit 132 that the decoding process of the picture has been completed (S208).

  Next, the second control unit 132 acquires the syntax information 113 of the picture that has been decoded in step S208 from the decoding unit 222 (S209).

  Next, the second control unit 132 updates the DPB list 141b when the picture that has been decoded in step S208 is used as a reference picture at the time of decoding the subsequent picture (S210).

  Next, the second control unit 132 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141b when there is a picture that is out of the DPB list due to the update of the DPB list 141b in step S210. When the index 201 is notified from the second control unit 132, the frame memory management unit 150 updates the frame memory mapping list 142b (S211).

  Next, the second control unit 132 temporarily stops the decoding process on the R side and sets a second completion flag in the flag storage unit 180 (S212).

  As described above, by alternately operating the first control unit 131 and the second control unit 132, the decoding processing of the encoded bit streams 110 and 111 can be synchronized as shown in FIG.

  For example, the decoding process on the L side is started at time t11 shown in FIG. That is, the process shown in FIG. 12 described above is performed. At time t12, the L-side decoding process ends, and then the R-side decoding process is started. That is, the process shown in FIG. 13 described above is performed. At time t13, the R-side decoding process ends, and then the L-side decoding process starts.

  Thus, the control unit 130 causes the selection unit 221 to select one of the encoded bitstreams 110 and 111. The control unit 130 also causes the selection unit 221 to select the other signal of the encoded bitstreams 110 and 111 after the decoding process of a predetermined decoding process unit of the one signal is completed.

  Specifically, when the second completion flag is stored in the flag storage unit 180, the first control unit 131 causes the decoding unit 222 to start the decoding process of the encoded bitstream 110 in the next decoding process unit.

  In addition, when the first completion flag is stored in the flag storage unit 180, the second control unit 132 causes the decoding unit 222 to start the decoding process of the encoded bitstream 111 in the next decoding processing unit.

  Next, the relationship of the DPB lists 141a and 141b, the frame memory mapping lists 142a and 142b, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time is shown in FIG. 19 will be used for explanation.

(Period T30)
(Decoding process of L-side encoded bit stream 110)
The decoded I picture (LI) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “0” of the DPB list 141a. Also, the base address is registered in the index number “0” of the frame memory mapping list 142a.

  Also, the syntax information 113 of the I picture (LI) is registered in the index number “0” of the inter-view reference syntax information list 143. Also, the base address of the P picture (LI) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Period T31)
(Decoding process of R side encoded bit stream 111)
The decoded I picture (RI) is stored in the frame memory, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. Also, the base address is registered in the index number “0” of the frame memory mapping list 142b.

(Period T32)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P1) is stored in the frame memory, and the syntax information 113 is registered in the index number “1” of the DPB list 141a. Also, the base address is registered in the index number “1” of the frame memory mapping list 142a.

  Also, the syntax information 113 of the P picture (L-P1) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P1) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Period T33)
(Decoding process of R side encoded bit stream 111)
The decoded P picture (R-P1) is stored in the frame memory, and the syntax information 113 is registered in the index number “1” of the DPB list 141b. Also, the base address is registered in the index number “1” of the frame memory mapping list 142b.

(Period T34)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P2) is stored in the frame memory, and the syntax information 113 is registered in the index number “2” of the DPB list 141a. Also, the base address is registered in the index number “2” of the frame memory mapping list 142a.

  Also, the syntax information 113 of the P picture (L-P2) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P2) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Image display of L side encoded bit stream 110)
Although the I picture (LI) is displayed after the decoding process, the syntax information 113 registered in the index number “0” of the DPB list 141a is not deleted because it can become a reference picture when the subsequent picture is decoded. . Similarly, the base address registered in the index number “0” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
The I picture (RI) is displayed after the decoding process, but the syntax information 113 registered in the index number “0” of the DPB list 141b is not deleted because it can become a reference picture when decoding the subsequent picture. . Similarly, the base address registered in the index number “0” of the frame memory mapping list 142b is not deleted.

(Period T35)
(Decoding process of R side encoded bit stream 111)
The decoded P picture (R-P2) is stored in the frame memory, and the syntax information 113 is registered in the index number “2” of the DPB list 141b. Also, the base address is registered in the index number “2” of the frame memory mapping list 142b.

(Period T36)
(Decoding process of L-side encoded bit stream 110)
The decoded B picture (L-B1) is stored in the frame memory, and the base address is registered in the index number “3” of the frame memory mapping list 142a. However, since it is not referred to when decoding subsequent pictures, it is stored in the DPB list 141a. The syntax information 113 is not registered.

(Image display of L side encoded bit stream 110)
The I picture (L-P1) is displayed after decoding processing, but the syntax information 113 registered in the index number “1” of the DPB list 141a is not deleted because it can become a reference picture when decoding the subsequent picture. . Similarly, the base address registered in the index number “1” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
Although the I picture (R-P1) is displayed after decoding processing, the syntax information 113 registered in the index number “1” of the DPB list 141b is not deleted because it can become a reference picture when the subsequent picture is decoded. . Similarly, the base address registered in the index number “1” of the frame memory mapping list 142b is not deleted.

(Period T37)
(Decoding process of R side encoded bit stream 111)
The decoded B picture (R-B1) is stored in the frame memory, and the base address is registered in the index number “3” of the frame memory mapping list 142b. However, since it is not referred to when decoding subsequent pictures, it is stored in the DPB list 141b. The syntax information 113 is not registered.

(Period T38)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P3) is stored in the frame memory, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. The base address is registered in the index number “4” of the frame memory mapping list 142b. At this time, the syntax information 113 of the I picture (LI) registered in the index number “0” of the DPB list 141a and the I picture registered in the index number “0” of the frame memory mapping list 142a ( LI) base address is deleted. Also, the syntax information 113 of the P picture (L-P3) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P3) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Image display of L-side encoded bitstream 111)
Since the B picture (L-B1) is displayed and the B picture (L-B1) is not used for inter-view reference, the base registered from the index number “3” of the frame memory mapping list 142a is used. The address is deleted.

(Image display of R side encoded bit stream 111)
Since the B picture (R-B1) is displayed and the B picture (R-B1) is not used for inter-view reference, the base registered from the index number “3” of the frame memory mapping list 142a is used. The address is deleted.

(Period T39)
(Decoding process of R side encoded bit stream 111)
The decoded P picture (R-P3) is stored in the frame memory, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. The base address is registered in the index number “4” of the frame memory mapping list 142b. At this time, the syntax information of the I picture (LI) registered in the index number “0” of the DPB list 141b and the I picture (L) registered in the index number “0” of the frame memory mapping list 142b. The base address of -I) is deleted.

  As described above, the image decoding apparatus 100A according to Embodiment 2 of the present invention operates the first control unit 131 and the second control unit 132 alternately to perform the decoding process of the encoded bit streams 110 and 111. Can be synchronized.

  In addition, the image decoding apparatus 100A according to Embodiment 2 of the present invention performs L-side encoding via the inter-view reference syntax information list 143 that can be commonly accessed when decoding the R-side encoded bitstream 111. It is possible to know the syntax management information of the reference picture of the bitstream 110. As a result, the image decoding device 100A can decode the encoded bitstream encoded using inter-view prediction.

  Note that the image decoding device 100A according to Embodiment 2 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the first control unit 131 and the second control unit 132 are synchronized by waiting in picture units (frame units). However, in the case of a field structure or the like, two field picture units are used. It is also possible to synchronize with. In addition, the first control unit 131 and the second control unit 132 are H.264. Synchronization may be performed in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 131 and the second control unit 132 described in the present embodiment may actually be equipped with two processors or may be two logical threads.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  Further, in the present embodiment, the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b are individually managed by the R side encoded bit stream 111 and the L side encoded bit stream 110. At least one of the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b may be managed by one list, and a unified index may be assigned.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

(Embodiment 3)
FIG. 20 is a diagram showing a configuration of an image decoding device 100B according to Embodiment 3 of the present invention.

  The configuration of image decoding apparatus 100B according to Embodiment 3 of the present invention will be described below. Note that the same elements as those in FIG. 16 are denoted by the same reference numerals, and redundant description is omitted.

  Specifically, in the image decoding device 100B illustrated in FIG. 20, the configuration of the management information storage unit 240 is different from the configuration of the management information storage unit 140 illustrated in FIG. More specifically, the management information storage unit 240 stores a frame memory mapping list 142c instead of the frame memory mapping lists 142a and 142b shown in FIG.

  In this embodiment, the state of the frame memory 170 is managed using this frame memory mapping list 142c.

  FIG. 21 shows the structure of the frame memory mapping list 142c. For example, the frame memory mapping list 142c has a table format as shown in FIG. Elements similar to those in FIG. 7 are denoted by the same reference numerals.

  As shown in FIG. 21, the frame memory mapping list 142a includes a plurality of entries associated with each decoded picture. Each entry includes a frame memory address 203, DPB list indexes 204a and 204b, an inter-view reference syntax information index 205, an image display status 206, and state information 207.

  The DPB list index 204a indicates the index 201 of the DPB list 141a corresponding to the decoded picture.

  The DPB list index 204b indicates the index 201 of the DPB list 141b corresponding to the decoded picture.

  In the example shown in FIG. 21, entries whose frame memory addresses 203 are “0” and “3” are used as decoded picture buffers (DPBs) for L-side decoding processing, and the corresponding indexes assigned to the DPB list 141a. 201 is assigned. In addition, entries whose frame memory addresses 203 are “1” and “4” are used as decoded picture buffers (DPBs) for the R-side decoding process, and the corresponding index 201 assigned to the DPB list 141b is assigned. Yes. An entry whose frame memory address 203 is “5” is used as an inter-view reference picture, and the corresponding index 201 assigned to the inter-view reference syntax information list 143 is assigned. An entry whose frame memory address 203 is “6” cannot be used because it is used as a temporary buffer (a picture not displayed is stored). Therefore, in this example, it can be specified that the entries having “1” and “6” in the frame memory address 203 that are not used as a decoded picture buffer (DPB) and are not used as a temporary buffer are unused.

  The encoded bit streams 110 and 111 input to the image decoding apparatus 100B according to the present embodiment are H.264 and H.264. It is encoded using 264-MVC. H. The description of the characteristics of the 264-MVC encoding and the relationship with the configuration of the image decoding apparatus of the present embodiment is as already described in the first embodiment.

  As described above. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100B according to the third embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  In the present embodiment, as in the second embodiment, the number of pictures required for reference per stream is three. Further, as described above, one image is required for storing the picture being decoded. Further, it is assumed that the image display is displayed at the timing as shown in FIG. 17 after the decoding of the L side picture and the R side picture belonging to the same access unit. Therefore, one storage area is provided for display in order to start decoding of the next picture during display. However, by managing this storage area in common for the L-side encoded bit stream and the R-side encoded bit stream, the storage area can be reduced by one as compared with the second embodiment. Therefore, it is assumed that the frame memory 170 has an area capable of storing a total of nine pictures.

  First, the image decoding device 100B according to the present embodiment starts decoding processing from the L-side encoded bitstream 110. Further, the first control unit 131 and the second control unit 132 synchronize the decoding processing of the encoded bit streams 110 and 111 by operating alternately as in the second embodiment.

  Hereinafter, the relationship of the DPB lists 141a and 141b, the frame memory mapping list 142c, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time will be described with reference to FIG. I will explain.

(Period T40)
(Decoding process of L-side encoded bit stream 110)
The decoded I picture (LI) is stored in the frame memory 170, and the syntax information 113 is registered in the index number “0” of the DPB list 141a. Also, the base address is registered in the index number “0” of the frame memory mapping list 142a.

  Also, the syntax information 113 of the I picture (LI) is registered in the index number “0” of the inter-view reference syntax information list 143. Also, the base address of the P picture (LI) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Period T41)
(Decoding process of R side encoded bit stream 111)
The decoded I picture (RI) is stored in the frame memory, and the syntax information 113 is registered in the DPB list 141b index number “0”. Also, the base address is registered in the index number “1” of the frame memory mapping list 142b.

(Period T42)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P1) is stored in the frame memory, and the syntax information 113 is registered in the index number “1” of the DPB list 141a. Also, the base address is registered in the index number “2” of the frame memory mapping list 142a.

  Also, the syntax information 113 of the P picture (L-P1) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P1) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Period T43)
(Decoding process of R side encoded bit stream 111)
The decoded P picture (R-P1) is stored in the frame memory, and the syntax information 113 is registered in the index number “1” of the DPB list 141b. Also, the base address is registered in the index number “3” of the frame memory mapping list 142b.

(Period T44)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P2) is stored in the frame memory, and the syntax information 113 is registered in the index number “2” of the DPB list 141a. Further, the base address is registered in the index number “4” of the frame memory mapping list 142a.

  Also, the syntax information 113 of the P picture (L-P2) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P2) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Image display of L side encoded bit stream 110)
Although the I picture (LI) is displayed after the decoding process, the syntax information 113 registered in the index number “0” of the DPB list 141a is not deleted because it can become a reference picture when the subsequent picture is decoded. . Similarly, the base address registered in the index number “0” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
The I picture (RI) is displayed after the decoding process, but the syntax information 113 registered in the index number “0” of the DPB list 141b is not deleted because it can become a reference picture when decoding the subsequent picture. . Similarly, the base address registered in the index number “1” of the frame memory mapping list 142b is not deleted.

(Period T45)
(Decoding process of R side encoded bit stream 111)
The decoded P picture (R-P2) is stored in the frame memory, and the syntax information 113 is registered in the index number “2” of the DPB list 141b. The base address is registered in the index number “5” of the frame memory mapping list 142b.

(Period T46)
(Decoding process of L-side encoded bit stream 110)
The decoded B picture (L-B1) is stored in the frame memory, and the base address is registered in the index number “6” of the frame memory mapping list 142a. However, since it is not referred to when decoding subsequent pictures, it is stored in the DPB list 141a. The syntax information 113 is not registered.

(Image display of L side encoded bit stream 110)
The I picture (L-P1) is displayed after decoding processing, but the syntax information 113 registered in the index number “1” of the DPB list 141a is not deleted because it can become a reference picture when decoding the subsequent picture. . Similarly, the base address registered in the index number “2” of the frame memory mapping list 142a is not deleted.

(Image display of R side encoded bit stream 111)
Although the I picture (R-P1) is displayed after decoding processing, the syntax information 113 registered in the index number “1” of the DPB list 141b is not deleted because it can become a reference picture when the subsequent picture is decoded. . Similarly, the base address registered in the index number “3” of the frame memory mapping list 142b is not deleted.

(Period T47)
(Decoding process of R side encoded bit stream 111)
The decoded B picture (R-B1) is stored in the frame memory, and the base address is registered in the index number “7” of the frame memory mapping list 142b. However, since it is not referred to when the subsequent picture is decoded, it is stored in the DPB list 141b. The syntax information 113 is not registered.

(Period T48)
(Decoding process of L-side encoded bit stream 110)
The decoded P picture (L-P3) is stored in the frame memory, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. The base address is registered in the index number “8” of the frame memory mapping list 142b. At this time, the syntax information of the I picture (LI) registered in the index number “0” of the DPB list 141a and the I picture (L) registered in the index number “0” of the frame memory mapping list 142a. The base address of -I) is deleted. Also, the syntax information 113 of the P picture (L-P3) is registered in the index number “0” of the inter-view reference syntax information list 143. Further, the base address of the P picture (L-P3) is registered in the index number “0” of the inter-view reference frame memory mapping list 144.

(Image display of L-side encoded bitstream 111)
Since the B picture (L-B1) is displayed and the B picture (L-B1) is not used for inter-view reference, the base registered from the index number “6” in the frame memory mapping list 142a is used. The address is deleted.

(Image display of R side encoded bit stream 111)
Since the B picture (R-B1) is displayed and the B picture (R-B1) is not used for inter-view reference, the base registered from the index number “7” in the frame memory mapping list 142a is used. The address is deleted.

(Period T49)
(Decoding process of R side encoded bit stream 111)
The decoded P picture (R-P3) is stored in the frame memory, and the syntax information 113 is registered in the index number “0” of the DPB list 141b. Also, the base address is registered in the index number “0” of the frame memory mapping list 142b. At this time, the syntax information of the I picture (LI) registered in the index number “0” of the DPB list 141b and the I picture (L) registered in the index number “1” of the frame memory mapping list 142b. The base address of -I) is deleted.

  As described above, the image decoding device 100B according to Embodiment 3 of the present invention operates the first control unit 131 and the second control unit 132 alternately to perform the decoding process of the encoded bit streams 110 and 111. Can be synchronized.

  In addition, the image decoding apparatus 100B according to Embodiment 3 of the present invention performs L-side encoding via the inter-view reference syntax information list 143 that can be commonly accessed during decoding of the R-side encoded bitstream 111. It is possible to know the syntax management information of the reference picture of the bitstream 110. As a result, the image decoding device 100B can decode the encoded bitstream encoded using inter-view prediction.

  Furthermore, the image decoding apparatus 100B can reduce the capacity of the frame memory 170 by managing the frame memory mapping list 142c in common for the L-side encoded bit stream 110 and the R-side encoded bit stream 111. Therefore, the image decoding device 100B can reduce the cost.

  Note that the image decoding apparatus 100B according to Embodiment 3 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the first control unit 131 and the second control unit 132 are synchronized by waiting in picture units (frame units). However, in the case of a field structure or the like, two field picture units are used. It is also possible to synchronize with. In addition, the first control unit 131 and the second control unit 132 are H.264. Synchronization may be performed in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 131 and the second control unit 132 described in the present embodiment may actually be equipped with two processors or may be two logical threads.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  In this embodiment, the DPB lists 141a and 141b are individually managed by the R-side encoded bitstream 111 and the L-side encoded bitstream 110. However, the index managed and unified by one list is used. It may be given.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

(Embodiment 4)
FIG. 23 is a block diagram showing a configuration of an image decoding apparatus 100C according to Embodiment 4 of the present invention. Elements similar to those in FIG. 2 are denoted by the same reference numerals, and redundant description is omitted.

  An image decoding device 100C illustrated in FIG. 23 further includes a stream combination unit 190 in addition to the configuration of the image decoding device 100 illustrated in FIG. The image decoding device 100 </ b> C includes an image decoding unit 320 instead of the image decoding unit 120.

  The stream combining unit 190 combines the encoded bit stream 110 and the encoded bit stream 111 into one encoded bit stream 310. Specifically, the stream combining unit 190 generates one encoded bit stream 310 by alternately arranging the encoded bit stream 110 and the encoded bit stream 111 for each predetermined encoding unit.

  The image decoding unit 320 includes a decoding unit 321. The decoding unit 321 alternately decodes the encoded bit stream 110 and the encoded bit stream 111 included in the encoded bit stream 310 for each encoding unit. The configuration of the decoding unit 321 is the same as the configuration of the first decoding unit 121 and the second decoding unit 122 shown in FIG.

  The encoded bit streams 110 and 111 input to the image decoding apparatus 100C of the present embodiment are H.264 and H.264. It is encoded using 264-MVC. H. The description of the characteristics of the 264-MVC encoding and the relationship with the configuration of the image decoding apparatus of the present embodiment is as already described in the first embodiment.

  Here, H. It supplements about the structure of the stream of 264-MVC.

  H. In the H.264 standard, it is defined that encoded information is separated into layers called NAL units. Each NAL unit is assigned an identifiable code called a start code. Further, the type of the NAL unit can be discriminated by the NAL header following the start code. Examples of the NAL unit type include a sequence parameter set, a picture parameter set, and a slice.

  H. In 264-MVC, a view (base view or non-base view) to which the above-described sequence parameter set, picture parameter set, slice, and the like belong can be determined by analyzing the NAL header.

  The stream combining unit 190 performs stream analysis of the encoded bit streams 110 and 111, and outputs the encoded bit stream 310 by alternately outputting the encoded bit streams 110 and 111 in the state of the encoded bit stream in a predetermined unit. Generate. In the present embodiment, the stream combining unit 190 starts the L-side encoded bitstream 110 at the head, and alternately outputs the encoded bitstreams 110 and 111 in units of one picture, thereby encoding the encoded bitstream 310 that is picture-interleaved. Is generated. In other words, the stream combining unit 190 arranges the encoded bitstream 110 in units of one picture out of the encoded bitstream 110 and encoded bitstream 111 in units of one picture included in the same access unit, An encoded bit stream 310 is generated by arranging encoded bit streams 111 of units.

  Hereinafter, as described above, H.P. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100C according to the fourth embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  In the image decoding device 100C of the present embodiment, the first control unit 131 and the second control unit 132 operate as follows to synchronize the decoding process of the encoded bitstream 310.

  First, the decoding process for the picture on the L side will be described.

  The first control unit 131 controls the image decoding unit 320 and performs a picture decoding process of the L-side encoded bitstream 110 combined with the encoded bitstream 310 in the following processing steps.

  Note that the processing procedure by the first control unit 131 is the same as the processing shown in FIG. 12, and therefore will be described with reference to FIG.

  First, the first control unit 131 clears the first completion flag stored in the flag storage unit 180 (S101).

  Next, the first control unit 131 acquires from the frame memory management unit 150 the base address of the frame memory 170 that stores the decoding result of the decoding target picture (S102).

  Next, the first control unit 131 acquires the DPB list 141a on the L side from the management information storage unit 140 (S103).

  Next, the first control unit 131 acquires the frame memory mapping list 142a. Also, the first control unit 131 refers to the acquired DPB list 141a and the frame memory mapping list 142a, and acquires the base address of the reference picture for reference within the view (S104).

  Next, the first control unit 131 sets the decoding target picture and the base address of the reference picture obtained in steps S102 to S104, and the DPB list 141a, and activates the decoding unit 321 (S105). Since the streams are combined into one, the setting of the decoding target stream as in the second embodiment is not necessary.

  Next, the decoding unit 321 acquires a reference picture from the base address of the reference picture set in step S105, and generates a decoded picture by decoding the encoded bitstream 110 using the acquired reference picture. Also, the decoding unit 321 stores the generated decoded picture at the base address of the decoding target picture set in step S105. In addition, the decoding unit 321 notifies the first control unit 131 that the decoding process for the decoding target picture has been completed (S106).

  Next, the first control unit 131 obtains the syntax information 113 of the picture that has been decoded in step S106 from the decoding unit 321 (S107).

  Next, when the picture for which decoding processing has been completed in step S105 is used as a reference picture when decoding subsequent pictures, the first control unit 131 uses the syntax information 113 acquired in step S107 to manage information The DPB list 141a in the storage unit 140 is updated (S108).

  Next, when there is a picture out of the DPB list 141a due to the update of the DPB list 141a in step S108, the first control unit 131 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141a. . When the index 201 is notified from the first controller 131, the frame memory manager 150 updates the frame memory mapping list 142a (S109).

  Next, the first control unit 131 stores the syntax information 113 obtained in step S107 in the inter-view reference syntax information list 143 as inter-view reference syntax information (S110).

  Next, the first control unit 131 stores the base address obtained in step S102 in the inter-view reference frame memory mapping list 144 (S111).

  Next, if the input encoded bitstream is R-side encoded bitstream data, the first control unit 131 temporarily stops the L-side decoding process and first completes the flag storage unit 180. A flag is set (S112).

  Next, the decoding process for the picture on the R side will be described.

  After the above steps S101 to S112, the second control unit 132 controls the image decoding unit 320 and performs the decoding process of the R-side encoded bitstream 111 combined with the encoded bitstream 310 in the following processing steps. .

  The processing procedure by the second control unit 132 is the same as the processing shown in FIG. 13, and will be described with reference to FIG.

  First, the second control unit 132 clears the second completion flag stored in the flag storage unit 180 (S201).

  Next, the second control unit 132 uses the inter-view reference syntax information list in the management information storage unit 140 for the syntax information 113 of the L-side decoded picture that can be referred to in the decoding process of the R-side encoded bitstream 111. 143, the inter-view reference picture is specified (S202).

  Next, the second control unit 132 obtains the base address of the L-side decoded picture that can be referenced in the decoding process of the R-side encoded bitstream 111 from the inter-view reference frame memory mapping list 144 in the management information storage unit 140. Obtain (S203).

  Next, the second control unit 132 acquires the base address of the frame memory 170 that stores the decoding result of the decoding target picture (R-side picture) from the frame memory management unit 150 (S204).

  Next, the second control unit 132 acquires the DPB list 141b on the R side from the management information storage unit 140 (S205).

  Next, the second control unit 132 acquires the frame memory mapping list 142b. Further, the second control unit 132 refers to the acquired DPB list 141b and the frame memory mapping list 142b, and acquires the base address of the reference picture for reference within the view (S206).

  Next, the second control unit 132 determines the decoding target picture obtained in steps S203, S204, S205, and S206, the base address of the reference picture for intra-view reference and the reference picture for inter-view reference, and the decoding target picture. The DPB list 141b is set in the decryption unit 321 and the decryption unit 321 is activated (S207). Since the streams are combined into one, the setting of the decoding target stream as in the second embodiment is not necessary.

  Next, the decoding unit 321 acquires a reference picture from the base address for intra-view reference or inter-view reference set in step S207, and decodes the encoded bitstream 111 using the acquired reference picture. A decoded picture is generated. Also, the decoding unit 321 stores the generated decoded picture at the base address of the decoding target picture set in step S207. In addition, the decoding unit 321 notifies the second control unit 132 that the decoding process of the picture has been completed (S208).

  Next, the second control unit 132 acquires the syntax information 113 of the picture that has been decoded in step S208 from the decoding unit 321 (S209).

  Next, the second control unit 132 updates the DPB list 141b when the picture that has been decoded in step S208 is used as a reference picture when decoding the subsequent picture (S210).

  Next, the second control unit 132 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141b when there is a picture that is out of the DPB list due to the update of the DPB list 141b in step S210. When the index 201 is notified from the second control unit 132, the frame memory management unit 150 updates the frame memory mapping list 142b (S211).

  Next, if the input encoded bit stream is data of the L-side encoded bit stream, the second control unit 132 temporarily stops the R-side decoding process and completes the second completion in the flag storage unit 180. A flag is set (S212).

  The relationship of the DPB lists 141a and 141b, the frame memory mapping lists 142a and 142b, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time is as follows. This is the same as in the second mode.

  As described above, the image decoding apparatus 100C according to Embodiment 4 of the present invention can synchronize the decoding process by combining the encoded bit streams 110 and 111 into one encoded bit stream 310. it can.

  Also, the image decoding device 100C according to Embodiment 4 of the present invention performs the L-side encoding via the inter-view reference syntax information list 143 that can be commonly accessed during the decoding process of the R-side encoded bitstream 111. It is possible to know the syntax management information of the reference picture of the normalized bitstream 110. As a result, the image decoding apparatus 100C can decode the encoded bitstream encoded using inter-view prediction.

  Note that the image decoding apparatus 100C according to Embodiment 4 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the stream combining unit 190 generates the encoded bitstream 310 by alternately outputting the encoded bitstreams 110 and 111 in units of pictures (frame units). In this case, the data may be alternately output in units of 2 field pictures. In addition, the stream combining unit 190 is an H.264. The encoded bit streams 110 and 111 may be alternately output in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 131 and the second control unit 132 described in the present embodiment may actually be equipped with two processors or may be two logical threads.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  Further, in the present embodiment, the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b are individually managed by the R side encoded bit stream 111 and the L side encoded bit stream 110. At least one of the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b may be managed by one list, and a unified index may be assigned.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

  In this embodiment, the NAL header is analyzed to determine whether the data is the encoded bit stream 110 data or the 111 data. For example, a bit string different from the start code that can be identified by the decoding unit 321 is used as the stream combining unit. 190 may be inserted into the encoded bitstream 310.

(Embodiment 5)
FIG. 24 is a block diagram showing a configuration of an image decoding device 100D according to Embodiment 5 of the present invention. Elements similar to those in FIG. 23 are denoted by the same reference numerals, and redundant description is omitted.

  Specifically, in the image decoding device 100D illustrated in FIG. 24, the configuration of the management information storage unit 240 is different from the configuration of the management information storage unit 140 illustrated in FIG. More specifically, the management information storage unit 240 stores a frame memory mapping list 142c instead of the frame memory mapping lists 142a and 142b shown in FIG. In other words, the configuration of the management information storage unit 240 is the same as that of the management information storage unit 240 shown in FIG.

  The encoded bit streams 110 and 111 input to the image decoding device 100D of the present embodiment are H.264 and H.264. It is encoded using 264-MVC. H. The description of the characteristics of the 264-MVC encoding and the relationship with the configuration of the image decoding apparatus of the present embodiment is as already described in the first embodiment.

  As described above. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100D according to the fifth embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  In the present embodiment, as in the third embodiment, the frame memory is assumed to have an area that can store a total of nine pictures.

  First, the image decoding device 100D according to Embodiment 5 starts decoding processing from the L-side encoded bitstream 110. Also, the first control unit 131 and the second control unit 132 synchronize the decoding processing of the encoded bitstreams 110 and 111 by operating alternately as in the fourth embodiment.

  Further, the relationship of the DPB lists 141a and 141b, the frame memory mapping list 142c, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time is the same as in FIG. is there.

  As described above, the image decoding apparatus 100D according to Embodiment 5 of the present invention can synchronize the decoding process by combining the encoded bit streams 110 and 111 into one encoded bit stream 310. it can.

  Also, the image decoding device 100D according to Embodiment 5 of the present invention performs the L-side encoding via the inter-view reference syntax information list 143 that can be accessed in common during the decoding process of the R-side encoded bitstream 111. It is possible to know the syntax management information of the reference picture of the normalized bitstream 110. As a result, the image decoding device 100D can decode the encoded bitstream encoded using inter-view prediction.

  Furthermore, the image decoding apparatus 100D according to Embodiment 5 of the present invention manages the frame memory mapping list 142c in common for the L-side encoded bitstream 110 and the R-side encoded bitstream 111, thereby reducing the frame memory capacity. The cost can be reduced.

  Note that the image decoding device 100D according to Embodiment 5 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the stream combining unit 190 generates the encoded bitstream 310 by alternately outputting the encoded bitstreams 110 and 111 in units of pictures (frame units). In this case, the data may be alternately output in units of 2 field pictures. In addition, the stream combining unit 190 is an H.264. The encoded bit streams 110 and 111 may be alternately output in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 131 and the second control unit 132 described in the present embodiment may actually be equipped with two processors or may be two logical threads.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  In this embodiment, the DPB lists 141a and 141b are individually managed by the R-side encoded bitstream 111 and the L-side encoded bitstream 110. However, the index managed and unified by one list is used. It may be given.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

  In this embodiment, the NAL header is analyzed to determine whether the data is the encoded bit stream 110 data or the 111 data. For example, a bit string different from the start code that can be identified by the decoding unit 321 is used as the stream combining unit. 190 may be inserted into the encoded bitstream 310.

(Embodiment 6)
FIG. 25 is a block diagram showing a configuration of an image decoding device 100E according to Embodiment 6 of the present invention. Elements similar to those in FIG. 23 are denoted by the same reference numerals, and redundant description is omitted.

  In the image decoding device 100E illustrated in FIG. 25, the configuration of the control unit 230 is different from the configuration of the control unit 130 illustrated in FIG. Further, the image decoding device 100E is different from the image decoding device 100C illustrated in FIG. 23 in that the flag storage unit 180 is not provided.

  The control unit 230 includes a first control unit 231.

  The first control unit 231 controls the decoding process of the encoded bit stream 310 in which the encoded bit stream 110 and the encoded bit stream 111 are combined.

  The first control unit 231 generates the encoded bitstream 310 based on the DPB list 141a, the inter-view reference syntax information list 143, the frame memory mapping list 142a, and the inter-view reference frame memory mapping list 144. The decoding process of the combined encoded bit stream 110 is controlled. The first control unit 231 also encodes the encoded bitstream based on the DPB list 141b, the inter-view reference syntax information list 143, the frame memory mapping list 142b, and the inter-view reference frame memory mapping list 144. The decoding process of the encoded bitstream 111 combined with 310 is controlled.

  FIG. 26 is a diagram illustrating processing of the first control unit 231.

  As illustrated in FIG. 26, the first control unit 231 stores the syntax information 113 of the reference picture of the L-side encoded bitstream 110 that is the base view in the inter-view reference syntax information list 143. Also, the first control unit 231 acquires the inter-view reference syntax information list 143 when decoding the R-side encoded bitstream 111 that is a non-base view.

  The encoded bit streams 110 and 111 input to the image decoding apparatus 100E according to the present embodiment are H.264 and H.264. It is encoded using 264-MVC. H. The description of the characteristics of the 264-MVC encoding and the relationship with the configuration of the image decoding apparatus of the present embodiment is as already described in the first embodiment.

  As described above. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100E according to the sixth embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  The image decoding apparatus 100E according to the present embodiment performs a decoding process on the encoded bitstream 310 by operating as follows.

  First, the decoding process for the picture on the L side will be described.

  The first control unit 231 controls the decoding unit 321 and performs the picture decoding process of the L-side encoded bitstream 110 combined with the encoded bitstream 310 in the following processing steps.

  FIG. 27 is a flowchart of decoding processing on the L side by the image decoding device 100E.

  First, the first control unit 231 obtains the base address of the frame memory 170 that stores the decoding result of the decoding target picture from the frame memory management unit 150 (S301).

  Next, the first control unit 231 acquires the L-side DPB list 141a from the management information storage unit 140 (S302).

  Next, the first control unit 231 acquires the frame memory mapping list 142a. Further, the first control unit 231 refers to the acquired DPB list 141a and the frame memory mapping list 142a, and acquires the base address of the reference picture for reference within the view (S303).

  Next, the first control unit 231 sets the decoding target picture and the base address of the reference picture obtained in steps S301 to S303, and the DPB list 141a, and activates the decoding unit 321 (S304).

  Next, the decoding unit 321 acquires a reference picture from the base address of the reference picture set in step S304, and generates a decoded picture by decoding the encoded bitstream 110 using the acquired reference picture. Also, the decoding unit 321 stores the generated decoded picture at the base address of the decoding target picture set in step S304. In addition, the decoding unit 321 notifies the first control unit 231 that the decoding process for the decoding target picture has been completed (S305).

  Next, the first control unit 231 acquires the syntax information 113 of the picture that has been decoded in step S305 from the decoding unit 321 (S306).

  Next, when the picture for which decoding processing has been completed in step S304 is used as a reference picture when decoding subsequent pictures, the first control unit 231 uses the syntax information 113 acquired in step S306 to manage the management information. The DPB list 141a in the storage unit 140 is updated (S307).

  Next, when there is a picture out of the DPB list 141a due to the update of the DPB list 141a in step S307, the first control unit 231 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141a. . The frame memory management unit 150 updates the frame memory mapping list 142a when the index 201 is notified from the first control unit 231 (S308).

  Next, the first control unit 231 stores the syntax information 113 obtained in step S306 in the inter-view reference syntax information list 143 as syntax information for inter-view reference (S309).

  Next, the first control unit 231 stores the base address obtained in step S301 in the inter-view reference frame memory mapping list 144 (S310).

  In addition, if the input encoded bit stream is data of the R-side encoded bit stream, the first control unit 231 then starts the R-side decoding process.

  Next, the decoding process for the picture on the R side will be described.

  After the above steps S401 to S410, the first control unit 231 controls the image decoding unit 320 and performs the decoding process of the R-side encoded bitstream 111 combined with the encoded bitstream 310 in the following processing steps. .

  FIG. 28 is a flowchart of the decoding process on the R side by the image decoding device 100E.

  First, the first control unit 231 uses the inter-view reference syntax information list 143 in the management information storage unit 140 for the syntax information 113 of the L-side decoded picture that can be referred to in the decoding process of the R-side encoded bitstream 111. And the inter-view reference picture is specified (S401).

  Next, the first control unit 231 determines the base address of the L-side decoded picture that can be referred to in the decoding process of the R-side encoded bitstream 111 from the inter-view reference frame memory mapping list 144 in the management information storage unit 140. Obtain (S402).

  Next, the first control unit 231 acquires the base address of the frame memory 170 that stores the decoding result of the decoding target picture (R-side picture) from the frame memory management unit 150 (S403).

  Next, the first control unit 231 acquires the R-side DPB list 141b from the management information storage unit 140 (S404).

  Next, the first controller 231 acquires the frame memory mapping list 142b. Also, the first control unit 231 refers to the acquired DPB list 141b and the frame memory mapping list 142b, and acquires the base address of the reference picture for reference within the view (S405).

  Next, the first control unit 231 includes the base address of the decoding target picture, the reference picture for intra-view reference and the reference picture for inter-view reference obtained in steps S402, S403, S404, and S405, and the decoding target picture. The DPB list 141b is set in the decryption unit 321 and the decryption unit 321 is activated (S406).

  Next, the decoding unit 321 acquires a reference picture from the base address for intra-view reference or inter-view reference set in step S406, and decodes the encoded bitstream 111 using the acquired reference picture. A decoded picture is generated. Also, the decoding unit 321 stores the generated decoded picture at the base address of the decoding target picture set in step S406. In addition, the decoding unit 321 notifies the first control unit 231 that the decoding processing of the picture has been completed (S407).

  Next, the first control unit 231 acquires the syntax information 113 of the picture that has been decoded in step S407 from the decoding unit 321 (S408).

  Next, the first control unit 231 updates the DPB list 141b when the picture that has been decoded in step S407 is used as a reference picture when decoding a subsequent picture (S409).

  Next, when there is a picture out of the DPB list due to the update of the DPB list 141b in step S409, the first control unit 231 notifies the frame memory management unit 150 of the index 201 of the corresponding picture in the DPB list 141b. When the index 201 is notified from the first control unit 231, the frame memory management unit 150 updates the frame memory mapping list 142 b (S 410).

  In addition, if the input encoded bitstream is data of the L-side encoded bitstream, the first control unit 231 next starts the L-side decoding process.

  The relationship of the DPB lists 141a and 141b, the frame memory mapping lists 142a and 142b, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time is as follows. This is the same as in the second mode.

  As described above, the image decoding apparatus 100E according to Embodiment 6 of the present invention can synchronize the decoding process by combining the encoded bit streams 110 and 111 into one encoded bit stream 310. it can.

  In addition, the image decoding device 100E according to Embodiment 6 of the present invention controls the encoded bitstreams 110 and 111 with one first control unit 231, and thereby via the inter-view reference syntax information list 143. It is possible to know the syntax management information of the reference picture of the L-side encoded bitstream 110. Accordingly, the image decoding device 100E can decode the encoded bitstream encoded using inter-view prediction.

  Note that the image decoding device 100E according to Embodiment 6 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the stream combining unit 190 generates the encoded bitstream 310 by alternately outputting the encoded bitstreams 110 and 111 in units of pictures (frame units). In this case, the data may be alternately output in units of 2 field pictures. In addition, the stream combining unit 190 is an H.264. The encoded bit streams 110 and 111 may be alternately output in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 231 described in the present embodiment may actually be equipped with one processor or may be a logical single thread.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  Further, in the present embodiment, the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b are individually managed by the R side encoded bit stream 111 and the L side encoded bit stream 110. At least one of the DPB lists 141a and 141b and the frame memory mapping lists 142a and 142b may be managed by one list, and a unified index may be assigned.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

  In this embodiment, the NAL header is analyzed to determine whether the data is the encoded bit stream 110 data or the 111 data. For example, a bit string different from the start code that can be identified by the decoding unit 321 is used as the stream combining unit. 190 may be inserted into the encoded bitstream 310.

(Embodiment 7)
FIG. 29 is a block diagram showing a configuration of an image decoding device 100F according to Embodiment 7 of the present invention. Elements similar to those in FIG. 25 are denoted by the same reference numerals, and redundant description is omitted.

  Specifically, in the image decoding device 100F illustrated in FIG. 29, the configuration of the management information storage unit 240 is different from the configuration of the management information storage unit 140 illustrated in FIG. More specifically, the management information storage unit 240 stores a frame memory mapping list 142c instead of the frame memory mapping lists 142a and 142b shown in FIG. In other words, the configuration of the management information storage unit 240 is the same as that of the management information storage unit 240 shown in FIG.

  The encoded bit streams 110 and 111 input to the image decoding apparatus 100F of the present embodiment are H.264 and H.264. It is encoded using 264-MVC. H. The description of the characteristics of the 264-MVC encoding and the relationship with the configuration of the image decoding apparatus of the present embodiment is as already described in the first embodiment.

  As described above. H.264 standard and H.264 standard. The operation of the image decoding apparatus 100F according to the seventh embodiment of the present invention configured and associated with the 264-MVC standard will be described.

  In the present embodiment, as in the third embodiment, the frame memory 170 is assumed to have an area capable of storing a total of nine pictures.

  First, the image decoding device 100F according to the present embodiment starts decoding processing from the L-side encoded bitstream 110. The first control unit 231 performs the decoding process of the encoded bitstream 310 by operating in the same manner as in the sixth embodiment.

  The relationship of the DPB lists 141a and 141b, the frame memory mapping list 142c, the inter-view reference syntax information list 143, and the inter-view reference frame memory mapping list 144 for each picture time is the same as in FIG. is there.

  As described above, the image decoding device 100F according to Embodiment 7 of the present invention can synchronize the decoding process by combining the encoded bit streams 110 and 111 into one encoded bit stream 310. it can.

  In addition, the image decoding device 100F according to Embodiment 7 of the present invention controls the encoded bitstreams 110 and 111 with one first control unit 231, and thereby via the inter-view reference syntax information list 143. It is possible to know the syntax management information of the reference picture of the L-side encoded bit stream. Accordingly, the image decoding device 100F can decode the encoded bitstream encoded using inter-view prediction.

  Furthermore, the image decoding apparatus 100F according to Embodiment 7 of the present invention reduces the capacity of the frame memory 170 by managing the frame memory mapping list 142c in common for the L-side encoded bitstream and the R-side encoded bitstream. can do. Therefore, the image decoding device 100F can reduce the cost.

  Note that the image decoding device 100F according to Embodiment 7 of the present invention is an H.264 standard. The present invention is not limited to the H.264 standard and can be applied to decoding of an encoded bit stream based on other standards.

  In the present embodiment, the stream combining unit 190 generates the encoded bitstream 310 by alternately outputting the encoded bitstreams 110 and 111 in units of pictures (frame units). In this case, the data may be alternately output in units of 2 field pictures. In addition, the stream combining unit 190 is an H.264. The encoded bit streams 110 and 111 may be alternately output in units of NAL units such as a parameter set unit (header decoding processing unit) or a slice unit defined in the H.264 standard.

  Further, the first control unit 231 described in the present embodiment may actually be equipped with one processor or may be a logical single thread.

  In this embodiment, the L-side encoded bit stream 110 is set as a base view and the R-side encoded bit stream 111 is set as a non-base view. However, the R-side encoded bit stream 111 is set as a base view and L-side encoded. The bit stream 110 may be a non-base view.

  In addition, although the two-view coded bitstream is described in the present embodiment, it goes without saying that the effect of the present invention can be obtained with a coded bitstream having three or more viewpoints.

  In this embodiment, the DPB lists 141a and 141b are individually managed by the R-side encoded bitstream 111 and the L-side encoded bitstream 110. However, the index managed and unified by one list is used. It may be given.

  Also, the syntax information stored in the DPB lists 141a and 141b in the present embodiment is an example, and is not limited by the present embodiment.

  In this embodiment, the NAL header is analyzed to determine whether the data is the encoded bit stream 110 data or the 111 data. For example, a bit string different from the start code that can be identified by the decoding unit 321 is used as the stream combining unit. 190 may be inserted into the encoded bitstream 310.

  Each processing unit included in the image decoding devices according to the first to seventh embodiments is typically realized as an LSI that is an integrated circuit. These may be individually made into one chip, or may be made into one chip so as to include a part or all of them.

  Further, the circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.

  Moreover, you may implement | achieve part or all of the function of the image decoding apparatus based on Embodiment 1-7 of this invention, when processors, such as CPU, run a program.

  Further, the present invention may be the above program or a recording medium on which the above program is recorded. Needless to say, the program can be distributed via a transmission medium such as the Internet.

  Moreover, you may combine at least one part among the functions of the image decoding apparatus which concerns on the said Embodiments 1-7, and its modification.

  Moreover, all the numbers used above are illustrated to specifically describe the present invention, and the present invention is not limited to the illustrated numbers. In addition, the connection relationship between the components is exemplified for specifically explaining the present invention, and the connection relationship for realizing the function of the present invention is not limited to this.

  Further, the configuration of the image decoding device is for illustration in order to specifically describe the present invention, and the image decoding device according to the present invention does not necessarily have all of the above configurations. In other words, the image decoding apparatus according to the present invention only needs to have a minimum configuration capable of realizing the effects of the present invention.

  Similarly, the above-described image decoding method by the image decoding apparatus is for illustrative purposes only, and the image decoding method according to the present invention is not necessarily required to include all of the above steps. Absent. In other words, the image decoding method according to the present invention may include only the minimum steps that can realize the effects of the present invention. In addition, the order in which the above steps are executed is for illustration in order to specifically describe the present invention, and may be in an order other than the above. Also, some of the above steps may be executed simultaneously (in parallel) with other steps.

  Further, various modifications in which the present embodiment is modified within the scope conceivable by those skilled in the art are also included in the present invention without departing from the gist of the present invention.

  The present invention can be applied to an image decoding apparatus. This is useful for a stereoscopic image system that uses a plurality of encoded bitstreams compressed by an encoding method using inter-view prediction such as 264-MVC. For example, the present invention is useful for an optical disc reproducing device, an optical disc recording device, a digital television receiving device, a movie device, and a mobile phone.

100, 100A, 100B, 100C, 100D, 100E, 100F, 500, 600 Image decoding device 110, 111, 310 Encoded bit stream 112 Output image signal 113 Syntax information 114 RPL
120, 220, 320 Image decoding unit 121 First decoding unit 122 Second decoding unit 123 Header analysis unit 124 Entropy decoding unit 125 Inverse quantization unit 126 Inverse orthogonal transform unit 127 Intra prediction unit 128 Motion compensation unit 129 Deblocking filter processing unit 130, 230 Control unit 131, 231 First control unit 132 Second control unit 140, 240 Management information storage unit 141a, 141b DPB list 142a, 142b, 142c Frame memory mapping list 143 Inter-view reference syntax information list 144 Between views Reference frame memory mapping list 150 Frame memory management unit 160 Image display unit 170 Frame memory 180 Flag storage unit 190 Stream combination unit 201 Index 202 Syntax information 203 Frame Memory address 204, 204a, 204b DPB list index 205 Inter-view reference syntax information index 206 Image display status 207 State information 221 Selection unit 222, 321 Decoding unit 502 Input unit 503 Header analysis unit 504 Decoding control unit 505, 506, 507 Arithmetic decoding unit 508, 509, 510 Storage unit 511, 512, 513 Image decoding unit 601 VBV buffer 602 Variable length decoder 603 Inverse quantizer 604 IDCT unit 605 Adder 606 Image memory 607 Motion compensation predictor 611 Demultiplexer 612 , 613, 614 Signal switcher 621 Decoding controller 622 Display controller 623 Switch controller

Claims (15)

Image decoding for decoding a first encoded signal obtained by encoding a first image signal of a first viewpoint and a second encoded signal obtained by encoding a second image signal of a second viewpoint different from the first viewpoint A device,
An image decoding unit that generates a first decoded image signal and a second decoded image signal by decoding the first encoded signal and the second encoded signal with reference to a reference image;
A frame memory for storing the first decoded image signal and the second decoded image signal as the reference image;
The image decoding unit refers to the first decoded image signal as the reference image, decodes the first encoded signal, and refers to the first decoded image signal and the second decoded image signal as the reference image. And decoding the second encoded signal,
The image decoding device further includes:
First management information for specifying an area of the frame memory in which the first decoded image signal is stored, and second management information for specifying an area of the frame memory in which the second decoded image signal is stored A management information storage unit for storing
Referring to the first management information, the image decoding unit is notified of the area of the frame memory in which the reference image to be referred to when decoding the first encoded signal is stored, and the first management information And a control unit that refers to the second management information and notifies the image decoding unit of an area of the frame memory in which the reference image to be referred to when decoding the second encoded signal is stored. Image decoding device. The first management information is
For each of the first decoded image signals, first syntax information included in the first encoded signal, for specifying the first decoded image signal, and first information associated with the first syntax information A first syntax information list including an identification number;
A first mapping list indicating a correspondence relationship between the first identification number and the area of the frame memory for each first decoded image signal;
The control unit obtains the first identification number corresponding to the first decoded image signal by referring to the first syntax information list, and refers to the first mapping list to obtain the first identification number. Obtaining an area of the frame memory corresponding to the identification number, and notifying the obtained image area of the frame memory as an area of the frame memory in which the reference image is stored to the image decoding unit;
The second management information is
Second syntax information included in the second encoded signal for identifying the second decoded image signal and second information associated with the second syntax information for each second decoded image signal A second syntax information list including an identification number;
A second mapping list indicating a correspondence between the second identification number and the area of the frame memory for each second decoded image signal;
The control unit obtains the second identification number corresponding to the second decoded image signal by referring to the second syntax information list, and refers to the second mapping list to obtain the second identification number. The image according to claim 1, wherein an area of the frame memory corresponding to an identification number is acquired, and the acquired area of the frame memory is notified to the image decoding unit as an area of the frame memory in which the reference image is stored. Decoding device. The first syntax information and the second syntax information are H.264. The image decoding apparatus according to claim 2, wherein the image decoding apparatus is syntax information necessary for updating a decoded picture buffer (DPB) defined in the H.264 standard. The first syntax information list is:
Intra-view reference syntax information list including the first syntax information and the first identification number for each first decoded image signal referred to when the first decoding signal is decoded by the image decoding unit. When,
Inter-view reference syntax information list including the first syntax information and the first identification number for each first decoded image signal referred to when the image decoding unit decodes the second encoded signal. Including
The controller is
When the image decoding unit decodes the first encoded signal, the first identification number corresponding to the first decoded image signal is acquired by referring to the in-view reference syntax information list. The frame memory area corresponding to the first identification number is acquired by referring to the first mapping list, and the acquired frame memory area is the frame memory area in which the reference image is stored. To the image decoding unit,
When the image decoding unit decodes the second encoded signal, the first identification number corresponding to the first decoded image signal is acquired by referring to the inter-view reference syntax information list. The frame memory area corresponding to the first identification number is acquired by referring to the first mapping list, and the acquired frame memory area is the frame memory area in which the reference image is stored. The image decoding device according to claim 2, which is notified to the image decoding unit. The first mapping list is:
Intra-view reference mapping list indicating the correspondence between the first identification number and the area of the frame memory for each first decoded image signal referred to when the first encoded signal is decoded by the image decoding unit. When,
Inter-view reference mapping list indicating the correspondence between the first identification number and the area of the frame memory for each first decoded image signal referred to when the second encoded signal is decoded by the image decoding unit. Including
The controller is
When the image decoding unit decodes the first encoded signal, the first identification number corresponding to the first decoded image signal is acquired by referring to the in-view reference syntax information list. The frame memory area corresponding to the first identification number is acquired by referring to the in-view reference mapping list, and the frame memory in which the reference image is stored in the acquired frame memory area As an area of the image decoding unit,
When the image decoding unit decodes the second encoded signal, the first identification number corresponding to the first decoded image signal is acquired by referring to the inter-view reference syntax information list. The frame memory area corresponding to the first identification number is obtained by referring to the inter-view reference mapping list, and the frame memory area in which the reference image is stored in the obtained frame memory area The image decoding device according to claim 4, wherein the image decoding unit is notified of the region as the region. The image decoding device further includes:
An image display unit for outputting the first decoded image signal and the second decoded image signal stored in the frame memory to the outside;
The first mapping list further includes, for each first decoded image signal, first image display information indicating whether the first decoded image signal is output to the outside by the image display unit, and the first decoding list. First reference information indicating whether an image signal is used as the reference image,
The control unit indicates that the first decoded image signal is output to the outside by the first image display information, and the first decoded image signal is used as the reference image by the first reference information. 3. The image decoding according to claim 2, wherein if there is no data, the first decoded image signal newly decoded by the image decoding unit is stored in an area of the frame memory in which the first decoded image signal is stored. apparatus. The image decoding unit
A first decoding unit that generates a first decoded image signal by decoding the first encoded signal;
A second decoding unit that generates a second decoded image signal by decoding the second encoded signal,
The controller is
After the first decoding unit and the second decoding unit complete the decoding process of a predetermined decoding process unit of the first encoded signal and the second encoded signal, the first decoding unit and the second decoding unit The image decoding apparatus according to claim 1, wherein the second decoding unit starts decoding processing for the next decoding processing unit. The image decoding device further includes:
A flag storage unit for storing the first completion flag and the second completion flag;
The controller is
The first decoding unit controls the first decoding unit, and stores the first completion flag in the flag storage unit when the decoding processing unit of the first encoded signal by the first decoding unit is completed. 1 control unit;
The second decoding unit controls the second decoding unit, and stores the second completion flag in the flag storage unit when the decoding processing unit of the second encoded signal by the second decoding unit is completed. 2 control units,
The first control unit, when the decoding process unit of the first encoded signal by the first decoding unit is completed, and the second completion flag is stored in the flag storage unit, Causing the first decoding unit to start decoding the first encoded signal of the next decoding processing unit;
The second control unit, when the decoding process unit of the second encoded signal by the second decoding unit is completed and the first completion flag is stored in the flag storage unit, The image decoding device according to claim 7, wherein the second decoding unit starts decoding processing of the second encoded signal of the next decoding processing unit. The image decoding unit
A selector for selecting one of the first encoded signal and the second encoded signal;
A decoding unit that decodes the first encoded signal or the second encoded signal selected by the selection unit,
The control unit causes the selection unit to select one of the first encoded signal and the second encoded signal, and the decoding process of a predetermined decoding process unit of the one signal is completed. The image decoding apparatus according to claim 1, further comprising: causing the selection unit to select the other signal of the first encoded signal and the second encoded signal. The image decoding device further includes:
A flag storage unit for storing the first completion flag and the second completion flag;
The controller is
Controls the front Kifuku No. section, when the decoding process of the decoding processing unit of the first coded signal according to prior Kifuku signal block is completed, the storing the first completion flag in the flag storage unit 1 control unit;
Controls the front Kifuku No. section, when the decoding process of the decoding processing unit of the second coded signal by front Kifuku signal block is completed, the storing the second completion flag in the flag storage unit 2 control units,
The first control unit, when the second completion flag is stored in the flag storage unit, causes the decoding unit to start the decoding process of the first encoded signal of the next decoding processing unit,
The said 2nd control part makes the said decoding part start the decoding process of the said 2nd encoded signal of the next decoding process unit, when the said 1st completion flag is stored in the said flag storage part. Image decoding apparatus. The image decoding device further includes:
A combining unit that generates one third encoded signal by alternately arranging the first encoded signal and the second encoded signal for each predetermined encoding unit;
The image decoding device according to claim 1, wherein the image decoding unit alternately decodes the first encoded signal and the second encoded signal included in the third encoded signal for each encoding unit. The image decoding apparatus according to claim 11, wherein the encoding unit is one of a frame unit, a field unit, and a slice unit. The combining unit arranges the first encoded signal after arranging the first encoded signal of the encoding unit among the first encoded signal and the second encoded signal corresponding to the encoding unit. The image decoding apparatus according to claim 11, wherein the third encoded signal is generated by arranging the second encoded signal of the encoding unit corresponding to a signal. The image decoding device according to any one of claims 7 to 10 , wherein the decoding processing unit is one of a frame unit, a field unit, a header decoding processing unit, and a slice unit. Image decoding for decoding a first encoded signal obtained by encoding a first image signal of a first viewpoint and a second encoded signal obtained by encoding a second image signal of a second viewpoint different from the first viewpoint A method,
An image decoding step of generating a first decoded image signal and a second decoded image signal by decoding the first encoded signal and the second encoded signal with reference to a reference image;
Storing the first decoded image signal and the second decoded image signal in the frame memory as the reference image,
In the image decoding step, the first encoded signal is decoded with reference to the first decoded image signal as the reference image, and the first decoded image signal and the second decoded image signal are referred to as the reference image. And decoding the second encoded signal,
The image decoding method further includes:
First management information for specifying an area of the frame memory in which the first decoded image signal is stored, and second management information for specifying an area of the frame memory in which the second decoded image signal is stored A management information storage step for storing
With reference to the first management information, before Symbol identify areas of the frame memory in which the reference picture to be referred to is stored in the decoding of the first coded signal, the first management information and the second by referring to the management information, before Symbol image decoding method comprising a control step of the reference image to identify an area of the frame memory storing reference during decoding of the second coded signal.

Images