JP2008072336A - Decoding device and decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a decoding device and decoding method which solves the problem, wherein advanced programming is required for listing displaying a plurality of animation thumbnails on a table. <P>SOLUTION: A changeover section 14 reads the encoding data of an encoding stream by a changeover, to one among a plurality of encoding streams, when frames for a dynamic image are given a plurality of the encoded encoding streams. A video decoder 20 decodes the encoding data of the encoding stream read by the changeover section 14 and generates the frames, and writes the frames to a frame buffer 26. The frame buffer 26 contains a shared memory region 28, for temporarily holding a fixed number of the frames decoded from each of a plurality of the encoding streams for a display as a single-memory region shared among a plurality of the encoding streams. The stream buffer 26 is configured in a storable manner, even in either space region, without being subjected to the restrictions of a storage place in the shared memory region 28, even if the frames is one decoded from either encoding stream. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a decoding apparatus and a decoding method for decoding a stream in which a moving image is encoded.

  Media players and personal computers capable of reproducing large-capacity optical disc media such as DVDs (Digital Versatile Discs) have become widespread, and high-quality video content is stored and provided on DVDs or the like. In addition, DVD recorders that can record content provided by television broadcasting or the like on a DVD are also widespread. In order to store video and audio on a recording medium such as a DVD, a compression coding technique is indispensable, MPEG (Moving Picture Experts Group) and H.264. There is a standard for moving picture compression coding technology such as H.264 / AVC (Advanced Video Coding). The compression-coded video data and audio data are multiplexed into one stream together with header information necessary for synchronous reproduction such as reproduction time information, and stored in a recording medium.

  When playing back video content on a DVD player, personal computer, portable terminal, or the like, an interface is provided in which a plurality of video content stored in a hard disk or a recording medium is first displayed as a list of moving image thumbnails on the screen at the same time so that the user can browse. Sometimes. When the user selects any one of the moving image thumbnails in the moving image thumbnail list display, the moving image is reproduced in full size.

Patent Document 1 discloses an image display device that decompresses and simultaneously displays a plurality of compressed image data.
Japanese Patent Laid-Open No. 7-271345

  In order to display a list of a plurality of video thumbnails, it is necessary to decode a plurality of video streams to be displayed simultaneously using a plurality of decoders in parallel. In the case of mounting with the above, a large amount of system resources such as memory and CPU are consumed. In addition, it is necessary to facilitate the configuration of scalable decoding processing that can adjust the number of decoders according to the number of video streams to be displayed at the same time, but it is difficult to implement a decoding circuit having such scalability in hardware. is there. In order to provide scalability in software, it is necessary to implement parallel processing, which requires sophisticated and complicated programming. Further, when system resources are exhausted in the process of decoding a plurality of moving picture streams in parallel, decoding of a part of the moving picture streams to be displayed at the same time may not be in time, resulting in frame dropping.

  The present invention has been made in view of these problems, and an object of the present invention is to provide a decoding apparatus and a decoding method capable of efficiently and simultaneously reproducing a plurality of streams in which moving images are encoded.

In order to solve the above problems, a decoding device according to an aspect of the present invention switches to one of a plurality of encoded streams when a plurality of encoded streams in which a frame of a moving image is encoded are given. A switching unit for reading out encoded data of the encoded stream;
A decoder that decodes encoded data of the encoded stream selected by the switching unit to generate a frame; and a frame buffer that temporarily buffers a frame decoded from each encoded stream. The frame buffer is a single memory area shared between the plurality of encoded streams, and temporarily stores a predetermined number of frames decoded from each of the plurality of encoded streams. A frame including a shared memory area and decoded from any encoded stream can be stored in any free area without being restricted by the storage position in the shared memory area.

  Another aspect of the present invention is a decoding method. In this method, when a plurality of encoded streams in which a frame of a moving image is encoded are given, a step of switching to any one of the plurality of encoded streams and reading out encoded data of the encoded stream, A step of generating a frame by decoding encoded data of the encoded stream that is output, and a single memory area shared between the plurality of encoded streams, each of the plurality of encoded streams In the shared memory area for temporarily holding a predetermined number of frames decoded from the frame, the shared memory can be obtained from any of the encoded streams decoded from each encoded stream. Storing in any empty position without restricting the storage position in the area.

  Yet another embodiment of the present invention is a program. This program has a switching function of switching to any one of a plurality of encoded streams and reading the encoded data of the encoded stream when a plurality of encoded streams in which a frame of a moving image is encoded is given, A decoding function for decoding the encoded data of the encoded stream selected by the switching function to generate a frame, and a single memory area shared between the plurality of encoded streams, A frame decoded from each encoded stream is decoded from any encoded stream in a shared memory area for temporarily holding a predetermined number of frames decoded from each encoded stream. In addition, the storage location in the shared memory area is not limited, and a buffering function that stores data in any free location is used. To realize to.

  This program may be provided as a library of video and audio decoders, or as part of firmware that is built into the equipment to perform basic control of hardware resources such as video and audio decoders. May be. This firmware is stored, for example, in a semiconductor memory such as a ROM or a flash memory in the device. In order to provide the firmware or to update a part of the firmware, a computer-readable recording medium storing the program may be provided, and the program may be transmitted through a communication line. Good.

  It should be noted that any combination of the above-described constituent elements and the expression of the present invention converted between a method, an apparatus, a system, a computer program, a data structure, a recording medium, etc. are also effective as an aspect of the present invention.

  According to the present invention, a plurality of streams in which moving images are encoded can be simultaneously and efficiently reproduced.

  FIG. 1 is a configuration diagram of a decoding device 100 according to an embodiment. This figure depicts a block diagram focusing on functions, and these functional blocks can be realized in various forms by hardware only, software only, or a combination thereof.

  The decoding apparatus 100 decodes and reproduces a stream in which audio data and video data are encoded and multiplexed (hereinafter referred to as “encoded stream” or simply “stream”). In the encoded stream, the video data is MPEG (Moving Picture Experts Group) -2, MPEG-4, H.264, or the like. The audio data is encoded in accordance with a standard such as H.264 / AVC, and the audio data is encoded in accordance with a standard such as MPEG audio. The decoding apparatus 100 separates audio data and video data from the encoded stream, and then decodes each to reproduce audio and video synchronously.

  The decoding apparatus 100 decodes a frame from each of a plurality of encoded streams and simultaneously displays a plurality of moving image thumbnails on the screen of the display 110. The user browses a plurality of moving image thumbnails on the screen of the display 110 and selects any one of the moving image thumbnails, thereby reproducing the selected moving image in full size. At this time, the audio is also reproduced in synchronization.

  A plurality of encoded streams are recorded in the storage 10. The storage 10 is, for example, a large capacity recording device such as a hard disk or a recording medium such as a DVD. In the figure, the storage 10 is inside the decoding apparatus 100. However, the storage 10 may be in an external server, and the encoded stream may be read from the storage 10 of the external server via the network.

  The plurality of encoded streams read from the storage 10 are temporarily stored in the stream buffer 12 for decoding processing. A plurality of encoded stream data are sequentially arranged in the stream buffer 12, and by specifying a byte position, data of a certain encoded stream can be read from the byte position.

  Video data and audio data are multiplexed in the encoded stream, and generally video data is inter-frame prediction encoded in an encoding unit called GOP (Group of Picture). The encoded stream header stores entry point map information indicating the stream attribute and GOP delimiter, and the byte position at which the GOP delimiter can be known from the entry point map information. The entry point map information may be provided as a separate file from the encoded stream, in addition to being provided as header information of the encoded stream.

  H. In H.264 / AVC, there is a frame called an IDR (Instantaneous Decoding Refresh) frame at the GOP delimiter. An IDR frame is a frame in which subsequent frames can be decoded even if there is no information about the previous frame. When the IDR frame is read, the buffer that temporarily holds the frame to be referred to in the inter-frame prediction encoding is reset. In other words, the IDR frame is a frame that prohibits a subsequent frame from referring to a frame before itself. A GOP delimited by an IDR frame is called a closed GOP, and interframe predictive coding is performed with reference to only frames constituting the GOP.

  A GOP that does not use an image included in the previous GOP as a reference image is called a closed GOP, whereas a GOP that uses an image included in the previous GOP as a reference image is called an open GOP. In a video stream that has been subjected to inter-frame predictive encoding, decoding can be performed within the closed GOP without referring to frames of other GOPs, and the closed GOP provides a set of encoded frames of the smallest unit that can be decoded independently. .

  The switching unit 14 reads the data of any one encoded stream in units of GOP while switching to any one of the plurality of encoded streams stored in the stream buffer 12, and provides the data to the demultiplexing unit 18. The switching unit 14 reads out encoded data of a frame constituting one GOP from a certain encoded stream, and when reading of one GOP is completed, the switching unit 14 switches to the next encoded stream and similarly switches one encoded stream from the encoded stream. Read the encoded data of the frames constituting the GOP.

  The switching unit 14 refers to the entry point map information described above in order to cut out the encoded data in GOP units. The GOP delimiter is detected by detecting the byte position of a frame indicating the head of an image sequence that is a minimum decoding unit such as an IDR frame in H.264 / AVC. As another method, the GOP delimiter may be detected by analyzing the header of the video stream multiplexed in the encoded stream in order from the top without depending on the meta information such as the map information of the entry point.

  When the switching unit 14 switches a plurality of encoded streams in GOP units, the encoded stream selected by the switching unit 14 is decoded in GOP units in the demultiplexing unit 18, the video decoder 20, and the audio decoder 22 hereinafter. The In the case of a closed GOP, inter-frame predictive coding is performed without referring to a frame of another GOP. Therefore, the closed GOP can be completely closed within the GOP to perform decoding, and the demultiplexing unit 18 and the video decoder 20 Even if the audio decoder 22 is one instance, it is possible to decode a plurality of encoded streams in parallel while switching in units of GOPs.

  The switching unit 14 sequentially switches a plurality of encoded streams by a round robin method, and reads data of any one encoded stream in units of GOPs. Alternatively, the priority control unit 16 gives some priority among the plurality of encoded streams, and the switching unit 14 gives priority to the data of the encoded stream with high priority according to the priority specified by the priority control unit 16. May be read out. The configuration of the priority control unit 16 is optional, and the configuration of the priority control unit 16 may be omitted. Since priority control by the priority control unit 16 may be related to subsequent processing, a specific example will be described later.

  The demultiplexing unit 18 separates audio encoded data and video encoded data from the encoded stream data supplied from the switching unit 14, and the video encoded data is sent to the video decoder 20 for audio encoding. The data is given to the audio decoder 22.

  The audio decoder 22 decodes the encoded audio. The decoded audio is reproduced in synchronization with the video frame decoded by the video decoder 20. Known techniques can be used for audio decoding processing and video / audio synchronous reproduction processing.

  The video decoder 20 decodes the encoded video data read in GOP units from the encoded stream selected by the switching unit 14. The video decoder 20 includes components necessary for decoding processing, such as a variable length decoding unit, an inverse quantization unit, an inverse orthogonal transform unit, and a motion compensation unit, and decodes each encoded frame in the GOP to restore the frame. When the frame in the GOP is an I (Intra) frame that has been subjected to intraframe coding, the intraframe coding is decoded, and an interframe predictive coded P (Predictive) frame or B (Bidirectionally predictive) frame It is stored as a reference image when generating a predicted image. When the frame in the GOP is a P frame or a B frame, the image data is a difference image. Therefore, the original frame is restored by adding the difference image and a predicted image obtained by motion compensation. The video decoder 20 writes each frame in the GOP thus decoded from the encoded stream in the shared memory area 28 of the frame buffer 26.

  FIG. 2 is a diagram for explaining the data structure of the shared memory area 28 of the frame buffer 26. The frame buffer 26 includes a single shared memory area 28 that is shared among a plurality of encoded streams. The shared memory area 28 is temporarily held to display frames decoded from each encoded stream in GOP units, and is decoded into the number of frames constituting the GOP (referred to as “GOP frame number”) M. It has a capacity to hold MN frames multiplied by the number N of target encoded streams. In general, the number of GOP frames is variable and differs for each encoded stream. Therefore, the number M of GOP frames is estimated by the average number of frames constituting the GOP or the maximum number of frames that can be taken, and the capacity of the shared memory area 28 is designed.

  The shared memory area 28 is divided into blocks (represented by symbols a to z in the figure) for holding the decoded frame, and by specifying the byte position of each block, Thus, the decoded frame data can be read and written. Each block in the shared memory area 28 can store a frame decoded from any encoded stream, and the storage position is not limited by the encoded stream. That is, the shared memory area 28 is a single memory area shared between a plurality of encoded streams, and the storage area is not divided for each encoded stream.

  Of the N streams, for the i-th stream #i, the decoded frame displayed at the j-th is denoted as frame # i-j. A frame decoded from each stream can be stored in an arbitrary empty block in the shared memory area 28. In the figure, frames # 1-1, # 1-2, # 1-3, and # 1-M of the first stream # 1 are stored in blocks a, e, i, and t of the shared memory area 28, respectively. Has been.

  Similarly, frames # 2-1, # 2-2, and # 2-M of the second stream # 2 are respectively stored in the blocks b, c, and s of the shared memory area 28, and the Nth stream #N Frame # N-1 and frame # N-M are stored in blocks h and z of the shared memory area 28, respectively.

  The block in which the frame is stored in the shared memory area 28 is invalidated and becomes an empty block when the reproduction of the frame is completed. Free blocks are used to hold frames from other streams.

  The video decoder 20 writes a frame decoded from each stream into an empty block in the shared memory area 28 of the frame buffer 26. At this time, the video decoder 20 records the information on the storage position of the frame in the shared memory area 28, the information on the playback order of the frame, and the attribute of the frame in the buffer management table 24 for each stream.

  FIG. 3 is a diagram for explaining the data structure of the buffer management table 24. The buffer management table 24 stores free block information of the shared memory area 28 and management information of streams # 1 to #N to be decoded. The empty block information is a list of byte positions of empty blocks (blocks j, k, m, and p in the example shown in the figure) that have been invalidated after the reproduction of the frame is completed in the shared memory area 28. The empty block information may be managed by holding binary data indicating whether or not the invalid flag array for the number of blocks is empty. The video decoder 20 refers to the empty block information in the buffer management table 24 and writes a frame decoded from each stream to the empty block in GOP units.

  The management information of each stream # 1 to #N includes the storage destination block and the frame attribute of the decoded frame stored in the shared memory area 28 in a state of waiting for reproduction. For example, in correspondence with the example of the shared memory area 28 in FIG. 2, the management information of the stream # 1 is stored in the storage destination blocks a, e, i in the frame numbers 1, 2, 3,. ,..., T are associated with each other, and an attribute indicating whether each frame is an I picture, a P picture, or a B picture is also associated. The frame number indicates the reproduction order, and each stream can be reproduced as a moving image by reading out and displaying the frame from the storage destination block associated with the frame number in the order of the frame number for each stream.

  The display control unit 30 refers to the buffer management table 24 and obtains information regarding the playback order and storage position of frames decoded from each encoded stream. Then, the display control unit 30 reads out one frame of each stream from the shared memory area 28 of the frame buffer 26 in synchronization with a vertical synchronizing signal (VSYNC) of the display 110, and displays each frame on the screen of the display 110. Control to display the video thumbnails of the stream at the same time. The display control unit 30 reads the next frame of each stream from the frame buffer 26 at the vertical synchronization frequency indicated by the vertical synchronization signal, and refreshes the moving image thumbnail of each stream displayed on the screen. The display control unit 30 may refresh the moving image thumbnail every two vertical synchronization signals (2VSYNC). For example, in the case of the progressive method, the screen is refreshed once for each vertical synchronization signal given at a period of (1/60) seconds. In the case of the interlace method, for example, every two vertical synchronization signals, that is, (1/30). ) The field consisting of even-numbered scanning lines or the field consisting of odd-numbered scanning lines is alternately refreshed once every second.

  Here, when there is a difference in the frame rate of each stream, the display control unit 30 reads out one frame of each stream from the shared memory area 28 of the frame buffer 26 at a cycle according to the frame rate of each stream, By writing to the VRAM or the like of the display 110, a moving image can be displayed on the display 110 at the frame rate of each stream. Thereby, for example, a case where a stream of 24 frames / second and a stream of 30 frames / second are mixed can be handled. Even in this case, the timing at which pixel data is read from a frame memory such as a VRAM and output to the display 110, that is, the refresh rate, is controlled by the above-described vertical synchronization signal.

  The display control unit 30 deletes the frame number of the buffer management table 24 from the management information of each stream for the frame that has been displayed for each stream, and changes the storage block associated with the frame number to an empty block. Then, the empty block is added to the empty block information of the buffer management table 24.

  By the operation of the display control unit 30, the frames to be displayed next to each stream # 1 to #N are read one by one from the shared memory area 28 of the frame buffer 26 at the timing of the vertical synchronization signal, and the frames of each stream are The stored block becomes a free block. That is, the same number of frames as the number of streams (“simultaneous display stream number”) N simultaneously displayed as moving image thumbnails are simultaneously read from the shared memory area 28, and N empty blocks are generated.

  If the number N of simultaneous display streams is equal to or greater than the number M of GOP frames, the display control unit 30 only stores frames in one GOP in the shared memory area 28 every time the moving image is refreshed in synchronization with the vertical synchronization signal. Free blocks are generated, and the video decoder 20 can write a set of frames decoded from a stream in units of GOPs to these free blocks. If the number of simultaneous display streams N is half of the number of GOP frames M, the video decoder 20 can write a frame set of 1 GOP in an empty block when the display control unit 30 refreshes twice.

  As described above, since the display control unit 30 simultaneously displays the moving images decoded from the plurality of streams on the screen, one frame of each stream is consumed from the frame buffer 26, and a plurality of empty blocks are generated at the same time. . Focusing on this point, in the present embodiment, a shared memory area 28 is provided in the frame buffer 26 so that it can be shared among streams displayed at the same time, and any frame of any stream is not restricted by the storage position. , So that you can write. As a result, instead of performing parallel decoding on the simultaneously displayed streams, even if the single video decoder 20 performs GOP unit decoding only for one stream selected by the switching unit 14, the simultaneous display is performed. As the video thumbnails of the stream to be streamed are refreshed all at once, the set of frames in 1 GOP decoded from the selected stream can be written to the empty block of the shared memory area 28 at a stroke. It is possible to continue the decoding process in units of GOP by switching to this stream.

  Looking at the shared memory area 28 as a center, the display control unit 30 performs a process of reading the next frame of each stream from the shared memory area 28, and the video decoder 20 performs GOP unit for the stream selected by the switching unit 14. A process of writing the decoded frame into an empty block in the shared memory area 28 is performed. The reading process from the shared memory area 28 by the display control unit 30 and the writing process to the shared memory area 28 by the video decoder 20 are performed asynchronously.

  The display control unit 30 reads out the frames of each stream at a cycle in which the vertical synchronization signal is given. On the other hand, when the video decoder 20 has no 1 GOP free block in the shared memory area 28, until the 1 GOP free block is generated. Suspend the decryption process. Since the display control unit 30 simultaneously reads frames of the number of moving image thumbnails displayed on the screen from the shared memory area 28, an empty block for 1 GOP is immediately generated in the shared memory area 28. As soon as an empty block for 1 GOP is generated, the video decoder 20 writes the already decoded frame for 1 GOP, resumes the suspended process, and then shifts to decoding processing for another stream.

  However, since the video decoder 20 itself has an internal buffer in terms of implementation, the video decoder 20 performs the decoding process without suspending the process even when there is no free block for 1 GOP in the shared memory area 28 of the frame buffer 26. The decoded frame can be stored in an internal buffer. Even if there is no free block for 1 GOP in the shared memory area 28, the video decoder 20 can move the decoded frame from the internal buffer to the free block in the shared memory area 28 as much as possible. Since the number of frames included in one GOP cannot be known without actually decoding, it is efficient for the video decoder 20 to proceed as much as possible regardless of the number of empty blocks in the shared memory area 28, System resources can be used effectively. In the following FIGS. 4A to 4F, the operation will be described without directly touching such a device in order to prevent the description from becoming unnecessarily complicated. However, in FIGS. 4A to 4F, the operation is described here. It should be noted that the efficiency can be further improved by adding ingenuity on mounting.

  When all the frames decoded in units of GOP are read and reproduced, the display control unit 30 needs the frame data of the next GOP. Before the display control unit 30 moves to the next GOP, the video decoder 20 completes the decoding of the next GOP for all the streams simultaneously displayed on the screen and writes the frame data to the shared memory area 28. There must be.

  When the number of simultaneous display streams is N and the number of GOP frames is M, if the display control unit 30 refreshes M frames, for example, if the period of the vertical synchronization signal is L seconds, during LM seconds, The video decoder 20 must complete the decoding process for the next 1 GOP for all of the N encoded streams, that is, the decoding of a total of MN frames and write it to the shared memory area 28. That is, the video decoder 20 is required to have a capability of decoding MN frames in LM seconds, that is, N frames in L seconds. For example, if the simultaneous display moving image thumbnail number N is 10 and the period L of the vertical synchronization signal is 33 milliseconds, a performance of decoding about 300 frames per second is required.

  The video decoder 20 may be implemented by hardware, software, or a combination of both as long as the above performance is guaranteed. For example, the video decoder 20 may be a high-performance CPU or multiprocessor, or a program executed on a multicore processor in which a plurality of processor cores are integrated in one package. Alternatively, the video decoder 20 may be a dedicated circuit that performs a decoding process according to a standard such as MPEG.

  FIG. 4A to FIG. 4F are diagrams for explaining how frames are read from and written to the shared memory area 28 in chronological order. Referring to these figures, a procedure in which a frame decoded from each encoded stream is written in shared memory area 28 by video decoder 20 and a frame is read from shared memory area 28 by display control unit 30 and reproduced. Will be explained. Here, for the sake of simplicity, it is assumed that the simultaneous display stream number N is 3 and the GOP frame number M is 4 or 5.

  FIG. 4A is a diagram showing a state in which the frames constituting the GOP of each stream are written in the shared memory area 28. The shared memory area 28 includes 15 blocks a to o, and all the blocks are initially empty. As shown in the figure, the five frames # 1-1 to # 1-5 constituting the first GOP of the first stream # 1 are stored in five free blocks a to e in the shared memory area 28, The five frames # 2-1 to # 2-5 constituting the first GOP of the second stream # 2 are stored in five empty blocks f to j in the shared memory area 28, and the first of the third stream # 3 is stored. The five frames # 3-1 to # 3-5 constituting the GOP are stored in the five empty blocks k to o in the shared memory area 28.

  FIG. 4B is a diagram showing a state in which the first frame of each stream is read and displayed one by one from the shared memory area 28 in the state shown in FIG. 4A in synchronization with the first vertical synchronization signal. The display control unit 30 synchronizes with the first vertical synchronization signal from the shared memory area 28 for the first frame # 1-1 of the first stream # 1, the first frame # 2-1 of the second stream # 2, the first The first frame # 3-1 of 3 stream # 3 is read out and displayed as moving image thumbnails 111, 112, and 113 on the screen of the display 110, respectively. In the shared memory area 28, the blocks a, f, and k in which the first frames # 1-1, # 2-1, and # 3-1 of each stream # 1, # 2, and # 3 are stored become empty blocks. Of course, if the frame data stored in these blocks a, f, and k is required continuously from the shared memory area 28 due to the difference in the frame rate, the frame data stored in these blocks a, f, and k are required. The blocks may not be empty blocks. Only blocks that do not require frame data at the next refresh timing may be set as empty blocks.

  FIG. 4C is a diagram illustrating a state in which the second frame of each stream is read and displayed one by one from the shared memory area 28 in the state illustrated in FIG. 4B in synchronization with the second vertical synchronization signal. . The display control unit 30 synchronizes with the second vertical synchronization signal, from the shared memory area 28, the second frame # 1-2 of the first stream # 1, the second frame # 2- of the second stream # 2. 2 and the second frame # 3-2 of the third stream # 3 are read out, and the moving image thumbnails 111, 112, and 113 of the streams # 1, # 2, and # 3 displayed on the screen of the display 110 are respectively read. Update at the second frame. In the shared memory area 28, the blocks b, g, and i in which the second frames # 1-2, # 2-2, and # 3-2 of the respective streams # 1, # 2, and # 3 are stored are also empty blocks. .

  FIG. 4D is a diagram illustrating a state in which a frame constituting the next GOP of the first stream # 1 is written in an empty block in the shared memory area 28 in the state illustrated in FIG. 4C. After the second frames # 1-2, # 2-2, and # 3-2 of the respective streams # 1, # 2, and # 3 are displayed as thumbnails on the screen of the display 110 by the display control unit 30, the video decoder 20 Assume that decoding of five frames # 1-6 to # 1-10 constituting the next GOP for the first stream # 1 is completed. The video decoder 20 writes the five newly decoded frames # 1-6 to # 1-10 into the empty blocks in the shared memory area 28. Since the six blocks a, b, f, g, k, and l are in an empty state in the shared memory area 28 (FIG. 4C), the video decoder 20 determines that the five blocks in the GOP next to the first stream # 1. Frames # 1-6 to # 1-10 are stored in five empty blocks a, b, f, g, and k.

  FIG. 4E is a diagram illustrating a state in which the third frame of each stream is read out and displayed one by one from the shared memory area 28 in the state illustrated in FIG. 4D in synchronization with the third vertical synchronization signal. . The display control unit 30 synchronizes with the third vertical synchronization signal, from the shared memory area 28, the third frame # 1-3 of the first stream # 1, and the third frame # 2- of the second stream # 2. 3. The second frame # 3-3 of the third stream # 3 is read out, and the moving image thumbnails 111, 112, 113 on the screen of the display 110 are updated. Blocks c, h, and m in which the third frames # 1-3, # 2-3, and # 3-3 of the streams # 1, # 2, and # 3 are stored in the shared memory area 28 are newly free blocks. become. As a result, the four blocks including the block l which has already been vacant are in an empty state.

  FIG. 4F is a diagram showing a state in which a frame constituting the next GOP of the second stream # 2 is written in the empty block of the shared memory area 28 in the state shown in FIG. 4E. Assume that after the third frame of each stream is displayed as a thumbnail on the screen of the display 110, four frames # 2-6 to # 2-9 constituting the next GOP are decoded for the second stream # 2. Since four blocks c, h, l, and m are empty in the shared memory area 28 (FIG. 4E), four frames # 2-6 to # 2 in the next GOP of the second stream # 2 -9 is stored in these four empty blocks c, h, l, m.

  Here, touching on the ingenuity on implementation, since the number of frames constituting the GOP is not known until decoding, the video decoder 20 advances the decoding of the next GOP for the second stream # 2, and at least the first four frames Frames are sequentially stored in four empty blocks in the shared memory area 28. Even if the number of frames constituting the GOP is five instead of four as a result of decoding, the first four frames can be decoded and buffered in the shared memory area 28. Since the video decoder 20 also has a buffer, the video decoder 20 can finish decoding for 1 GOP and store it in the internal buffer even if the shared memory area 28 is not empty.

  Further, as a mechanism for controlling the suspend and resume of the decoding process of the video decoder 20, the number of free blocks in the shared memory area 28 is compared with the number of frames constituting the decoded GOP, and the number of free blocks is determined from the number of GOP frames. If the number of free blocks is too large, all the frames constituting the GOP are written into the shared memory area 28 and switched to the decoding process of the next stream. If the number of free blocks is smaller than the number of GOP frames, the number of free blocks is the GOP frame. You may provide the control part which suspends the decoding process after that until it reaches a number.

  Thus, when the display control unit 30 reads the frames of the three streams # 1 to # 3 displayed simultaneously from the shared memory area 28 one by one and updates the video thumbnails 111 to 113, three empty blocks are generated. When the video thumbnails 111 to 113 are updated in synchronization with the next vertical synchronization signal, the next frames of the three streams # 1 to # 3 are read out one by one from the shared memory area 28, and three more empty blocks are read out. Occurs. If the video decoder 20 decodes the GOP of a stream and a free block is generated in the shared memory area 28 for writing all the frames in the GOP, the video decoder 20 immediately converts the frame in the GOP into a free block. Write and proceed with the decoding process of the GOP of the next stream. The video decoder 20 interrupts the decoding process and enters a waiting state until an empty block sufficient to store a frame constituting 1 GOP is generated in the shared memory area 28.

  In the present embodiment, a shared memory area 28 that is shared by a plurality of streams is provided in the frame buffer 26 so that a frame decoded from any stream is stored in an arbitrary empty block without being distinguished. Configured. By displaying a plurality of moving image thumbnails simultaneously on the screen, a plurality of frames are simultaneously read from the shared memory area 28 every time the moving image thumbnails are refreshed, and a plurality of empty blocks are generated discontinuously in the shared memory area 28. . The video decoder 20 can asynchronously write the decoded GOP unit frame in this empty block and proceed with the GOP decoding process of the next encoded stream.

  Here, in order to better understand the significance of providing the shared memory area 28 in the frame buffer 26, referring to FIGS. 5A to 5D, the shared memory area 28 is not provided in the frame buffer 26, and each stream is independent. A frame read / write procedure when the dedicated memory area is provided will be described in chronological order. In this case, in the frame buffer 26, the memory area is divided for each stream, and the memory area allocated for a certain stream stores only frames decoded from the stream and stores frames decoded from other streams. There is a restriction that it cannot be done.

  FIG. 5A is a diagram showing a state in which frames constituting the GOP of each stream are written in a dedicated memory area provided independently for each stream. In the frame buffer 26, a dedicated memory area for the first stream # 1 (referred to as “first dedicated memory area”) 29a and a dedicated memory area for the second stream # 2 (referred to as “second dedicated memory area”) 29b, a dedicated memory area (referred to as “third dedicated memory area”) 29c for the third stream # 3 is provided independently. The first, second, and third dedicated memory areas 29a, 29b, and 29c include five blocks 1a to 1e, 2a to 2e, and 3a to 3e, respectively, and these blocks store only the frames of the respective streams. It cannot be used to store frames of other streams.

  The five frames # 1-1 to # 1-5 constituting the GOP of the first stream # 1 are stored in the five blocks 1a to 1e of the first dedicated memory area 29a. Similarly, the five frames # 2-1 to # 2-5 constituting the GOP of the second stream # 2 are transferred to the five blocks 2a to 2e of the second dedicated memory area 29b to the third stream # 3. The five frames # 3-1 to # 3-5 constituting the GOP are stored in the five blocks 3a to 3e of the third dedicated memory area 29c.

  FIG. 5B shows a state in which the first frame of each stream is read out and displayed one by one from each dedicated memory area 29a, 29b, 29c in the state shown in FIG. 5A in synchronization with the first vertical synchronization signal. FIG. The first frames # 1-1, # 2-1, and # 3-1 are read from the dedicated memory areas 29a, 29b, and 29c in synchronization with the first vertical synchronization signal, and the moving image thumbnails 111, 112, and 113 are displayed on the screen. Is displayed. At this time, one free block 1a, 2a, 3a is generated in each of the dedicated memory areas 29a, 29b, 29c.

  FIG. 5C shows a state in which the second frame of each stream is read and displayed one by one from each dedicated memory area 29a, 29b, 29c in the state shown in FIG. 5B in synchronization with the second vertical synchronization signal. FIG. The second frames # 1-2, # 2-2, and # 3-2 are read from the dedicated memory areas 29a, 29b, and 29c in synchronization with the second vertical synchronization signal, and the moving image thumbnails 111 and 112 on the screen are read. 113 are updated. Another free block 1b, 2b, 3b is generated in each dedicated memory area 29a, 29b, 29c.

  At this point, it is assumed that the five frames # 1-6 to # 1-10 constituting the next GOP of the first stream # 1 have been decoded. The first dedicated memory area 29a has two empty blocks 1a and 1b, the second dedicated memory area 29b has two empty blocks 2a and 2b, and the third dedicated memory area 29c has two empty blocks 3a and 3b. There are a total of 6 free blocks. However, since the five frames # 1-6 to # 1-10 of the first stream # 1 can be stored only in the first dedicated memory area 29a, these frames can be written to the frame buffer 26. Therefore, the video decoder 20 enters a waiting state and cannot proceed to the decoding process for the next stream.

  FIG. 5D is a diagram illustrating a state in which all the remaining frames of each stream are read from the dedicated memory areas 29a to 29c through the third to fifth vertical synchronization signals. As shown in the figure, the fifth frames # 1-5, # 2-5, and # 3-5 of the first, second, and third streams # 1, # 2, and # 3 are the first and second frames, respectively. Until the moving image thumbnails 111, 112, 113 on the screen are read out from the third dedicated memory areas 29a, 29b, 29c and updated, five empty blocks do not occur in the first dedicated memory area. Finally, five frames # 1-6 to # 1-10 constituting the next GOP of the first stream # 1 are written into the first dedicated memory area 29a, and the video decoder 20 transmits the next encoded stream. The decryption process can proceed.

  However, since the next GOP decoding process has not yet been performed for the second and third streams # 2 and # 3 at this point, there are frames to be displayed in the second and third dedicated memory areas 29b and 29c. Not buffered. When the moving image thumbnail is updated at the timing of the sixth vertical synchronization signal, the image can be updated for the first stream # 1, but the image cannot be updated for the second and third streams # 2 and # 3. , The frame is missing.

  As described above, when the dedicated buffer memory area is provided separately for each stream in the frame buffer 26, it takes time until a free area that can store a frame of 1 GOP is generated in the dedicated memory area. Since the process is stopped and the decoding process cannot proceed by switching to the next stream, it becomes difficult to display a plurality of moving image thumbnails without dropping frames simultaneously. Therefore, in the present embodiment, a shared memory area 28 is provided in the frame buffer 26 so that a frame can be written into an empty block without distinguishing between a plurality of encoded streams. Video thumbnails can be displayed without dropping frames.

  The frame buffer 26 according to the present embodiment means a “buffer pool” for storing frame data for 1 GOP decoded from each of a plurality of streams, and holds pixel data displayed on the display. Note that this has a different meaning from VRAM (Video Random Access Memory). A frame memory such as a VRAM for outputting pixel data to the display 110 may be provided separately from the frame buffer 26, or the frame buffer 26 may be used as a frame memory for outputting pixel data to the display 110. Good.

  In addition, the video decoder 20 has its own buffer for decoding processing, and can store the reference frame data necessary for decoding the frame subjected to the interframe prediction encoding in the internal buffer. Therefore, the video decoder 20 can decode regardless of whether it is a closed GOP or an open GOP. In the case of an open GOP, that is, a GOP that needs to reference a frame of another GOP, the video decoder 20 leaves a frame that needs to be referred to when decoding in a buffer in the decoder and uses it for decoding the next GOP. do it.

  However, since the video decoder 20 of the present embodiment is configured to switch and decode streams in units of GOPs, in the case of an open GOP, a frame to be referred to in the decoding of the next GOP for a certain stream is buffered in the decoder. If this is held, the switching of the stream is completed, and the reference frame must be held in the buffer until returning to the stream. When the capacity of the internal buffer of the video decoder 20 is limited, buffering the reference frame used for decoding the next GOP of a certain stream until the stream switching is completed is not a good strategy in terms of memory capacity.

  Therefore, the reference frame used for decoding the next GOP for a certain stream remains in the frame buffer 26 instead of the internal buffer of the video decoder 20, and the video is switched when the stream is switched once and returned to the stream. The decoder 20 may be configured to read the reference frame from the frame buffer 26 and use it for decoding. In general, the frame buffer 26 capable of storing a plurality of streams of frame data is designed to have a larger memory capacity than the internal buffer of the video decoder 20. The configuration in which the reference frame is held in the frame buffer 26 and the video decoder 20 refers to the frame buffer 26 is advantageous in terms of the memory design of the system.

  Here, the display control of the moving image thumbnail by the display control unit 30 and the priority control of a plurality of encoded streams to be decoded by the priority control unit 16 will be described with some specific examples.

  6A to 6C are diagrams illustrating an example of display control of a moving image thumbnail. The display control unit 30 displays moving image thumbnails arranged vertically on the screen of the display 110, and controls to switch the moving image thumbnails displayed on the screen by an up and down scroll operation. In this case, even video thumbnails that are not displayed on the screen may be displayed by scrolling up and down, so the video decoder 20 can also display streams that may be displayed by scrolling up and down. Decryption processing is executed and written to the shared memory area 28. The display control unit 30 reads and displays a frame from the shared memory area 28 only for the stream displayed as the moving image thumbnail on the screen.

  In the example of FIG. 6A, among the nine streams # 1 to # 9 decoded by the video decoder 20, only five streams # 3 to # 7 are displayed as moving image thumbnails 121 to 125 on the screen. The other four streams # 1, # 2, # 8, and # 9 are currently outside the screen and are not displayed as moving image thumbnails. The moving image thumbnail 123 of the central stream # 5 is displayed larger than the moving image thumbnails 121, 122, 124, and 125 of the other streams # 3, # 4, # 6, and # 7, and the audio decoder 22 displays the central moving image thumbnail 124. For audio, the audio is decoded and played in sync with the video.

  The user can switch the moving image thumbnail displayed in the center by scrolling the screen up and down by operating an arrow key or the like. For example, when the scroll operation is performed so that the moving image thumbnail row is shifted upward on the screen, the moving image thumbnail 123 of the stream # 5 that has been in the center until then is moved to the upper position as shown in FIG. The moving image thumbnail 124 of the stream # 6 that has been moved and located immediately below is moved to the center and displayed large. At this time, the stream # 8 outside the screen in FIG. 6 (a) is newly displayed as the moving image thumbnail 126 at the lowermost position of the screen, and is displayed at the uppermost position of the screen in FIG. 6 (a). Since the moving image thumbnail 121 of the stream # 3 that has been displayed appears outside the screen, it is not displayed thereafter.

  Since the video decoder 20 also decodes the streams # 1, # 2, # 8, and # 9 outside the screen in FIG. 6A and stores the frames in the shared memory area 28, the user can perform a scroll operation. The stream that newly appears on the screen can be immediately displayed as a moving image thumbnail even if the operation is performed. In the example of FIG. 6A, even when scrolling up and down by two moving image thumbnails, the moving image thumbnails are displayed on the screen without delay.

  The priority control unit 16 converts the five streams # 3 to # 7 displayed on the screen in FIG. 6A to the four streams # 1, # 2, #, which are outside the screen and are not displayed. 8 and # 9 are set to be higher in priority than # 9. The switching unit 14 switches the nine streams # 1 to # 9 to be decoded by the round robin method with priority according to the priority set by the priority control unit 16. As a result, the five displayed streams # 3 to # 7 are decoded more frequently than the four non-displayed streams # 1, # 2, # 8, and # 9, and can be used for the decoding process. Even when there is a limit to the computational resources that can be obtained, the decoding process can be reliably performed in time for the five streams # 3 to # 7 that are being displayed.

  The priority control unit 16 may make the priority of the stream # 5 displayed at the center of the screen in FIG. Further, the priority may be higher for a stream closer to the center of the screen. For example, the priority order of the streams may be set so that the priority decreases in the order of # 5, # 4, # 6, # 3, # 7, # 2, # 8, # 1, # 9.

  When the screen scroll operation results in the state of FIG. 6B, the switching unit 14 replaces the decoding target stream, and the uppermost stream # 1 is excluded from the decoding target as shown in FIG. Stream # 10 is newly included in the decoding target. As a result, nine new streams # 2 to # 10 are to be decoded by the video decoder 20. In this way, moving picture thumbnails can be smoothly displayed at any time by adapting to the screen scrolling in real time by switching the streams to be decoded in accordance with the screen scrolling operation.

  Various variations are conceivable for the display control by the display control unit 30 and the priority control by the priority control unit 16. In the above example, the display control unit 30 provides a user interface that displays a moving image thumbnail at the center of the screen in a large size and moves the moving image thumbnail to be noticed to the center by scrolling the screen. The display control unit 30 may display a list of moving image thumbnails, provide an interface for the user to select a moving image thumbnail to be noticed by moving the cursor with an arrow key or the like, and display the selected moving image thumbnail in a large size.

  As another method of highlighting the video thumbnail that the user is paying attention to, the display control unit 30 plays the video thumbnail that is focused on at a higher frame rate than other video thumbnails, or The video thumbnails of interest may be decoded at a normal frame rate, and other video thumbnails may be played back at a reduced frame rate, so that the quality of the video thumbnails of interest may be relatively improved.

  Variations of priority control by the priority control unit 16 will be described together below. These variations of priority control can be used alone or in combination.

  (A) The priority control unit 16 sets a priority order depending on the number of GOP frames of each stream. For example, a higher priority is set for a stream having a smaller number of GOP frames. A stream with a smaller number of frames in the GOP finishes reproduction at an earlier timing, so that frames in the shared memory area 28 of the frame buffer 26 are not consumed earlier. By preferentially decoding a stream with a small number of GOP frames, it is possible to prevent the frames necessary for reproduction from being depleted in the shared memory area 28 and to prevent frame dropping.

  (B) The priority control unit 16 refers to the buffer management table 24 to check the number of frames remaining without being reproduced in the shared memory area 28 of the frame buffer 26 for each stream, and depends on the number of remaining frames. Set the priority. For example, the priority is set higher for a stream with a smaller number of frames remaining in the shared memory area 28. As a result, the number of remaining frames can be averaged between a plurality of streams, and a situation in which frames necessary for reproduction are exhausted and frames are dropped can be prevented.

  (C) The priority control unit 16 sets a priority order depending on the frame rate of each stream. Since the higher the frame rate stream, the faster the frame is consumed from the shared memory area 28, the priority is increased. As a result, for the high frame rate stream, the frame data is decoded in time for the reproduction time and buffered in the shared memory area 28.

  (D) In addition to the stream currently displayed on the screen, a stream that may be displayed in the near future may be decoded in advance. As an example of decoding a stream that may be displayed in advance, in addition to the case of the scroll operation described with reference to FIGS. In some cases, an operation displays a video thumbnail of a link destination stream. In such a case, the priority control unit 16 sets a priority order that gives priority to a stream displayed on the screen over a stream not displayed on the screen. As a result, the stream displayed on the screen is more reliably decoded and stored in the frame buffer 26, and the encoded stream that may be displayed on the screen is decoded with low priority to the frame buffer 26. You can store and prepare for the potential of playback.

  (E) When the display control unit 30 performs control to change the reproduction quality of a plurality of moving image thumbnails displayed on the screen, the priority control unit 16 sets a priority order according to the reproduction quality of the moving image thumbnails. For example, for a stream with a high playback quality requirement, a priority is set such that the higher the playback quality of the video thumbnail, the higher the priority, and the higher playback quality request stream is decoded over other streams. Highlight display such as playback at a high frame rate or high resolution becomes possible.

  As described above, according to the decoding device 100 of the present embodiment, a single video decoder 20 decodes in units of GOPs while switching a plurality of streams, buffers them in the shared memory area 28, and displays the display control unit. 30 can read frames of a plurality of streams to be displayed from the shared memory area 28 and display them simultaneously as moving images on the screen. The video decoder 20 operates asynchronously with the display control unit 30. If there is no free area in the shared memory area 28 for storing frames decoded in GOP units, the decoding process is temporarily stopped. Therefore, it is not necessary to provide a capacity larger than the shared memory area 28, and the amount of memory can be saved.

  Since the display control unit 30 displays a plurality of streams at the same time, an empty area corresponding to the number of simultaneous display streams is generated in the shared memory area 28 every time the moving image is refreshed. If an empty area for storing a frame for 1 GOP is generated in the shared memory area 28, the video decoder 20 can immediately write the frame for 1 GOP to the empty area and immediately proceed to the decoding process of the next stream. Frame data is not depleted in the shared memory area 28 and frame dropping does not occur.

  In the decoding apparatus 100 according to the present embodiment, since only one demultiplexing unit 18 and video decoder 20 is prepared and a plurality of streams are decoded, when implemented in hardware, the plurality of demultiplexing units 18 and the video are provided. The circuit scale can be reduced as compared with the configuration using the decoder 20. Even when implemented by software, since only one instance of the demultiplexing unit 18 and the video decoder 20 needs to be executed, the system control program can be reduced in weight, and scalable according to the number of simultaneous display streams. Can take various software configurations. Furthermore, the usage rate of computing resources such as memory and CPU is inevitably reduced. Two or more instances of the video decoder 20 may be provided according to system performance and system resources. In any case, there is an advantage in that a large number of video decoders 20 can be reproduced in parallel by decoding a large number of streams substantially in parallel by using as few instances of the video decoder 20 as possible.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. . Such a modification will be described.

  In the above embodiment, the present invention has been described based on an example in which a plurality of encoded streams is decoded and displayed as a list of moving image thumbnails at the same time. However, the present invention decodes a plurality of encoded streams and reproduces them simultaneously. It can be applied to other uses as well. For example, even when a plurality of moving images are pasted and displayed on a virtual space screen generated by computer graphics, the present invention can be applied to efficiently reproduce a plurality of moving images simultaneously with a single decoder. .

  In the embodiments, video decoding has been described. However, the present invention is also applicable to audio decoding in principle. Similarly to video, audio has a unit of encoding and decoding called an audio frame. As in the case of a video stream, a plurality of audio streams are switched, and a set of audio frames is set in a minimum decoding unit corresponding to a video GOP. You may decode. As a need for reproducing a plurality of audios simultaneously by applying the present invention, for example, the following applications can be considered. In a virtual space generated by computer graphics, an environment where different voices and music can be heard depending on the location is realized, and when the character operated by the user approaches the location, the sound can be heard loudly, and the sound can be heard quietly when moving away. Constitute. In such a virtual space environment, a plurality of audio streams are simultaneously reproduced so that the voice or music of each place can be heard simultaneously from around the user's character.

  In the embodiment, the example in which the coded stream is cut out and decoded in units of GOP which is the minimum decoding unit on the assumption that the encoded stream is subjected to inter-frame predictive coding has been described. However, inter-frame predictive coding such as Motion-JPEG is described. For example, a moving image stream encoded according to a standard that does not perform the extraction may be cut out with the number of frames designed according to the number of simultaneously displayed streams, and a plurality of streams may be switched and decoded.

It is a block diagram of the decoding apparatus concerning embodiment. It is a figure explaining the data structure of the shared memory area | region of the frame buffer of FIG. It is a figure explaining the data structure of the buffer management table of FIG. It is a figure which shows the state by which the frame which comprises GOP of each stream was written in the shared memory area | region of FIG. FIG. 4B is a diagram showing a state in which the first frame of each stream is read and displayed one by one from the shared memory area in the state shown in FIG. 4A in synchronization with the first vertical synchronization signal. FIG. 4B is a diagram illustrating a state in which the second frame of each stream is read and displayed one by one from the shared memory area in the state illustrated in FIG. 4B in synchronization with the second vertical synchronization signal. FIG. 4D is a diagram illustrating a state in which a frame constituting the next GOP of the first stream is written in an empty block in the shared memory area in the state illustrated in FIG. 4C. FIG. 5D is a diagram showing a state in which the third frame of each stream is read and displayed one by one from the shared memory area in the state shown in FIG. 4D in synchronization with the third vertical synchronization signal. FIG. 4B is a diagram illustrating a state in which a frame constituting the next GOP of the second stream is written in an empty block in the shared memory area in the state illustrated in FIG. 4E. It is a figure which shows the state by which the frame which comprises GOP of each stream was written in the dedicated memory area provided independently for every stream. FIG. 5B is a diagram showing a state in which the first frame of each stream is read out and displayed one by one from each dedicated memory area in the state shown in FIG. 5A in synchronization with the first vertical synchronization signal. FIG. 5B is a diagram showing a state in which the second frame of each stream is read and displayed one by one from each dedicated memory area in the state shown in FIG. 5B in synchronization with the second vertical synchronization signal. FIG. 6 is a diagram illustrating a state in which all the remaining frames of each stream have been read from each dedicated memory area in the state illustrated in FIG. 5B through the third to fifth vertical synchronization signals. 6A to 6C are diagrams illustrating an example of display control of a moving image thumbnail.

Explanation of symbols

  10 storage, 12 stream buffer, 14 switching unit, 16 priority control unit, 18 demultiplexing unit, 20 video decoder, 22 audio decoder, 24 buffer management table, 26 frame buffer, 28 shared memory area, 30 display control unit, 100 Decoding device, 110 display.

Claims (12)

When a plurality of encoded streams in which a frame of a moving image is encoded are given, a switching unit that switches to any one of the plurality of encoded streams and reads out the encoded data of the encoded stream;
A decoder that decodes encoded data of the encoded stream selected by the switching unit to generate a frame;
A frame buffer that temporarily buffers frames decoded from each encoded stream;
The frame buffer is a single memory area shared between the plurality of encoded streams, and temporarily stores a predetermined number of frames decoded from each of the plurality of encoded streams. Any frame including a shared memory area and decoded from any encoded stream can be stored in any free area without being limited by the storage position in the shared memory area. A characteristic decoding apparatus. A display control unit that reads a frame decoded from at least one of the plurality of encoded streams from the shared memory area and simultaneously displays the frame as a moving image on a screen;
The decoder is configured to write a predetermined number of frames decoded from the encoded stream selected by the switching unit by causing the display control unit to simultaneously display at least one encoded stream as a moving image. When the free area is generated in the shared memory area, the predetermined number of frames are written in the free area, and the process proceeds to the decoding process of the next selected encoded stream by the switching unit. Item 4. The decoding device according to Item 1. For each frame decoded from each of the plurality of encoded streams stored in the shared memory area, information on the storage position of the frame in the shared memory area and information on the playback order of the frames are held for each encoded stream. A buffer management table for
The display control unit refers to information on the storage position and information on the playback order in the buffer management table, and displays frames decoded from at least one of the plurality of encoded streams. The decoding device according to claim 2, wherein reading is performed from the shared memory area.   The switching unit switches a group of a predetermined number of frames, which is a minimum decodable unit for each of the plurality of encoded streams, to a switching unit and switches to any one of the plurality of encoded streams. 4. The decoding apparatus according to claim 1, wherein encoded data of a group is read and supplied to the decoder.   The switching unit switches to any one of the plurality of encoded streams based on a priority order in which the priority becomes higher as the number of frames included in the frame group of each of the plurality of encoded streams decreases. 5. The decoding apparatus according to claim 4, wherein the encoded data of the group of frames is read out.   The switching unit switches to any one of the plurality of encoded streams based on a priority order in which the priority becomes higher as the encoded stream has a smaller number of remaining frames that are not reproduced in the shared memory area. The decoding apparatus according to claim 1, wherein the encoded data of the encoded stream is read out.   The switching unit, based on a priority order that gives priority to an encoded stream that is reproduced as a moving image on a screen over an encoded stream that is not reproduced as a moving image on a screen, among the plurality of encoded streams. 5. The decoding apparatus according to claim 1, wherein the encoded data of the encoded stream is read by switching to any one of the encoded streams. 6.   When the quality of the plurality of moving images displayed on the screen is different, the switching unit determines whether the plurality of encoded streams are based on the priority order in which the higher the priority is as the encoded stream has a higher display quality as a moving image. 5. The decoding device according to claim 1, wherein the encoded data of the encoded stream is read by switching to any one of them.   When the arrangement of a plurality of moving images displayed on the screen scrolls according to a user's operation, the switching unit moves the encoded stream currently displayed on the screen as a moving image to the next screen by the user's scrolling operation. The encoded data of the encoded stream is read out by switching to any one of the plurality of encoded streams based on a priority order that has priority over the encoded stream on which a moving image is to be displayed. The decoding device according to any one of claims 1 to 4. When a plurality of encoded streams in which a frame of a moving image is encoded are given, a step of switching to any one of the plurality of encoded streams and reading the encoded data of the encoded stream;
Decoding encoded data of the read encoded stream to generate a frame;
A single memory area shared between the plurality of encoded streams, each of which is a shared memory area for temporarily holding a predetermined number of frames decoded from each of the plurality of encoded streams; Storing a frame decoded from the encoded stream in any empty position without limiting the storage position in the shared memory area, regardless of which encoded stream is used for decoding. Including decryption method. When a plurality of encoded streams in which a frame of a moving image is encoded are given, a switching function of switching to any one of a plurality of encoded streams and reading the encoded data of the encoded stream;
A decoding function for decoding the encoded data of the encoded stream selected by the switching function to generate a frame;
A single memory area shared between the plurality of encoded streams, each of which is a shared memory area for temporarily holding a predetermined number of frames decoded from each of the plurality of encoded streams; A buffering function for storing frames decoded from an encoded stream in any empty position without restricting the storage position in the shared memory area, regardless of which encoded stream is used for decoding. A program characterized by causing a computer to realize the above. For frames decoded from the plurality of encoded streams stored in the shared memory area, information on the storage position of the frames in the shared memory area and information on the reproduction order of the frames are buffered for each of the plurality of encoded streams. A buffer management function held in the management table;
With reference to the information about the storage position and the information about the playback order in the buffer management table, a frame decoded from at least one of the plurality of encoded streams is read from the shared memory area. The program according to claim 11, further causing a computer to realize a display control function for simultaneously displaying a moving image on a screen.

Images