JP6742515B2 - CODED VIDEO REPRODUCTION DEVICE AND CODED VIDEO REPRODUCTION METHOD - Google Patents

Description

The present invention relates to a reproducing apparatus for coded video recorded on an information recording medium such as an SD card and an optical disc.

2. Description of the Related Art In recent years, electronic advertisements (digital signage) that display moving image on a screen of an image display device have become popular as advertisement media. A PC (personal computer) equipped with a high-performance processor is often used as an image display device used for digital signage, but from the viewpoint of cost, reliability, environment resistance, etc., electronic advertisement display It is desirable to apply a dedicated image display device (embedded device) with built-in functions. However, in general, embedded devices are limited in processor performance, memory size, and the like, and since an AV decoder that decodes encoded video is mounted in hardware, operation restrictions are severe and decoding processing is difficult. There is a problem that the degree of freedom is low.

As one form of digital signage, a form in which coded images stored in a portable recording medium are displayed by loop reproduction is widely used. However, when sequentially reproducing a plurality of video streams, it is necessary to perform display control in accordance with the display time in video frame units recorded in the video stream itself, and thus initialization processing of the AV decoder is required for each video stream. Was there. The initialization processing of the AV decoder includes presetting of the resolution and frame rate of the video supplied to the AV decoder, initialization processing of the buffer of the AV decoder, and the like.

Further, initialization processing is often required even when the multiplexing format of the video stream is changed. The time required for such an initialization process causes a gap between images which is a black image display period when the video stream is switched. In digital signage, the occurrence of a waiting time for not displaying an image due to a gap leads to a loss of an opportunity for displaying an image, which is regarded as a very problem. Further, if the video brightness and the audio output level are greatly different at the time of scene switching, there is a problem that the user feels uncomfortable and is uncomfortable.

As one means for solving such a problem, for example, as disclosed in Patent Document 1, video/audio information that overlaps a joint between video and audio is separately created, and natural video scene switching is viewed. There is known a video editing device that can do this.

JP, 7-23329, A

In the conventional technique disclosed in the prior art document 1, the video and audio in the vicinity of the joint between the video scenes are temporarily recorded, and the video and audio to be overlapped are separately created offline. This method is useful when there is a processor capable of high-speed processing, such as a general-purpose computer, and when the processing of synthesizing video scenes is performed by an embedded device, the load on the CPU (Control Processing Unit) increases and Is not practical because it requires processing time.

In general, an AV decoder installed in an embedded device generally performs a reproduction process for each video file unit, and it is necessary to perform an initialization process for each reproduction process. There is a problem that loop reproduction cannot be performed without creating a gap.

The present invention has been made to solve the above problems, and when video files are sequentially played back, it is possible to continuously display video without creating a gap when switching video streams. Another object of the present invention is to provide a coded video reproduction device.

An encoded video reproduction device according to the present invention is an encoded video reproduction device for reproducing a stream information file in which encoded video information and audio information are multiplexed, and the video information is reproduced from the stream information file. And a stream separation unit that separates the audio information into access units, and a buffer that records the video information and the audio information divided into access units in association with respective display start times. A decoder for reading and decoding the video information and the audio information from the buffer based on the display start time; a control for recording the display start time, the video information and the audio information in the buffer; A data supply control unit that controls the supply of the video information and the audio information from a buffer to the decoder, and the stream information prior to the recording of the video information and the audio information in the buffer by the data supply control unit. The data access control unit includes an analysis unit that analyzes the data of the first access unit and the last access unit of the file. The display start time of the last access unit of the information file is reset to the display start time of the first access unit of the subsequent stream information file prior to the supply of the video information and audio information of the subsequent stream information file , The analysis unit rearranges the access units so that the access units capable of completely decoding the frame image are the first access unit and the last access unit, and the data supply control unit rearranges them. The data is recorded in the buffer in the order described above .

According to the coded video playback device of the present invention, it is possible to continuously display video without causing a gap between videos when switching video streams.

It is a functional block diagram showing the system configuration of the coded video reproducing device according to the present invention. It is a schematic diagram which shows the logical file structure of an SD card. It is a figure explaining the syntax of a reproduction control information file. It is a schematic diagram explaining the relationship between the storage order arrangement|sequence of a coded compressed video file, and a display order arrangement|sequence. It is a figure explaining the syntax of playlist information. It is a figure explaining the preservation|save form of encoded video information and encoded audio information. It is a figure which shows the timing of image display and data supply. It is a figure explaining adjustment of image brightness and an audio output level in the case of reproducing a stream information file. It is a figure explaining the whole operation|movement in the coded video reproducing apparatus which concerns on this invention. It is a flow chart which shows operation of static analysis processing. It is a flow chart which shows operation of file reproduction processing.

<Embodiment>
<Device configuration>
FIG. 1 is a functional block diagram showing a system configuration of a coded video reproduction device 100 according to an embodiment of the present invention.

As shown in FIG. 1, the coded video playback device 100 includes a system control unit 101, a playback driver 102, a stream control unit 110, a video/audio decoder 114 (decoder), a video/audio adjustment unit 115, and a memory 120, and plays back. The driver 102, the stream control unit 110, the video/audio decoder 114, and the video/audio adjusting unit 115 form a decoder block 119. The decoder block 119 performs a reproduction process of stream information based on a decoder control signal from the system control unit 101.

The encoded video reproduction device 100 receives a request from a user or the like via the operation unit 130, reads out a video file or the like recorded in the SD card 140, reproduces it, and displays it on a display device 116 such as a liquid crystal display device. .. Generally, the operation unit 130 refers to an input device such as an operation panel or a remote controller arranged on the front panel of the encoded video reproduction device 100.

The system control unit 101 performs integrated control of the entire encoded video reproduction device 100, interprets the content of the instruction given via the operation unit 130, and controls the decoder block 119 to reproduce arbitrary stream information. For example, the reproduction driver 102 is controlled to read information from an information recording medium such as the SD card 140 and reproduce it.

The SD card 140 records stream information in which coded video information and coded audio information, which will be described later, are multiplexed, and reproduction control information of the stream information. The reproduction control information includes video and audio attribute information about encoded video and audio streams separated from stream information, access unit of stream information, usually reproduction start time information and reproduction start position information in GOP (Group of Picture) units. It includes information indicating the correspondence relationship of.

The stream information includes information corresponding to MPEG (Moving Picture Experts Group)-2 TS (Transport Stream) format, MP4 format, and WMV (Windows Media Video) format (Windows is a registered trademark).

The stream information file multiplexed in the MP4 format and the WMV format holds the reproduction control information inside the file. On the other hand, since the MPEG-2 TS format does not hold the playback control information in the stream information file, it separately holds the playback control information file associated with the stream information file.

The stream control unit 110 centrally controls the stream flow of the entire system, and the stream control unit 110 includes a stream separation unit 111 and an AV buffer 112 (buffer). In the stream information file read from the SD card 140 by the reproduction driver 102, the encoded video information and the encoded audio information are multiplexed, and the stream separation unit 111 separates the encoded video information and the encoded audio information. , Are stored in the AV buffer 112.

The video/audio decoder 114 performs a decoding process on the coded video information coded by MPEG-2 or the like to decode the video information, and also coded audio information coded by AC (Audio Coding number)-3 or the like. Is decoded into audio information.

The video/audio adjustment unit 115 receives the video information and the audio information output from the video/audio decoder 114, performs video image quality adjustment and output volume level adjustment, respectively, and outputs them to the display device 116 as an output video signal and an output audio signal. ..

The system control unit 101 has a data supply control unit 151, a static analysis unit 152 (analysis unit), and an output video/audio control unit 153.

The data supply control unit 151 controls the stream control unit 110 to read the video stream information from the reproduction driver 102 based on the playlist information 160 read from the memory 120, and the encoded video separated by the stream separation unit 111. The stream separation unit 111 is controlled so that the information and the encoded audio information are stored in the AV buffer 112 in access unit units.

The playlist information 160 is information indicating in what order the video files recorded on the SD card 140 are to be played back, and the file name of the playback target, playback start time and playback end time, start point and end point. The analysis information of the neighborhood is saved.

The static analysis unit 152 analyzes the video information and the audio information in the vicinity of the reproduction start time and the reproduction end time of the stream information file, for example, for 0.5 seconds in the encoding compression unit, and determines the start point of the playlist information 160 and Record as end point attribute information.

The output video/audio control unit 153 controls the video/audio adjustment unit 115 based on the attribute information of the start point and the end point recorded in the playlist information 160.

Each function of the data supply control unit 151, the static analysis unit 152, and the output video/audio control unit 153 may be realized by firmware or the like incorporated in the system control unit 101, but a hardware having similar functions. It may be realized by wear. Further, although the data supply control unit 151, the static analysis unit 152, and the output video/audio control unit 153 are shown as separate configurations, they may have a single configuration in which their respective functions are integrated.

Similarly, the functions of the stream separation unit 111 and the AV buffer 112 in the stream control unit 110 may be realized by firmware or the like incorporated in the stream control unit 110, but hardware having similar functions may be used. You can realize it.

<Decoding process flow>
Next, the flow of the decoding process in the encoded video reproducing device 100 will be described. When the reproduction driver 102 reproduces the stream information file read from the SD card 140, the system control unit 101 controls the stream control unit 110 to read the reproduction control information related to the stream information file to be reproduced from the SD card 140 in advance. Keep it. The system control unit 101 holds the read reproduction control information in the memory 120, and reads it as necessary. Based on the reproduction control information read from the memory 120, the system control unit 101 instructs each unit under control control to prepare for reproduction.

The stream information file read from the SD card 140 by the reproduction driver 102 is input to the stream control unit 110, and is separated by the stream separation unit 111 into encoded video information and encoded audio information.

After that, the stream separation unit 111 divides the coded video information and the coded audio information into access units, and stores them in the AV buffer 112 in access unit units. The access unit is a unit of processing of a plurality of data handled in the coded video information and the coded audio information. The coded video information is one frame unit and the coded audio information is a predetermined time (for example, 0.1 seconds). An access unit is formed by unit data.

The access unit information recorded in the AV buffer 112 is sequentially supplied to the video/audio decoder 114. The video/audio decoder 114 decodes the coded video information into video information and decodes the coded voice information into audio information.

The decoded video information and audio information are input to the video/audio adjustment unit 115, and the video image quality adjustment such as the brightness and sharpness of the entire video screen is performed, and the output volume level adjustment is performed as the output video signal and the output audio signal. Output to the display device 116.

The display device 116 displays an image based on the input output video signal, and emits sound based on the output audio signal through a speaker for viewing.

<Control by system controller>
The encoded video reproduction device 100 according to the present invention is characterized by the control by the system control unit 101. That is, before the data supply control unit 151 stores the data in the AV buffer 112, the static analysis unit 152 analyzes the data of the stream information file 221, and the start point of the picture that can completely decode the frame image is set as the start point. Sort so that the end point is reached.

Further, the static analysis unit 152 previously analyzes video information and audio information in the vicinity of the reproduction start time and the reproduction end time of the stream information file, for example, for 0.5 seconds in the encoding compression unit, and calculates the luminance distribution of the video information. , The brightness of the final video frame, the volume level distribution of the audio information, and the volume level of the final audio access unit are acquired and recorded in the playlist information 160 as attribute information of the start point and the end point.

The output video/audio control unit 153 controls the video/audio adjustment unit 115 at the seam of the video scenes based on the attribute information of the start point and the end point recorded in the playlist information 160, so that the output video/audio control unit 153 operates rapidly. Output control of image quality and sound quality is performed so that changes in image brightness and changes in audio output level do not occur. As a result, even if the video brightness and the audio output level greatly change at the video scene switching point, the video scene can be switched without causing any discomfort to the user.

<SD card logical file structure>
FIG. 2 is a schematic diagram showing the logical file structure of the SD card 140. The root directory 200 is a directory structure at the highest level of a file structure that logically forms a hierarchical structure. The root directory 200 is a folder in which the reproduction control information file 210, which is the content management information recorded in the SD card 140, is collectively recorded, and a stream information file 221 is collectively recorded in the folder. It has a stream management directory 220. A storage area in which the reproduction control information directory 210 is constructed is called a reproduction control information area, and a storage area in which the stream management directory 220 is constructed is called a stream information area.

In FIG. 2, the reproduction control information file 211 and the stream information file 221 are recorded by a 5-digit file name and are associated with each other on a one-to-one basis. The file name need only be a 5-digit number, and need not be a serial number. Further, regarding the MP4 format and the WMV format, since the reproduction control information is included in the stream information file 221, it is not necessary to provide the corresponding reproduction control information file 211.

Although FIG. 2 shows an example in which the reproduction control information file 211 and the stream information file 221 are arranged in separate directories, they may be arranged in the same directory, or these files may be arranged in the root directory 200. It may be placed directly. Further, the reproduction control information file 211 and the stream information file 221 are associated with each other one-to-one, but the corresponding stream information file 221 may be collectively recorded in the reproduction control information file 211. ..

<Syntax of playback control information file>
FIG. 3 is a diagram for explaining the syntax in the reproduction control information file 211. "Start_PTS301" indicates a PTS (Presentation Time Stamp) meaning the display start time of the first access unit of the target stream information file 221, and "End_PTS302" indicates the PTS meaning the display start time of the last access unit. ..

The "attribute information management table 310" defines video attribute information (resolution, bit rate, frame rate, whether or not HDR is supported) of the stream information file 221, audio attribute information (number of channels, sampling rate), and copyright protection. Necessity information of encryption, information such as MPEG-TS method or MPEG-PS (Program Stream) method, and multiplexing method information such as a packet identifier for separating video and audio are recorded.

In the "access point management table 320", information necessary for cue reproduction is recorded. For example, when the video data is encoded by a video stream of MPEG-2, the head of GOP (Group of Picture) corresponds to the access point. In the present embodiment, which packet of the stream information file 221 contains the head of the GOP is recorded as information necessary for the cue reproduction.

The “num_of_entry 321” of the “access point management table 320” indicates the total number of access points, and the loop statement (for or less) in the line immediately below the “num_of_entry 321” is repeated by the number of “num_of_entry 321”.

“PTS_GOP_Start 331” indicates the display start PTS of the access point, and “SPN_GOP_Start 332” is the number of relative packets from the head of the stream information file 221 to the head of the access point. Since the packet has a fixed length, the relative number of bytes from the beginning of the stream information file 221 to the beginning of the access point can be calculated by multiplying the value of “SPN_GOP_Start 332” by the number of bytes per packet. The encoded video reproducing device 100 determines the position of the access point on the disc from the number of bytes from the beginning of the stream information file 221, and performs cue reproduction.

<Picture rearrangement>
FIG. 4 is a schematic diagram illustrating the relationship between the storage order array and the display order array of the encoded compressed video file. The stream information file 221 is recorded in the order of accumulated frames as shown in the accumulation order array 401. However, in the present embodiment, the static analysis unit 152 performs the data analysis of the stream information file 221 before starting the reproduction. The display order array 402 is rearranged.

That is, the GOP, which is a coding compression unit, includes a data-compressed I picture (I) and a data-compressed P picture (P) that is motion-compensated by an I picture or P picture that is temporally forward. ) And a B picture (B) which is data-compressed by adding motion compensation by I and P pictures in the front and rear direction in terms of time.

In the storage order array 401, the I picture is located at the head of the GOP, the B picture is next in succession, then the P picture, and then the B picture is repeated in the order of B, B, and P.

Normally, the display start time of the stream information file 221 is specified by Start_PTS301 (abbreviated as S_PTS301), and the display end time is specified by End_PTS302 (abbreviated as E_PTS302). In the storage order array 401 of FIG. Is the S_PTS 301, and the last P picture is the E_PTS 302. However, the B picture that has become S_PTS301 in FIG. 4 cannot necessarily completely decode the frame image. For example, when the B picture is obtained by an open GOP, there is no picture to refer to to obtain the B picture, so that the frame image cannot be completely decoded and block noise occurs. There's a problem.

Therefore, in the present embodiment, before the reproduction is started, the static analysis unit 152 statically analyzes the data of the stream information file 221, so that the picture that can completely decode the frame image is set as the start point and the end point. Are rearranged to obtain a display order array 402. In the display order array 402 shown in FIG. 4, the head of the GOP is a B picture, the B picture is further followed by an I picture, and the I picture is set as S_VALID_PTS411 which is a start point, and the last of the GOP is set. The P picture is set as the end point E_VALID_PTS 412.

The I picture is completely decodable because it is data-compressed in the frame, and the last P picture is completely decodable because it is data-compressed with motion compensation by the P picture that is forward in time. ..

Further, the static analysis unit 152 detects the video luminance information of the video frames after S_VALID_PTS411 and the audio output level after the display start time in the GOP including the video frame of S_VALID_PTS411. Similarly, in the GOP including the video frame of E_VALID_PTS412, the video brightness information of the video frames up to E_VALID_PTS412 and the audio output level until the display end time are detected.

It should be noted that although an example in which information is held in one coding compression unit is shown in the present embodiment, for example, a configuration may be used in which information is held in a plurality of coding compression units, or one coding compression unit. The information may be stored with a smaller number of frames.

The S_VALID_PTS 411, E_VALID_PTS 412, video luminance information, and audio output level information obtained by the static analysis unit 152 are fed back to the playlist information 160 in the memory 120.

<Syntax of playlist information>
FIG. 5 is a diagram illustrating the syntax in the playlist information 160. “Num_of_PlayList501” holds the total number of playlists that are units for signage display, and “PlayList_Attribute502” holds playlist attribute information such as total playlist time, playlist name, and recording date and time.

The loop statement (below "for") in the line immediately below "PlayList_Attribute502" is repeated for the number of "num_of_PlayList501".

“Num_of_MovieFile 503” indicates the total number of stream information files 221 forming the playlist, and the loop statement (for or less) in the line immediately below “num_of_MovieFile 503” is repeated by the number of “num_of_MovieFile 503 ”.

“Stream_name 504” indicates the file name of the stream information file 221 to be played back, and “S_VALID_PTS 411” and “E_VALID_PTS 412” respectively indicate the playback start time and playback end time of a predetermined section of the stream information file 221 to be played back. .. In the present embodiment, PTSs of pictures that can completely decode a frame image are “S_VALID_PTS411” and “E_VALID_PTS412”. From these pieces of information, the encoded video reproducing device 100 can determine which section of the stream information file 221 should be reproduced.

The information of "stream_name 504", "S_VALID_PTS 411" and "E_VALID_PTS 412" is referred to as reproduction section information.

The image brightness information 510 and the audio level information 511 are information on the image brightness and the audio output level near the start point and the end point of the stream information file 221 analyzed by the static analysis unit 152 as described above.

The analysis date/time information 512 is information on the date/time when the stream information file 221 was analyzed by the static analysis unit 152, and is recorded in association with the file identification information that identifies the stream information file 221 to be analyzed.

The static analysis unit 152 analyzes the stream information file 221 prior to the video file reproduction process in order to obtain “S_VALID_PTS411”, “E_VALID_PTS412”, the video luminance information 510 and the audio level information 511. I need time. Therefore, by recording the analysis date/time information 512 and the file identification information in association with each other, if there is no change in the stream information file 221, that is, if there is no change from the previously analyzed result, no analysis processing is performed. To do so. In digital signage, a form in which encoded video is displayed by loop reproduction is widely used. Therefore, the analysis date/time information 512 is rarely changed after analysis once, and it is possible to avoid repeating analysis. , It is possible to avoid the delay of the decoding process due to the analysis.

Note that the file identification information is information that associates a file update date/time, a file size, a file name, and the like, and whether or not the stream information file 221 has changed is determined by whether or not the file identification information has been changed.

The "S_VALID_PTS411", "E_VALID_PTS412", the video brightness information 510, the audio level information 511, and the analysis date/time information 512 extracted by the static analysis unit 152 are referred to as static analysis information.

<Saving to AV buffer>
FIG. 6 is a diagram illustrating a storage mode in which the encoded video information and the encoded audio information separated by the stream separation unit 111 are stored in the AV buffer 112. As shown in FIG. 6, a plurality of stream information files 221 are recorded on the SD card 140. The stream information file 221 may include files created in various multiplexing formats such as an MTS file created in the MPEG-TS format, an MP4 file created in the MP4 format, and a WMV file created in the WMV format.

The stream information file 221 that is a multiplexed file is separated into encoded video information and encoded audio information by the stream separation unit 111, and is stored in the AV buffer 112 in access unit units as shown in FIG. ..

The AV buffer 112 includes a video buffer 601 and an audio buffer 602. Each buffer has a plurality of buffer areas for continuously supplying data, and information is stored in the buffer area in access unit units. Will be recorded. This information recording area is called an access unit information area.

In this embodiment, it is assumed that the video buffer 601 has 30 areas and the audio buffer 602 has 10 areas.

When the frame rate of video is 30 fps (frames per second), 1/30 second×30=1 second of video data can be stored. As for voice, if one access unit has 0.1 second, it is possible to store 0.1 second×10=1 second of voice data.

In the access unit information area of the video buffer 601, the encoded video information 610 and the video display start time 611 are recorded in association with each other. Similarly, in the access unit information area of the audio buffer 602, the encoded audio information 620 and the audio display start time 621 are recorded in association with each other.

Generally, in the multiplexed stream information file 221, the display start time is recorded for each access unit. The display start time is set to the video display start time 611 and the audio display start time 621, and the video/audio decoder 114 reads the encoded video information 610 and the encoded audio information from the access unit information area of the AV buffer 112 according to the time information. 620 is read and a decoding process is performed. Such control data flow is also used in OpenMAX, which is a general-purpose multimedia control specification for embedded devices.

<Adjustment of video brightness and audio output level>
FIG. 7 is a diagram showing the timing of image display and data supply, and the horizontal axis is the time axis. Here, a case will be described in which, as the playlist, the MTS file “00111.mts” is reproduced twice and then the MP4 file “99999.mp4” is reproduced.

First, the data supply control unit 151 reads the “00111.mts” file from the stream information file 221 of the SD card 140, and starts the supply to the stream separation unit 111 at time t1. After being separated into the encoded video information and the encoded audio information by the stream separation unit 111, the stream separation unit 111 extracts the information for each access unit of the encoded video information and the encoded audio information, and the access unit is stored in the AV buffer 112. Supply in units. At that time, a display time (time stamp) for each access unit recorded in the stream information file 221 is added to the set video display start time 611 and audio display start time 621.

After that, the access unit information recorded in the AV buffer 112 is sequentially supplied to the video/audio decoder 114, and the video/audio decoder 114 decodes the encoded information and starts the video display at time t2.

The data supply control unit 151 continues the data supply until the end of the file of the “00111.mts” file, and when all the data supply of the stream information file 221 read earlier is completed, immediately reads the subsequent stream information file 221 and sets the time. At t3, the supply to the stream separation unit 111 is started again. At this point, the reproduction (video display) of the preceding stream information file continues, but the video/audio decoder 114 starts the reproduction of the subsequent stream information file at time t4, and the video display is switched. The "99999.mp4" file also starts to be supplied to the stream separation unit 111 at time t5, and starts to be reproduced at time t6.

FIG. 8 is a diagram for explaining the adjustment of the video luminance and the audio output level when the stream information file is reproduced based on the timing chart as shown in FIG. FIG. 8 shows a timing chart of video display, video luminance information 510 and audio level information 511 obtained by analysis in the static analysis unit 152, and the horizontal axis is the time axis.

The output video/audio control unit 153 reads the video luminance information 510 and the audio level information 511 from the playlist information 160 recorded in the memory 120. As shown in FIG. 8, at the time T3 when the video display by the preceding “00111.mts” file is switched to the video display by the subsequent “00111.mts” file, there is a large difference in the video luminance and the audio output level. To do.

The output video/audio control unit 153 controls the video/audio adjustment unit 115 in the video reproduction section so as to reduce the difference when the difference is equal to or more than a predetermined set value set in advance. Specifically, regarding the video luminance, in the case of the example of FIG. 8, the output luminance is decreased in the section from time T2 to T3 until the start point of the subsequent stream information file 221, that is, the luminance at time T3. The video/audio adjustment unit 115 is controlled so as to perform the above operation. As a result, at time T3, the difference in image brightness is eliminated, so that it is possible to perform switching of image scenes that does not make the user feel uncomfortable.

Similarly, regarding the audio output level, the audio output level at the start point of the subsequent stream information file 221, that is, the audio output level at time T3 is adjusted so as to match the ending point of the preceding stream information file, that is, the audio output level at time T3. Lower the output level. After that, at time T4, the video/audio adjusting unit 115 is controlled so as to gradually increase the audio output level so that the original audio output level of the subsequent stream information file 221 is reached. As a result, at time T3, the difference in the audio output levels is eliminated, so that it is possible to switch the video scenes without causing the user to feel uncomfortable.

On the other hand, as shown in FIG. 8, at time T6 when the video display by the second “00111.mts” file is switched to the video display by the subsequent “99999.mp4” file, the video brightness and the audio output level are changed. , There is no big difference. The output video/audio control unit 153 determines that the adjustment is unnecessary and determines that the difference is less than a predetermined set value, and does not perform special control.

Note that the section from time T2 to T4 and the section from time T5 to T7 are the reproduction start time and reproduction end time of the stream information file in the context of being analyzed by the static analysis unit 152, that is, the vicinity of the connection point of the stream information file. Section, which corresponds to, for example, a 0.5 second section of the encoding compression unit. In FIG. 8, the video brightness and the audio output level for the entire period of the “00111.mts” file and the “99999.mp4” file are shown for convenience, but the analysis by the static analysis unit 152 is performed at the above-mentioned time. It suffices to perform the section from T2 to T4 and the section from time T5 to T7.

Further, in the above, whether or not the adjustment is necessary is determined by using the difference between the video brightness and the instantaneous value of the audio output level at the access unit at time T3. However, for example, the section between time T2 and T3, that is, at time T3 The average value of the video brightness and the audio output level at the end portion of the preceding stream information file including the last access unit of the preceding stream information file and the plurality of preceding access units before it, and the subsequent stream at time T3 Whether the adjustment is necessary or not is made by using the difference between the average value of the video luminance and the audio output level at the start portion of the subsequent stream information file including the first access unit of the information file and a plurality of subsequent access units after that. You may judge.

<Entire operation of coded video playback device>
Next, referring to FIG. 1, the overall operation of the coded video reproduction device 100 will be described using the flowchart shown in FIG. As shown in FIG. 9, the encoded video playback device 100 starts its operation by receiving a playback instruction based on a playlist via the operation unit 130. First, the system control unit 101 streams from the SD card 140. The reproduction driver 102 is controlled to read the information file 221.

The stream information file 221 is input to the stream control unit 110, and is separated by the stream separation unit 111 into encoded video information and encoded audio information. After that, the stream separation unit 111 extracts information for each access unit of encoded video information and encoded audio information, and stores the information in the AV buffer 112 in access unit units. This is the video file reproduction process of step S103, but the static analysis unit 152 executes the static analysis process before saving the access unit information in the AV buffer 112 (step S101). The video file reproduction process will be further described later.

The static analysis unit 152 analyzes the data of the stream information file 221, and rearranges the frame images so that the pictures that can be decoded are the start point and the end point, and the data supply control unit 151 rearranges them. The data are stored in the AV buffer 112 in units of access units in the order in which they are stored. In addition, video luminance analysis and audio level analysis are performed on the encoded video information and the encoded audio information, respectively. The information obtained by the static analysis process is fed back to the playlist information 160 in the memory 120. The static analysis process will be further described later.

The output video/audio control unit 153, based on the video brightness information 510 and the audio level information 511 obtained from the playlist information 160 recorded in the memory 120, the video in the connection point of the stream information file 221 and the section in the vicinity thereof. It is determined whether or not the difference in brightness and the difference in audio output level are equal to or greater than a predetermined set value. If it is equal to or more than the set value, the filter setting process for filtering the output signal is performed by controlling the video/audio adjustment unit 115 for the section (step S102).

The filtering of this output signal is the adjustment processing by the video/audio adjustment unit 115 described with reference to FIG. 8, and the processing of changing the setting of the video/audio adjustment unit 115 for this adjustment processing is the filter setting processing. In the static analysis processing, the analysis corresponding to the preceding stream information file 221 is completed, and the separation processing for the subsequent stream information file 221 is started by the stream separation unit 111, whereby the subsequent stream information file 221 is analyzed. Therefore, the connection point of the preceding stream information file 221 and the succeeding stream information file 221 and the section in the vicinity thereof have information on video luminance and audio output level. Therefore, as described above, it is possible to determine the difference in the video luminance and the difference in the audio output level in the section.

Next, the system control unit 101 sequentially supplies the access unit information stored in the AV buffer 112 for each access unit in step S103 to the video/audio decoder 114 by the procedure described with reference to FIG. The control unit 151 determines, as at times t3 and t5 in FIG. 7, whether or not all data supply of the read stream information file to the audiovisual decoder 114 has been completed (step S104).

When it is determined in step S104 that the supply of all the data of the read stream information file is completed (YES), it is determined whether or not there is a stream information file to be reproduced next to the stream information file based on the playlist. (Step S105). On the other hand, if all the data supply has not been completed (NO), the process of step S103 is continued.

In step S105, when it is determined that the next playlist exists (YES), the process proceeds to step S106, and when the next playlist does not exist (NO), the reproduction based on the playlist ends.

The data supply control unit 151 records the PTS (E_VALID_PTS412) indicating the end of the stream information file that was played back in advance in the AV buffer 112 as the offset time of the time stamp (step S106). Then, the supply of the next stream information file is started (step S107), and the processing from step S103 is repeated. The offset time is used to shift the display start time of the next stream information file.

The data supply control unit 151 records the video display start time 611 and the audio display start time 621 set for each access unit in the AV buffer 112 before supplying the next stream information file to the video/audio decoder 114. Immediately after changing to the offset time, the supply of the next stream information file is started. After that, the video display start time 611 and the audio display start time 621 are sequentially changed for each access unit in accordance with the read time interval of the access unit of the supplied stream information file.

As described above, the PTS indicating the file end of the stream information file that was reproduced in advance is set as the offset time of the time stamp, and the offset time is set as the video and audio display start time in the next stream information file. , It becomes possible to continuously reproduce the video scene without interruption.

<Static analysis processing>
Next, the static analysis process will be further described with reference to the flowchart showing the operation of the static analysis process shown in FIG.

In the static analysis process, the static analysis unit 152 first reads the analysis date/time information 512 (FIG. 5) from the playlist information 160 in the memory 120, and determines whether or not the playlist information 160 has valid static analysis information. Is determined (step S201).

This determination is made based on whether or not the analysis date/time information 512 of the stream information file to be reproduced has been changed, and whether or not the file update date/time, file size and file name included in the analysis date/time information 512 have been changed. If the analysis date/time information 512 has not been changed and there is valid static analysis information (YES), the processing is ended without performing the following analysis processing. .. Whether or not the file update date/time, file size, and file name have been updated can be confirmed by the file property information.

On the other hand, when the analysis date/time information 512 is changed, it is determined that there is no valid static analysis information (NO), and the process proceeds to step S202 to extract the static analysis information from the stream information file 221.

In step S202, video attribute information is acquired. As the video attribute information, for example, an MPEG sequence header is extracted, and the resolution, frame rate, and HDR (High Dynamic Range) compatibility are acquired from the header information. HDR is a standard having brightness information in a wider range than the brightness information of normal video, and whether or not HDR is supported is information indicating whether or not this standard is supported.

Also, the bit rate information is calculated from the start point/end point of the PES (Packetized Elementary Stream) header of the stream information file 221 and the file size. The PES header is a header for each access unit, holds PTS (Presentation Time Stamp) and DTS (Decoding Time Stamp) as internal information, and sets the file size to PTS difference (end PTS-start PTS=playback time). The bit rate can be calculated by dividing by.

After acquiring the video attribute information, the audio attribute information is acquired (step S203). As the audio attribute information, the number of channels and sampling rate are extracted from the audio header information.

Next, in step S204, the video start point is detected, and in step S205, the video end point is detected. Specifically, as described with reference to FIG. 4, in the stream information file 221 recorded in the order of accumulated frames, the picture S_PTS301 whose reference relationship has not been established is not used as S_VALID_PTS411, but the I picture is used as S_VALID_PTS411. .. Further, since the E_PTS 302 has established the reference relationship, the pictures, here, the access units are rearranged so as to be used as the E_VALID_PTS 412.

Since S_PT301 and E_PTS302 can be extracted from the PES (Packetized Elementary Stream) header of the stream information file 221, and the reference relationship can be known from the picture type (I, P, B), the start point (S_VALID_PTS) at which the video frame can be completely decoded. ) And the end point (E_VALID_PTS) can be set.

Next, in step S206, the image brightness is analyzed. For video luminance analysis, software decoding processing is performed on the video coding compression unit near the start point including S_VALID_PTS411, a luminance value histogram is generated for each video frame, and a video luminance average value and a video luminance maximum value are extracted. To do. Similarly, the software decoding process is also performed on the video coding compression unit near the end point including the E_VALID_PTS 412 to extract the video brightness average value and the video brightness maximum value.

When analyzing the image brightness, the image data is sampled and the value can be quantized to obtain the brightness value for each pixel. By expressing the brightness value on the horizontal axis and the number of pixels having each brightness value on the vertical axis, A luminance value histogram can be obtained. By using the brightness value histogram, it is possible to know the distribution of the brightness values of the pixels in the image, perform image processing such as brightness and color correction, and emphasize contrast, and also obtain the average brightness and the maximum brightness.

Next, in step S207, the sound level is analyzed. In the analysis of the audio level, the software decoding process of the audio encoding compression unit is performed on the section of the video encoding compression unit near the start point including S_VALID_PTS411 to extract the audio output level. Similarly, the software decoding process is also performed on the video coding compression unit near the end point including the E_VALID_PTS 412 to extract the audio output level. When the encoded and compressed voice information is decoded, the voice information has the output voice level, and the voice output level can be extracted.

Note that the vicinity of the start point including S_VALID_PTS 411 and the vicinity of the end point including E_VALID_PTS 412 in steps S206 and S207 described above are, for example, a 0.5 second video encoding compression unit centered on the start point and the end point, Corresponds to the section from time T2 to T4 and the section from time T5 to T7.

Finally, in step S208, the analysis date/time information is stored in the playlist information 160 of the memory 120. In the analysis date/time information, file identification information such as date/time information when the static analysis information is acquired, update date/time of the statically analyzed file, file size, file name, and SD card type information is recorded.

<File playback process>
Next, the file reproducing process will be further described with reference to the flowchart showing the operation of the file reproducing process shown in FIG.

In the file reproduction process, first, the reproduction driver 102 reads a stream information file to be reproduced from the SD card 140 based on an instruction from the data supply control unit 151 (step S301). The file read unit is a predetermined file size, and in the present embodiment, it is read every 1 MB (megabyte).

The stream information file read by the reproduction driver 102 is input to the stream separation unit 111 and separated into encoded video information and encoded audio information (step S302).

The separated encoded video information and encoded audio information are divided into access unit units, and the display start time (video display start time 611 and audio display start time 621) of each access unit is extracted and used as a time stamp. It is given to the access unit and written in the AV buffer 112 (step S303).

Thereafter, the coded video information and the coded audio information are supplied to the video/audio decoder 114 for each access unit and decoded by the video/audio decoder 114 (step S304).

After that, it is determined whether or not the data read for each 1 MB includes the file end of the stream information file (step S305), and if the file end is not included (NO), the process proceeds to step S301. , Read the next 1 MB of data. On the other hand, when the end of the file is detected (YES), it is determined that all the stream information files have been supplied to the video/audio decoder 114, and the file reproducing process is ended.

As described above, in the coded video reproducing apparatus 100 according to the present invention, as described with reference to the flowchart shown in FIG. 9, the PTS indicating the file end of the stream information file that was reproduced first is timed. By using the offset time of the stamp and the offset time in the next stream information file as the display start time of video and audio, the initialization processing in the video/audio decoder 114 can be omitted, and the time spent for the initialization processing is unnecessary. Therefore, it is possible to continuously reproduce the video scene without interruption. Therefore, it is possible to eliminate the gap caused by the transition of the video stream. The initialization processing includes presetting of the resolution and frame rate of the video supplied to the video/audio decoder 114, initialization of the buffer of the video/audio decoder 114, and the like.

Further, since it is not necessary to add special hardware to the encoded video reproduction device 100 in order to have the above-described functions, it is possible to suppress the cost increase of the device itself.

In addition, the static analysis unit 152 executes the static analysis process before executing the video file reproduction process of step S103, and streams the frame image so that the start point and the end point are pictures that can be completely decoded. Since the pictures of the information file 221 are rearranged and stored in the AV buffer 112 in access unit units according to the rearranged order, it is possible to reliably exclude video frames that cannot be completely decoded, and at the time of video stream transition, Video can be displayed without causing block noise.

In addition, the output video/audio control unit 153 controls the video/audio adjustment unit 115 at the seam of the video scenes based on the attribute information of the start point and the end point recorded in the playlist information 160, so that the output video/audio control unit 153 suddenly operates. Image quality and sound quality output control is performed so that rapid changes in video brightness and audio output level do not occur, so it is possible to prevent large changes in video brightness and audio output level during transition of the video stream, which makes users feel uncomfortable. You can switch the video scene without.

Although the present invention has been described in detail, the above description is illustrative in all aspects, and the present invention is not limited thereto. It is understood that innumerable variants not illustrated can be envisaged without departing from the scope of the invention.

It should be noted that the present invention can appropriately modify or omit the embodiments within the scope of the invention.

Claims (8)

A coded video playback device for playing back a stream information file in which coded video information and audio information are multiplexed,
A stream separation unit that separates the video information and the audio information from the stream information file and divides each into an access unit.
A buffer for recording the video information and the audio information divided in units of the access units in association with respective display start times,
A decoder for reading and decoding the video information and the audio information from the buffer based on the display start time;
A control of recording of the display start time to the buffer, the video information and the audio information, and a data supply control unit that controls supply of the video information and the audio information from the buffer to the decoder,
Prior to recording the video information and the audio information in the buffer by the data supply control unit, an analysis unit for performing data analysis of a first access unit of the stream information file and a final access unit is provided,
The data supply control unit,
When the supply of the video information and the audio information of the preceding stream information file is completed, the display start time of the last access unit of the preceding stream information file is set before the supply of the video information and the audio information of the succeeding stream information file. , Reset to the display start time of the first access unit of the subsequent stream information file ,
The analysis unit rearranges the access units so that the access units capable of completely decoding the frame image are the first access unit and the last access unit, and the data supply control unit rearranges them. A coded video reproducing device for recording in the buffer in the order of A video/audio adjusting unit that receives the video information and the audio information decoded by the decoder and adjusts the video brightness and the output volume level of each is provided.
The analysis unit is
Analyzing video luminance and audio output level of a start portion including the first access unit and an end portion including the final access unit of the stream information file;
The video and audio adjustment unit,
If the difference in the video luminance and the difference in the audio output level between the end portion of the preceding stream information file and the start portion of the subsequent stream information file are larger than the predetermined difference, the predetermined The coded video reproduction device according to claim 1, wherein the video brightness and the audio output level are reduced so that the difference is within the difference. The video and audio adjustment unit,
The difference in the video luminance and the difference in the audio output level between the final access unit of the preceding stream information file and the first access unit of the subsequent stream information file are larger than the predetermined difference. In this case, the coded video reproduction device according to claim 2, wherein the video brightness and the audio output level are reduced so that the difference is within the predetermined difference. The end portion of the preceding stream information file includes the final access unit and a plurality of preceding access units before it.
The start portion of the subsequent stream information file includes the first access unit and a plurality of subsequent access units after the first access unit,
The video and audio adjustment unit,
An average value of each of the video luminance and the audio output level in the final access unit and the plurality of preceding access units,
If the difference between the video access luminance and the average value of the audio output levels in the first access unit and the plurality of subsequent access units is larger than the predetermined difference, the difference is within the predetermined difference. The coded video reproduction device according to claim 3, wherein the video brightness and the audio output level are reduced as described above. A coded video reproduction method for reproducing a stream information file in which coded video information and audio information are multiplexed,
(A) separating the video information and the audio information from the stream information file, and (b) dividing the video information and the audio information into access unit units, and dividing the video information and the audio information. Recording information in a buffer in association with each display start time,
(C) reading the video information and the audio information from the buffer and supplying them to a decoder;
(D) decoding the video information and the audio information on the basis of the display start time in the decoder,
After step (d),
The display start time of the last access unit of the preceding stream information file is displayed before the supply of the video information and audio information of the following stream information file to the decoder. for example Bei the step of re-set to the start time,
Prior to the step (b),
(E) a step of performing data analysis of the first access unit and the last access unit of the stream information file,
The step (e) includes
Reordering the access units so that the access units capable of completely decoding the frame image are the first access unit and the final access unit,
The step (b) includes
A coded video reproducing method, wherein recording is performed in the buffer in the order rearranged in step (e). After step (d),
(F) a step of receiving video information and audio information decoded by the decoder and adjusting video luminance and output volume level of each,
The step (e) includes
Analyzing the video brightness and audio output level of a start portion including the first access unit and an end portion including the final access unit of the stream information file,
The step (f) includes
If the difference in the video luminance and the difference in the audio output level between the end portion of the preceding stream information file and the start portion of the subsequent stream information file are larger than the predetermined difference, the predetermined 6. The encoded video reproducing method according to claim 5 , further comprising the step of reducing the video luminance and the audio output level so that the difference is within the above difference. The step (f) includes
The difference in the video luminance and the difference in the audio output level between the final access unit of the preceding stream information file and the first access unit of the subsequent stream information file are larger than the predetermined difference. In this case, the coded video reproducing method according to claim 6, further comprising the step of reducing the video luminance and the audio output level so that the difference is within the predetermined difference. The end portion of the preceding stream information file includes the final access unit and a plurality of preceding access units before it.
The start portion of the subsequent stream information file includes the first access unit and a plurality of subsequent access units after the first access unit,
The step (f) includes
An average value of each of the video luminance and the audio output level in the final access unit and the plurality of preceding access units,
If the difference between the video access luminance and the average value of the audio output levels in the first access unit and the plurality of subsequent access units is larger than the predetermined difference, the difference is within the predetermined difference. 8. The encoded video reproduction method according to claim 7, further comprising the step of reducing the video brightness and the audio output level.

Images