JP2019125994A - Video and audio reproduction device - Google Patents

Abstract

To synchronize video data and audio data.SOLUTION: Video data and audio data are respectively compressed by a compression unit and input to a memory, and each is delayed by the memory. The delayed video data and audio data are output from the memory and then expanded by an expansion unit. Further, the control unit determines a delay time in the memory of each of the video data and the audio data on the basis of format information of the video data and the audio data. By adjusting the delay time, the video data and the audio data are synchronized.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a video and audio reproduction apparatus, and more particularly to a video and audio reproduction apparatus for synchronizing video and audio at the time of reproduction.

  In an apparatus for reproducing video and audio represented by a television receiver or the like, signal processing of video data and signal processing of audio data may be separately performed. In such a case, the time taken for signal processing of video data and the time taken for signal processing of audio data do not necessarily coincide. Generally, video data has a larger amount of data than audio data, so that the time taken for signal processing of video data is longer than the time taken for signal processing of audio data. Therefore, the reproduction timings of the video and the audio of the television receiver do not match, and as a result, there arises a problem that the content of the video and the audio are deviated.

  To address this problem, a technique for matching the video and audio reproduction timing of the television receiver (synchronizing the video and audio reproduction timing) is called lip sync.

  There is a technology disclosed in Patent Document 1 as a technology for obtaining such lip synchronization.

  Patent No. 5590186

  The technique disclosed in Patent Document 1 includes a memory in each of a system of signal processing of video data and a system of signal processing of audio data, and controls timing of writing / reading data to the memory to control video and audio. The synchronization of the playback timing of

  In the technology disclosed in Patent Document 1, the capacity of the memory determines the amount by which the deviation between the video and the audio can be corrected. If the video data and the audio data deviate by more than the memory capacity, there is a problem that the reproduction timing of the video and the audio can not be synchronized.

  In order to solve the above-mentioned subject, the present invention takes the following composition.

  That is, a video and audio reproduction apparatus that outputs video data and audio data at the same time and outputs video data and / or format information of the input video data, and video based on the format information. A control unit that specifies processing time of data and / or audio data and detects a time difference between video data and audio data, and video data and / or audio data based on the time difference between video data and audio data detected by the control unit And a memory for delaying.

  Here, the control unit may specify processing time of video data and / or audio data based on information of processing of video data and / or audio data in addition to the format information.

Further, a video and audio reproduction apparatus according to the present invention comprises a compression unit that compresses the video data and / or audio data, and a decompression unit that decompresses the compressed video data and / or audio data. The compression unit compresses video data and / or audio data before input to the memory, and the decompression unit decompresses the video data and / or audio data output from the memory.

  Here, the compression unit reduces the number of pixels, the frame rate, or the number of bits of the video data.

  Alternatively, the compression unit reduces the sampling rate or the number of bits of the audio data.

  According to the present invention, even with a small memory capacity, it is possible to correct a large deviation between video data and audio data and to take a lip sync.

1 is a block diagram of a video and audio reproduction apparatus according to the present invention. It is an example of a table of processing time by the method of format information and processing. It is an example of a table of the time which compression and decompression take. FIG. 7 is another block diagram of the video and audio reproduction apparatus according to the present invention.

Hereinafter, preferred embodiments of the video and audio reproduction apparatus of the present invention will be described with reference to the drawings. In the following description, components denoted by the same reference numerals in different drawings may be omitted as they are the same. The present invention is not limited to the examples in these embodiments, but includes all modifications within the scope and equivalents of the matters described in the claims.
First Embodiment
FIG. 1 is a block diagram of a video and audio reproduction apparatus according to the present invention.
In FIG. 1, reference numeral 1 denotes a video and audio reproduction apparatus. Specifically, a television receiver etc. correspond. 2 is a source of video and audio. Specifically, a DVD player or the like corresponds. Reference numeral 101 denotes a video processing unit, 102 denotes a video compression unit, 103 denotes a video memory, 104 denotes a video decompression unit, 105 denotes a display unit, and 106 denotes an audio processing unit. , 107 is an audio compression unit, 108 is an audio memory, 109 is an audio expansion unit, 110 is a speaker, and 111 is a control unit.

  Video data output from a DVD player or the like, which is a source source, is input to the video processing unit 101. Similarly, audio data output from a DVD player or the like as a source source is input to the audio processing unit 106.

The video processing unit 101 extracts format information of the input video data, and the video format includes, for example, 8K, 4K, 2K, and the like. The 8K video format is, for example, 7680 × 4320 pixels, 60 Hz frame rate progressive, 10 luminance bits, and 10 chrominance bits. The 4K video format is, for example, 3840 × 2160 pixels, progressive 60 Hz frame rate, 10 luminance bits, and 10 chrominance bits. The 2K video format is, for example, 1920 × 1080 pixels, 60 Hz frame rate interlace, 8 luminance bits, and 8 chrominance bits.
Information on the format of input video data extracted by the video processing unit 101 is input to the control unit 111.

Similarly, the audio processing unit 106 extracts format information of the input audio data. Examples of the audio format include AAC, ALS, and the like.
Information on the format of the input speech data extracted by the speech processing unit 106 is input to the control unit 111.

The control unit 111 has a table of format information and processing time according to the processing method. Here, in the case of video data, the processing method is referred to as “standard”, “theater”, “dynamic”, “custom”, etc., and the luminance level, contrast, color tone, etc. are adjusted respectively to It is a process of processing into a video signal that reflects the content (genre) and the preference of the user.
Similarly, in the case of audio data, it is referred to as "standard", "theater", "music", "auto", etc., and the frequency characteristics, sound pressure, etc. are adjusted respectively, and the type of music (genre), user Processing to an audio signal reflecting the preference of

The table as described above is represented, for example, as shown in FIG. FIG. 2A is an example of a table of format information of video data and a processing time according to a method of processing, and FIG. 2B is an example of a table of format information of audio data and a processing time according to a method of processing is there. Processing time is defined for each combination of format information and processing method.
From this table, the video processing unit 101 and the audio processing unit 106 can grasp the time taken to process the video data and the audio data. Thus, the control unit 111 can grasp the time difference between the video data and the audio data output from the video processing unit 101 and the audio processing unit 106.

  The video data output from the video processing unit 101 is then input to the video compression unit 102. The video compression unit 102 performs processing to reduce the data amount of video data. The reduction of the data amount of video data means, for example, conversion of 8K format video to 4K format video, similarly, conversion of 8K format video to 2K format video, and similarly 2K video of 4K format It can be considered to convert to a format video. Specifically, the number of pixels is reduced, the frame rate is reduced, or the number of bits is reduced.

  Similarly, the audio data output from the audio processing unit 106 is input to the audio compression unit 107 next. The audio compression unit 107 performs processing to reduce the amount of audio data. The reduction of the data amount of audio data may be, for example, lowering the sampling rate or reducing the number of bits.

  The control unit 111 has a table of time taken to reduce the data amount of each of the video data and the audio data. The table of the time taken to reduce the data amount of video data is, for example, as shown in FIG.

  The video data for which the data amount has been reduced by the video compression unit 102 is input to the video memory 103. The video memory 103 delays video data for a predetermined time by controlling the timing of writing and reading data to the memory.

Similarly, the audio data whose amount of data has been reduced by the audio compression unit 107 is input to the audio memory 108. The voice memory 108 delays voice data by a predetermined time by controlling the timing of writing and reading data to the memory.
Thus, by reducing the data amount of video data and audio data before input to the memory, it is possible to lengthen the time that can be delayed in the memory.
The method of determining the predetermined time will be described later.

Video data delayed for a predetermined time by the video memory 103 is input to the video decompression unit 104. The video decompression unit 104 performs processing to restore the amount of data reduced by the video compression unit 102. For example, when the video compression unit 102 reduces the 8K format video to the 4K format video data amount, the 4K format video whose amount is reduced is returned to the 8K format video data amount. Similarly, when the 8K format video is reduced to the 2K format video data amount, the data amount reduced 2K format video is returned to the 8K format video data amount. Similarly, when the 4K format video is reduced to the 2K format video data amount, the data amount reduced 2K format video is returned to the 4K format video data amount.
Specifically, the number of pixels is complemented, the frame rate is increased, the number of bits is increased, and the like.

Similarly, audio data delayed for a predetermined time by the audio memory 108 is input to the audio expansion unit 109. The audio expansion unit 109 performs processing to restore the amount of data reduced by the audio compression unit 107.
For example, when the voice compression unit 107 reduces the sampling rate to reduce voice data, the sampling rate is increased to return the voice data amount. Similarly, when the audio compression unit 107 reduces the number of bits to reduce audio data, the number of bits is increased and the amount of audio data is returned.

  The control unit 111 has a table of time taken to restore the data amount of each of the video data and the audio data. The table of the time taken to restore the data amount of the video data is, for example, as shown in FIG. 3 (b).

  As described above, the control unit 111 recognizes the time required for processing of each of the video processing unit 101, the video compression unit 102, and the video decompression unit 104. Therefore, the time taken by the video processing unit 101, the video compression unit 102, and the video decompression unit 104 as a whole can also be grasped. Similarly, the control unit 111 grasps the time required for processing of each of the audio processing unit 106, the audio compression unit 107, and the audio expansion unit 109. Therefore, the time taken for the audio processing unit 106, the audio compression unit 107, and the audio expansion unit 109 as a whole can also be grasped.

  From the above, the control unit 111 takes the time taken by the video processing unit 101, the video compression unit 102, and the video decompression unit 104, and the time taken by the audio processing unit 106, the audio compression unit 107, and the audio decompression unit 109 in total. Time difference can also be grasped. By controlling the delay time (the above-mentioned predetermined time) of the video memory 103 and the audio memory 108 so as to eliminate such a time difference, it is possible to take lip sync (synchronization) between the video data and the audio data.

  According to this embodiment, since video data and audio data are each compressed (data reduction) and delayed by the memory, a large amount of delay can be obtained even when the memory capacity is small, and video data and audio data can be obtained. Even when the gap between the video data and the audio data is large, lip synchronization (synchronization) can be performed.

The video data thus lip-synced is displayed on the display unit 105, and the audio data is emitted from the speaker 110.
Second Embodiment
The first embodiment has been described as performing compression / decompression processing for both video data and audio data. However, depending on the memory capacity and the amount of deviation between video data and audio data, it may not be necessary to perform compression / decompression processing for both video data and audio data.

In the second embodiment, an embodiment in which whether to perform compression / decompression processing for both video data and audio data is controlled based on the memory capacity and the shift amount between the video data and the audio data will be described.

The control unit 111 restores the data amount of each of the video data and the audio data, the table of the processing time according to the format information and the processing method, the table of the time required to reduce the data amount of each of the video data and the audio data I have a table of time.
The time taken by each processing of the video processing unit 101, the video compression unit 102, and the video decompression unit 104 can be grasped from these tables. Similarly, the time taken for processing of each of the audio processing unit 106, the audio compression unit 107, and the audio expansion unit 109 can be grasped.

In addition to this, the control unit 111 has information of the memory capacity of the video memory 103 and the memory capacity of the audio memory 108.
Thus, when the time difference between the time taken by the video processing unit 101 and the time taken by the audio processing unit 106 is shorter than the time that can be delayed by the video memory 103 and the audio memory 108, the control unit 111 compresses the video data and audio data. When the time difference between the time taken by the video processing unit 101 and the time taken by the audio processing unit 106 is longer than the time that can be delayed by the video memory 103 and the audio memory 108 without compression / decompression processing, compression of video data and audio data Control to perform decompression processing.
Here, control may be performed to perform processing of compressing / decompressing only one of video data and audio data.
Third Embodiment
In the first embodiment and the second embodiment, the video data and the audio data are respectively delayed by the memory to take a lip sync, but it is not necessary to provide the memory for both the video data and the audio data. In some cases.
For example, when an AV amplifier is inserted between a source (for example, a DVD player) and a video and audio reproduction apparatus, the audio and video reproduction apparatus clearly delays the video data compared to the video data. You only need to have a memory. The third embodiment describes such an embodiment.

FIG. 4 is a block diagram of a video and audio reproduction apparatus according to a third embodiment of the present invention.
In FIG. 4, 3 is an AV amplifier. Video data output from a source source 2 (for example, a DVD player) is input to the video processing unit 101 of the video and audio reproduction device 1. On the other hand, audio data output from the source 2 (for example, a DVD player) is input to the AV amplifier 3. The audio data output from the AV amplifier 3 is input to the audio processing unit 106 of the video and audio reproduction device 1.

Here, since the audio data is delayed from the video data by passing through the AV amplifier 3, in the video and audio reproduction device 1, only the video data is delayed by the memory 103.
Audio data is emitted from the speaker 110 without delay in the memory.
Thus, it is possible to take lip sync while reducing the memory of the audio data system.

1: Video / audio reproduction device 2: Source source 3: AV amplifier 101: Video processing unit 102: Video compression unit 103: Video memory 104: Video expansion unit 105: Display unit 106: Audio processing unit , 107: audio compression unit, 108: audio memory, 109: audio compression unit, 110: speaker, 111: control unit

Claims (5)

A video / audio reproduction apparatus that synchronizes and outputs video data and audio data,
A video processing unit that extracts format information of input video data and / or audio data;
A control unit that specifies processing time of video data and / or audio data based on the format information and detects a time difference between the video data and the audio data;
A memory for delaying video data and / or audio data based on a time difference between the video data and audio data detected by the control unit;
An audio / video reproduction apparatus characterized by comprising: The video and audio reproduction apparatus according to claim 1, wherein
The video / audio reproduction apparatus, wherein the control unit specifies processing time of video data and / or audio data based on information of processing of the video data and / or audio data in addition to the format information. The video and audio reproduction apparatus according to claim 1 or 2, wherein
A compression unit that compresses the video data and / or audio data;
An expansion unit for expanding the compressed video data and / or audio data;
And the compression unit compresses video data and / or audio data before input to the memory, and the decompression unit decompresses video data and / or audio data output from the memory.
A video and audio reproduction device characterized by The video and audio reproduction apparatus according to claim 3, wherein
The video and audio reproduction apparatus, wherein the compression unit reduces the number of pixels, the frame rate, or the number of bits of the video data. The video and audio reproduction apparatus according to claim 3, wherein
The video and audio reproduction apparatus, wherein the compression unit reduces the sampling rate or the number of bits of the audio data.

Images