JPH11112937A - Video-audio synchronization method and video audio signal recording and reproducing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To attain simply a synchronization of a video and an audio by controlling delay time of the video within one frame, calculating the time information of the video and audio, multiplying this time information together with a video and an audio signals, separating the video and audio signal and the time information from a multiplex signal recorded or transmitted and synchronizing the video and the audio, on the basis of the separated time information. SOLUTION: That a delay time of a video signal from an input of a frame synchronizer 14 to an output ranges 0 to 1 frame means that a delay time between an audio signal inputted and the video signal outputted from the frame synchronizer 14 also ranges 0 to 1 frame. Before multiplexing time audio information A-PTS by a multiplexer MUX 19, time dVFR-int for one frame of an internal reference synchronism is added to the time information A-PTS of audio through an adder inside a microcomputer 11. Thus, the delay time between the audio signal and the video signal is in a ranges of -1 to 0 frame on a time stamp.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a video / audio synchronization method and a video / audio signal recording / reproducing apparatus used in the technical field of video / audio synchronization.

[0002]

2. Description of the Related Art As a method of synchronizing video and audio, for example, a so-called MPEG (Moving Picture Image Coding Exp
A method of multiplexing the time information of the synchronization timing of each video and audio signal as a time stamp, which is used in an erts group, is famous for the encoded signal. MPEG means ISO / IEC JTC1 / SC29 (In
ternational Organization for Stan-dardization / Inte
rnational Electrotechnical Commission, Joint Techn
ical Commitee 1 / Sub Commitee 29: International Organization for Standardization /
International Electrotechnical Commission Joint Technical Committee 1 / Specialized Subcommittee 2
9) This is an abbreviation of the organization for studying the video coding for storage in 9).
There are ISO11172 as the PEG1 standard and ISO13818 as the MPEG2 standard. In these international standards, in the item of system multiplexing, ISO117172-1
And ISO13818-1 are ISO117172-2 and ISO138 in the item of video encoding (MPEG video).
18-2 is an item of audio encoding (MPEG audio), and ISO11172-3 and ISO13818-3 are
Each is standardized.

[0003]

In the above-mentioned synchronization method, the time of the reference counter is obtained at the timing of the synchronization signal of the input signal. Therefore, when the variable delay element such as the frame synchronizer is narrowed down to the signal, It is difficult to synchronize the audio with the audio if the time information of the video is completely matched with the input signal, and complicated processing must be performed at the time of encoding or decoding.

[0004] However, in the future, when a consumer device equipped with an encoder such as MPEG appears, a situation where a non-standard signal is input as an external input is inevitable, and a synchronization means such as a frame synchronizer is indispensable. Should come.

Accordingly, the present invention has been made in view of such circumstances, and in the case where there is a variable delay element such as a frame synchronizer, the delay between video and audio is defined within a range in which there is no audiovisual problem. A video and audio synchronization method that easily synchronizes video and audio, and video and audio synchronization can be realized with a simple configuration.Furthermore, the information of the frame synchronizer can be used for encoding image quality and variable-speed playback during decoding. It is an object of the present invention to provide an applicable video / audio signal recording / reproducing apparatus.

[0006]

A video / audio synchronizing method according to the present invention includes a variable delay process for synchronizing an input video signal with a reference synchronization signal. Within one frame, calculate video time information from the reference synchronization timing of the video signal, calculate audio time information from the reference synchronization of the audio signal and the reference synchronization timing of the video signal, and In order to solve the above-mentioned problem, video and audio signals and time information are separated from the multiplexed signal recorded or transmitted by multiplexing with the audio signal, and the video and audio are synchronized based on the separated time information. Is what you do.

Further, the video / audio signal recording / reproducing apparatus of the present invention has variable delay means for synchronizing an input video signal with a reference synchronizing signal. Means for controlling within a frame; means for calculating video time information from the reference synchronization timing of the video signal; means for calculating time information of audio from the reference synchronization of the audio signal and the reference synchronization timing of the video signal; Means for multiplexing the time information with the video and audio signals, means for recording and reproducing the multiplexed signal on the recording medium, and separation of the video, audio signal and time information from the multiplexed signal reproduced from the recording medium. The above-mentioned problem is solved by having means for performing synchronization of a video signal and an audio signal based on separated time information.

That is, according to the video / audio synchronization method and the video / audio signal recording / reproducing apparatus of the present invention, when the delay time of the means (frame synchronizer) for performing the variable delay processing of the video arbitrarily changes from 0 to 1 frame. However, without adding a circuit to the audio system, it is possible to simply synchronize the video and the audio within a range where there is no audiovisual problem, using only the time information uniquely calculated from the reference synchronization timing of the video and the reference synchronization timing of the audio. be able to.

In the video / audio synchronizing method and the video / audio signal recording / reproducing apparatus according to the present invention, encoding is performed using compression on the time axis (difference between preceding and following frames) such as MPEG before multiplexing. In this case, the image quality is improved by detecting frame skip information or frame repetition information by means for performing a variable delay process (frame synchronizer) of video and changing coding parameters in real time based on this information. I have to.

Further, in the video / audio synchronizing method and the video / audio signal recording / reproducing apparatus of the present invention, frame skip information or frame repetition information is detected by a means (frame synchronizer) for performing a variable delay process on video, and this information is also detected. By multiplexing in the same manner as video and audio signals, the frame skip information or the frame repetition information is used as additional information during variable speed reproduction, so that the quality of slow reproduction and frame forward reproduction can be improved.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Preferred embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows the configuration of a disk recorder as an embodiment to which the video / audio synchronization method and the video / audio signal recording / reproducing apparatus of the present invention are applied.

In FIG. 1, components indicated by reference numeral 10 are various keys (Key) provided on an operation panel of the disc recorder, and an operator presses any of these various keys with a finger. And a resistor ladder circuit that outputs a voltage corresponding to the pressed key when the key is pressed (hereinafter, the configuration is referred to as a key circuit). This key circuit 1
Output data corresponding to a key press from 0 is supplied to a microcomputer (hereinafter abbreviated as a microcomputer) 11. Here, for example, when a recording key is pressed among the various keys, the microcomputer 11 determines that a recording start instruction has been given from a voltage corresponding to the pressing of the key, and starts the recording operation. . The microcomputer 11 performs all of the following series of device controls during recording.

The S video signal input from the S video input terminal 12 is subjected to analog processing such as synchronization separation and NTSC chroma decoding by a VA / D circuit 13, and then A / D (Y / D) is converted to a YUV digital signal. (Analog / Digital) conversion and input to the frame synchronizer 14. The synchronization signal (VFR-ext) 21 synchronized and separated by the VA / D circuit 13 is a signal synchronized with the input, and serves as a control signal on the input side of the frame synchronizer 14.

On the other hand, a timing generator (TG) 2
The synchronization signal (VFR-int) 22 generated free from the 27 MHz crystal oscillator in step 4 is an internal reference synchronization signal for a video signal, and serves as a control signal on the output side of the frame synchronizer 14. The synchronization signal (VFR-in
t) The YUV signal output from the frame synchronizer 14 in synchronization with 22 is input to the MPEG-V encoder 15, encoded into an MPEG video signal, and stored in an internal code buffer.

The L (left) and R (right) stereo audio signals are input from audio signal input terminals 16L and 16R, are filtered by an A / A / D circuit 17, are A / D converted, and are MPEG-converted. -A Encoder 18 is supplied. M
The PEG-A encoder 18 synchronizes with a synchronizing signal (AFR-int) 23 for speech encoding generated from 44.1 KHz by a timing generator (TG) 24 and
It is encoded into a PEG audio signal and stored in an internal code buffer.

The encoded codes of the video and audio are converted by a multiplexer (MUX) 19 into an MPEG code.
System Program Stream (P
The header addition and the multiplexing process are performed in accordance with S), and the data are accumulated in the recording buffer (WBUF) 20. At this time, the multiplexer 19 outputs the reference time counter (Re)
cSTC) 25 and the synchronization signal (VFR-int) 2
2 and the time information at which the frame synchronizer processing was performed at the time latched at the timing of the synchronization signal (AFR-int) 23, respectively, is also a PES (Packetized Elementar).
y Stream) header contains the display time PTS (Presentation
Time Stamp) and decoding time DTS (Decoding Time S)
tamp). Details of this processing will be described later.

Finally, the MPEG bit stream stored in the recording buffer (WBUF) 20 is transferred in response to a request from the disk drive 26 and recorded on a disk as a recording medium. In addition, an optical disk, a hard disk, or the like can be used as the disk.

Next, the reproducing operation will be described.

As in the case of recording, when the reproduction key is pressed, for example, among the various keys, the microcomputer 11 determines that the reproduction start is instructed from the voltage corresponding to the pressing of the reproduction key, and performs the reproduction operation. Start. The microcomputer 11
Performs all of the following series of device controls during playback.

When this reproducing operation is started, the MPEG bit stream recorded on the disk is reproduced and stored in the reproducing buffer (RBUF) 27 as soon as the disk drive 26 is ready.

The bit stream stored in the reproduction buffer (RBUF) 27 is input to a demultiplexer (DMUX) 28 in accordance with the MPEG system, where the header is analyzed and then separated into video and audio. Decoder, MPEG-V decoder 29 for video
Are stored in a code buffer in the MPEG-A decoder 33 for audio. The encoded code in each code buffer is the decoding time DT obtained by the header analysis.
S, the phase management with the display time PTS is performed, and when the time matches the time of the reproduction reference time counter (PbSTC) 37, the encoding is performed by the respective MPEG decoders.
Actually, at each of the decode time DTS and the display time PTS, the reference time counter (PbSTC) 37 outputs the synchronization signals (VFR-pb) 30, (AFR-
pb) 34 is generated to determine the timing of the processing.

The MPEG-V decoder 29 operates in synchronization with the internal synchronization signal (VFR-pb) 30, and the decoded YUV data is input to the VD / A circuit 31. V
The D / A circuit 31 converts each of the YUV data into a D / A
After performing (digital / analog) conversion processing to convert the analog signal to an analog signal, analog processing such as NTSC chroma encoding and synchronization addition is performed, and the obtained S video signal is output from the S video output terminal 32.

The MPEG-A decoder 33 operates in synchronization with the internal synchronizing signal (AFR-pb) 34, and the decoded audio signal is input to the AD / A circuit 35.
In the AD / A circuit 35, the stereo L (left), R
(Right) Each signal is D / A converted and converted into an analog signal. These analog signals are then subjected to analog processing such as filtering and output from the audio signal output terminals 36L and 36R as stereo audio signals. .

Next, the control of the frame synchronizer 14 in the configuration of the above-described embodiment will be described with reference to FIG.
This will be described with reference to the timing chart shown in FIG. Note that the timing chart of FIG. 3 is drawn with a rather large frequency variation that cannot be normally obtained, in order to make the description easy to understand. Also, the numbers 1, 2, 3, 4,.
Indicates a frame.

The frame synchronizer 14 shown in FIG.
Is a circuit for switching clocks and synchronization timings using a frame memory, and is used to convert a non-standard signal or a signal with jitter to a standard signal and further adjust the timing to a desired frame synchronization timing. . Therefore, since the input and output of the frame synchronizer 14 are completely asynchronous and the clocks have different phases, it is necessary to adjust the time by removing or repeating one frame of the output in a certain cycle. In the example of FIG. 2, an external synchronization signal (VFR-ext) 21 synchronized with the video input signal,
The reference internal synchronization signal (VFR-int) 22 synchronized with the output signal of the frame synchronizer 14 is completely asynchronous, and the phases of the external clock (CLK) and the internal clock are different.

In FIG. 3, the timing indicated by OUT1 shows a case where the frequency of the internal reference synchronization signal (VFR-int) 22 is slightly higher than that of the input external synchronization signal (VFR-ext) 21, In this case, when the phase difference between the input external synchronizing signal (VFR-ext) 21 and the internal synchronizing signal (VFR-int) 22 becomes shorter than 0 frame, processing for repeating one frame is performed. FIG.
In the example, the frame of the number 5 is repeated.

The timing shown at OUT2 in FIG. 3 shows a case where the frequency of the internal reference synchronization signal (VFR-int) 22 is slightly lower than that of the input external synchronization signal (VFR-ext) 21. In this case, if the phase difference between the input external synchronizing signal (VFR-ext) 21 and the internal synchronizing signal (VFR-int) 22 becomes longer than one frame, a process of removing one frame is performed. The example of FIG. 3 shows a state in which the frame of number 6 is omitted.

With the above control, the phase relationship between the input signal and the output signal is always delayed by 0 to 1 frame.

Next, a method of generating time information to be multiplexed in the configuration of the above-described embodiment will be described with reference to a block diagram of FIG.

The delay time of the video signal from the input to the output of the frame synchronizer 14 being 0 to 1 frame as described above means that the delay time between the input audio signal and the video signal output from the frame synchronizer 14 also increases. 0 to
That is, one frame. Therefore, before the audio time information (A-PTS) is multiplexed by the multiplexer 19, the adder 30 in the microcomputer 11 calculates the audio time information (A-PTS) as shown in the following equation (1). Is the time for one frame of the internal reference synchronization (dVFR-in
t) is added in advance. By doing so, the input audio signal and the frame synchronizer 14
The delay time from the video signal output from the time stamp is -1 to 0 frames on the time stamp.

A-PTS '= A-PTS + dVFR-int (1) This is because the audio signal is delayed more visually than the video image in time, and is less noticeable in the visual sense. .3ms) should be well tolerated.

As described above, by performing this processing at the time of multiplexing in advance, decoding / output can be performed by simply looking at only time information at the time of reproduction / separation as in the normal case.

It should be noted that the present invention is not limited to the above-described embodiment, but can be applied to, for example, FIGS. 5 and 6 show only necessary parts of the configuration shown in FIG.

FIG. 5 is a block diagram showing an application of the MPEG-V encoder 15 to parameter control. In FIG. 5, the information obtained by skipping or repeating one frame output by the frame synchronizer 14 is sent to the microcomputer 11 in real time. The microcomputer 11 dynamically controls the encoding parameters of the MPEG-V encoder 15 based on the information.

For example, when a frame is repeated in the frame synchronizer 14, since the image of the repeated frame is exactly the same as the previous frame, the microcomputer 11 sets this frame to a P picture in MPEG, for example, and sets the motion vector as a motion vector. Control is performed so as to be fixed to 0. On the other hand, the frame synchronizer 14
In the above, when a frame is skipped, the temporal distance between the frame immediately after the frame is skipped and the previous frame is doubled, and the microcomputer 11 increases the motion vector search range and the like accordingly. Such control is performed.

In the case of the example shown in FIG. 5, the image quality can be improved by performing the above-described parameter control.

FIG. 6 is a block diagram showing an application of variable speed reproduction to display control. In the configuration of FIG. 6, a display controller 40 is newly provided.

In FIG. 6, first, in the recording system, information obtained by extracting or repeating one frame of the output by the frame synchronizer 14 is sent to the microcomputer 11, and the multiplexer is combined with the time information by matching the phase with the video signal. Multiplex at 19. In the reproduction system, the multiplexed frame missing / repeated information is separated and the microcomputer 1
1 and the microcomputer 11 sends the MPEG-
By controlling the V decoder 29 and the display controller 40, the control method of variable speed reproduction is dynamically changed.

For example, when there is a frame repetition point during slow reproduction or frame-by-frame reproduction, the microcomputer 11 controls the MPEG-V decoder 29 and the display controller 40 so that the repetition frame is skipped and displayed. . On the other hand, when there is a point where a frame has been omitted, the microcomputer 11 creates a frame by interpolating from the previous and next frames and displays the frame.
-Control the V decoder 29 and the display controller 40.

It should be noted that since the audio is not reproduced during the variable speed reproduction in most cases, there is no need to consider the synchronization between the video and the audio. Therefore, by performing such purely faithful control on the original video signal, The quality of variable speed reproduction can be improved.

As described above, according to the embodiment of the present invention, the following effects can be obtained.

1. It is possible to generate time information for synchronizing and reproducing the video and audio when a frame synchronizer is inserted in the video input within a range where there is no audiovisual problem by simple means.

2. Synchronization of video and audio when a frame synchronizer is inserted can be realized without requiring a special circuit in the audio system and the reproduction system.

3. With the minimum additional means, it is possible to improve the image quality of encoding and the quality at the time of variable speed reproduction.

[0046]

As is apparent from the above description, according to the present invention, even when the delay time of the means (frame synchronizer) for performing the variable delay processing of the video is arbitrarily changed from 0 to 1 frame, the reference of the video is not changed. By using only the time information uniquely calculated from the synchronization timing and the reference synchronization timing of the audio, a simple method and configuration (without adding a circuit to the audio system) allows the video and audio of the audiovisual range to be within a range without any problem. Synchronization can be easily achieved. Further, in the present invention, the information of the means (frame synchronizer) for performing the variable delay processing of the video can be applied to the improvement of the image quality of encoding and the control of variable speed reproduction at the time of decoding.

[Brief description of the drawings]

FIG. 1 is a block circuit diagram showing a schematic configuration of a disk recorder as one embodiment of the present invention.

FIG. 2 is a diagram illustrating a frame synchronizer of the configuration according to the embodiment of the present invention;

FIG. 3 is a timing chart showing control of the frame synchronizer in the configuration of the embodiment of the present invention.

FIG. 4 is a diagram used to describe a method of generating time information to be multiplexed in the configuration of the embodiment of the present invention.

FIG. 5 is a block circuit diagram showing an application of the MPEG encoder of the present invention to parameter control.

FIG. 6 is a block circuit diagram showing application to display control of variable-speed reproduction in the disk recorder according to the embodiment of the present invention.

[Explanation of symbols]

10 key circuit, 11 microcomputer, 12 S video signal input terminal, 13 V-A / D circuit, 14 frame synchronizer, 15 MPEG-V encoder,
16L, 16R stereo audio signal input terminal, 17
A-A / D circuit, 18 MPEG-A encoder,
19 Multiplexer (MUX), 20 Recording buffer (WBUF), 21 External video synchronization signal (VFR-
ext), 22 Internal video synchronization signal (VFR-in)
t), 23 internal audio synchronization signal (AFR-int),
24 timing generator (TG), 25 recording reference time counter (RecSTC), 26 disk drive, 27 reproduction buffer (RBUF),
28 Demultiplexer (DMUX), 29MPEG
-V decoder, 30 playback video synchronization signal (VFR-p
b), 31 V-D / A circuit, 32 S video signal output terminal, 33 MPEG-A decoder, 34 playback audio synchronization signal (AFR-pb), 35 A-D / A circuit, 36 L, 36 R stereo audio signal output terminal , 3
7 Reproduction reference time counter (PbSTC)

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Mami Tomita 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo Inside Sony Corporation (72) Inventor Taro Mizuto 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo No. 72 Inside Sony Corporation (72) Katsunari Miyata 6-7-35 Kita Shinagawa, Shinagawa-ku, Tokyo Sonny Corporation (72) Koichi Nagatoku 6-35 Kita Shinagawa, Shinagawa-ku, Tokyo Soniー Inc.

Claims (6)

[Claims] 1. A video / audio synchronization method having a variable delay process for synchronizing an input video signal with a reference synchronization signal, wherein a video delay time due to the variable delay process is controlled within one frame. Video time information is calculated from the reference synchronization timing, audio time information is calculated from the audio signal reference synchronization and the video signal reference synchronization timing, and the calculated time information is multiplexed with the video and audio signals. Generating a signal, separating the video, audio signal and time information from the recorded or transmitted multiplexed signal, and synchronizing video and audio based on the separated time information. Method. 2. The video and audio signals before multiplexing are compressed and encoded on a time axis, and frame jump information or frame repetition information by the variable delay processing is detected. 2. The video / audio synchronization method according to claim 1, wherein said encoding parameters are controlled based on information. 3. The frame skip information or frame repetition information by the variable delay processing is detected, the frame skip information or frame repetition information is also multiplexed, and the frame skip information or frame is obtained from the recorded or transmitted multiplexed signal. 2. The video / audio synchronizing method according to claim 1, wherein repetition information is also separated, and control during variable speed reproduction is performed based on the separated frame skip information or frame repetition information. 4. A video / audio signal recording / reproducing apparatus having variable delay means for synchronizing an input video signal with a reference synchronizing signal, wherein the delay time for controlling the delay time of the video signal by the variable delay means within one frame. Control means; video time information calculating means for calculating video time information from the video signal reference synchronization timing; and audio time information calculation for calculating audio time information from the audio signal reference synchronization and the video signal reference synchronization timing. Means, multiplexing means for multiplexing the time information calculated by the video time information calculating means and audio time information calculating means together with video and audio signals, and recording the multiplexed signal from the multiplexing means on a recording medium. ,
A recording / reproducing means for reproducing a multiplexed signal recorded on the recording medium;
A video / audio signal recording / reproducing apparatus comprising: a separating unit that separates an audio signal and time information; and a synchronizing unit that synchronizes a video signal and an audio signal based on the separated time information. 5. A compression encoding means for compressing and encoding the video and audio signals before multiplexing on a time axis, a detection means for detecting frame skip information or frame repetition information by said variable delay means, 5. The video / audio signal recording / reproducing apparatus according to claim 4, further comprising coding control means for controlling the coding parameter based on the detected frame skip information or frame repetition information. 6. A multiplexing apparatus comprising: detecting means for detecting frame skip information or frame repetition information by the variable delay means; and reproduction control means for performing control during variable speed reproduction based on the frame skip information or frame repetition information. Multiplexing means also multiplexes the detected frame skip information or frame repetition information. The separation means also separates the frame skip information or frame repetition information from the multiplexed signal reproduced from the recording medium. 5. The video / audio signal recording / reproducing apparatus according to claim 4, wherein the means performs the control at the time of the variable speed reproduction using the frame skip information or the frame repetition information separated by the separating means.