KR20060010829A - Device for recording video data and audio data - Google Patents

Abstract

When a pause request is made, an audio delay time as a delay time of an audio data frame based on a video data frame is calculated. During a pause, a frame shift time as a shift between the video data and audio data frame start time is monitored. When a pause release request is made, an audio correction time to be corrected in the pause request is calculated according to the audio delay time and the frame shift time. According to the audio correction time accumulated for each pause request, when it is judged that the audio data advances with respect to the video data, the video data is delayed by one frame with respect to the audio data, and when it is judged that the audio data is delayed with respect to the video data, the audio data is delayed by one frame with respect to the video data.

Description

Recording device for video data and audio data {DEVICE FOR RECORDING VIDEO DATA AND AUDIO DATA}

The present invention relates to an audio / video synchronization processing apparatus for synchronizing video data and audio data, an audio / video synchronization processing method, and an audio / video recording apparatus. In particular, the present invention relates to an AV (audio / video) synchronization technique at the time of pause (pause) of video data and audio data.

For example, in an audio / video synchronization processing device (AV recording device) such as an input device of an MPEG encoder, the frame length (frame period) of an input signal of video data and audio data is generally different. In addition, there is a feature in that a period of receiving audio data and video data is performed in units of frames, respectively. Hereinafter, the configuration and operation of such a conventional AV recording apparatus will be described.

15 is a system configuration diagram of a conventional AV recording apparatus.

This system is comprised of the data control part 2a and system encoder 3a which receive control instruction from the host HOST 1a.

The data control unit 2a receives the control instruction from the host 1a by the audio / video control unit (AV_CTRL) 21a and based on the time information from the timer (TIMER) 24a, the audio control unit 22a, Control is performed on the video control unit 26a.

In the following, the audio / video control unit is referred to as an AV control unit.

The AV control unit 21a controls the input of the voice data A_DATA by giving a control instruction to the voice control unit A_CTRL 22a. The input voice data is stored in the voice data memory (A_MEM) 23a.

The AV control unit 21a also controls the input of the video data V_DATA by giving a control instruction to the video control unit V_CTRL 26a. The input video data is stored in the video data memory (V_MEM) 25a.

Based on the time information from the timer 24a, the data control unit 2a adds audio data A_PTS and video data V_PTS to which the system encoder 3a is added a PTS (Presentation Time Stamp) as time information. To provide.

The system encoder 3a is controlled by the control instruction from the host 1a. The speech encoder (A_ENC) 31a encodes and encodes the speech data to which the PTS from the data control unit 2a is added. The video encoder (V_ENC) 33a encodes and encodes the video data to which the PTS from the data control unit 2a is added. The multiplexer (MPX) 32a multiplexes the data encoded by the audio encoder 31a and the video encoder 33a to generate a bit stream BSD.

However, in an AV recording apparatus including an encoder of MPEG, the frame periods of video data and audio data do not change due to hardware limitations. In such a case, if the pose processing is performed on the basis of the frame of the video data, there is a problem (AV synchronization misalignment) that a deviation of the audio data with respect to the video data occurs when the pose is released later.

If the proper treatment is not performed on this problem, disparity in motivation accumulates and the viewer becomes perceived as a sense of ecstasy.

Hereinafter, the conventional problem will be described in detail with reference to FIG.

Fig. 16 is a diagram showing an example of AV synchronization misalignment when controlling pose and release of pause.

In the conventional AV recording apparatus shown in Fig. 15, data reception control can be performed only in units of frames, and each frame period (video_frame_time, audio_frame_time) of video data and audio data cannot be changed during a pause.

In FIG. 16, when the pause request (represented by "P" in the figure) is received from the host 1a, the pause request is reflected by the data control unit 2a at t161, which is the short time of the frame of the video data 1. The audio data has a frame period midway at time t161, and the pause request is reflected in the next audio frame. Thus, tp161 is generated as a difference between video data and audio data at the time of pause.

During the pause, the frame period of the video data and the frame period of the audio data remain unchanged, and are not corrected with tp161 being the difference between the video data and the audio data during the pause.

When the pause release request (represented by " P_RL " in the figure) is received from the CPU 1a, it is the time t162 at which the pause release request is reflected in the data control unit 2a at the start of input of the video data n (VDn). ) Is the timing. Here, if the timing of the audio data with respect to the video data is adjusted in consideration of tp161, which is the difference between the video data and the audio data at the time of pause, the AV synchronization does not occur.

However, due to the difference between the frame periods of the video data and the audio data, the difference tp162 from t162 which is the time of pause release to the input start time of the audio input data n (ADn) is determined. As a result of the difference between the audio data and the video data, as a result, a deviation (tp163) of AV synchronization occurs when the pause is released from the time t161 and the time tp162.

In particular, when the frame periods of the video data and the audio data do not change, this tp163 may accumulate for each pause request, which may result in perceived discomfort.

SUMMARY OF THE INVENTION An object of the present invention is to provide an AV synchronization processing apparatus and method which do not cause AV synchronization deviation in an AV recording apparatus in which the frame lengths of the video data and the audio data are different and the frame lengths of the video data and the audio data cannot be changed. It is in doing it.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and a first aspect thereof is an audio / video synchronization processing apparatus which performs synchronization processing on video data and audio data having different predetermined frame lengths, respectively.

Timer means,

Storage means for storing the start time of each frame of the video data and audio data timed by the timer means, the time of a pause request, and the time of a pause release request;

On the basis of the start time of each frame of the video data and audio data, the time of the pause request, and the time of the pause release request, which one of the video data and the audio data is delayed after the pause release request in units of frames. Audio / video synchronization processing device having control means for determining whether or not to delay anything.

The control means,

When a pause request is made, an audio delay time which is a delay time of a frame of audio data is calculated on the basis of a paragraph of a frame of video data,

After the pause request, each frame start time of the frame of the video data is monitored for a frame shift time which is a difference of the frame start time of the audio data with respect to the video data,

Calculate a speech correction time based on the speech delay time and the frame shift time at the time of the pause release request for the pause request,

Based on the cumulative speech correction time accumulated in the speech correction time calculated for each pause request, it is determined whether one of the video data and the audio data is delayed by frame or none after the pause release request. .

According to the first aspect of the present invention, the delay time (audio delay time) of the audio data with respect to the video data at the time of the pause request is obtained, and the shift time of the frame of the video data and the audio data during the subsequent pause is always By monitoring, even if there is a pause release request, the reproduction timing of the audio data after the pause release is adjusted to suppress the deviation of the audio data with respect to the video data to one audio data frame or less, thereby significantly suppressing the AV synchronization deviation. Can be.

A second aspect of the present invention is an audio / video recording apparatus for generating multiplexed data including video data and audio data having different predetermined frame lengths, respectively.

Timer means,

Storage means for storing a start time of each frame of the video data and audio data, time of a pause request, and time of a pause release request, revealed by said timer means;

Synchronization control means for performing synchronization processing of the audio data after the pause release request on a frame-by-frame basis based on a start time of each frame of the video data and audio data, the time of the pause request, and the time of the pause release request;

And multiplexed data generating means for generating the multiplexed data by adding time information to the video data and the audio data synchronized by the synchronous control means.

According to the second aspect of the present invention, the delay time (audio delay time) of the audio data with respect to the video data at the time of the pause request is obtained, and the shift time of the frame of the video data and the audio data during the subsequent pause is always By monitoring, even when there is a pause release request, since the reproduction timing of the audio data after the pause release is adjusted so as to suppress the deviation of the audio data with respect to the video data to one audio data frame or less, the AV synchronization shift is significantly suppressed. Multiplexed data can be generated.

BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 shows the system configuration of an AV recording apparatus as one embodiment of the present invention.

2 is a flowchart showing processing when the AV control unit 21 receives a START request from the host 1;

3 is a timing chart for explaining an image PTS (V_PTS) and an audio PTS (A_PTS) generated according to the start of data input.

4 is a flowchart showing a process of adding a PTS when the data control unit 2 provides voice data to the system encoder 3;

5 is a flowchart showing a process of adding a PTS when the data control unit 2 provides the video data to the system encoder 3;

FIG. 6 is a flowchart showing a process performed by the AV control unit 21 based on a pause request from the host 1. FIG.

7 is a timing chart showing processing for a pause request.

FIG. 8 is a flowchart showing a process (process during pose) after a pause request process from the host 1; FIG.

9 illustrates a method of measuring frame shift time f_count.

FIG. 10 is a flowchart showing processing performed by the AV control unit 21 when a pause release request is made from the host 1. FIG.

Fig. 11 is a timing chart illustrating a method of calculating the speech correction time a_diff when the shift time is measured during a pause.

12 is a timing chart illustrating a method of calculating a speech correction time (a_diff) when the shift time is not measured during a pause.

FIG. 13 is a diagram for explaining a process of eliminating AV synchronization deviation by a process of delaying input resume of video data by one frame; FIG.

FIG. 14 is a diagram for explaining a process of eliminating AV synchronization deviation by a process of delaying input resume of audio data by one frame; FIG.

15 is a diagram showing a system configuration of a conventional AV recording apparatus.

Fig. 16 is a timing chart showing a pause and de-pause process of a conventional AV recording apparatus.

EMBODIMENT OF THE INVENTION Hereinafter, preferred embodiment of this invention is described with reference to an accompanying drawing.

1 is an AV recording apparatus which is an embodiment of an audio / video synchronization processing apparatus according to the present invention. In addition, the AV recording apparatus shown in FIG. 1 has the same system configuration as the conventional AV recording apparatus shown in FIG. 15, but is characterized by control by the AV control unit 21.

In the following procedure, the processing based on the START request from the host HOST 1, the normal processing, and the pause request from the host 1 in the AV control unit AV_CTRL 21 will be described below. Processing, processing during pause, processing based on a pause release request from the host 1, and a process of eliminating AV synchronization shift caused by the pause and pause release request will be described.

First, the processing performed by the AV control unit 21 based on the START request from the host 1 will be described.

2 is a flowchart showing processing when the AV control unit 21 receives a START request from the host 1.

When the AV control unit 21 receives the START request from the host 1, the AV control unit 21 obtains time information from the timer 24 and stores it in a memory (not shown) as STC_offset. The timer (TIMER) 24 is, for example, a timer that operates with a clock of 90 kHz.

2 shows a processing flow of the START request from the host 1 in the data control unit 2.

First, when the AV control unit 21 receives the START request from the host 1, the AV control unit 21 waits for a short circuit of the video data frame and detects a short circuit of the video data frame (ST21). It acquires and stores the time information as STC_offset (ST22).

Next, an instruction to start inputting video data is given to the video controller (V_CTRL) 26 (ST23), and an instruction to start input of audio data to the audio controller (A_CTRL) 22 (ST24), and the START from the CPU1. Processing of the request ends.

3 is a timing chart for explaining an image PTS (V_PTS) and an audio PTS (A_PTS) generated according to the start of data input.

In FIG. 3, upon receiving the START request from the host 1, the AV control unit 21 of the data control unit 2 starts input of the video data and the audio data on the basis of the frame of the video. Then, the time t31 at the start is obtained from the timer 24 and stored as STC_offset.

After that, the current time is sequentially obtained from the timer 24 in the section of each frame of video data and audio data, and the value obtained by subtracting STC_offset (t31) at the start time is output to the system encoder 3 as a PTS. .

For example, in FIG. 3, when detecting the short circuit of a video data frame, time t32 is acquired from the timer 24, and the system encoder 3 notifies PTS of video data with video input data. Similarly, when a short circuit of the audio data frame is detected, the time t33 is obtained from the timer 24, and the system encoder 3 is notified of the PTS of the audio data together with the audio data.

Next, the normal processing after the START request processing from the host 1 will be described.

4 is a flowchart showing a process of adding a PTS when the data control unit 2 provides voice data to the system encoder 3.

The AV control unit 21 acquires and stores time information from the timer 24 when detecting the short circuit of the frame of the audio data (ST41) (ST42). Then, the voice PTS is generated from the STC_offset stored at the start time and the obtained time information (ST43). Finally, the voice encoder (A_ENC) 31 of the system encoder 3 is notified of information in which PTS information is added to the voice frame data (ST44).

The above processing is performed for each audio input frame in the normal processing.

FIG. 5 is a flowchart showing a process of adding a PTS when the data control unit 2 provides the video data to the system encoder 3.

The AV control unit 21 acquires and stores time information from the CST51 and the timer 24 when detecting the short circuit of the frame of the video data (ST52). Then, the video PTS is generated from the STC_offset stored at the start time and the obtained time information (ST53). Finally, the video encoder 33 of the system encoder 3 is informed of the information in which the PTS information is added to the video frame data (ST54).

The above processing is performed for each video input frame in the normal processing.

In accordance with the flowcharts shown in Figs. 4 and 5, input of each data is started, and audio data and video data obtained by AV synchronization with a PTS is provided from the data controller 2 to the system encoder 3; .

Next, the processing for the pause request from the host 1 will be described.

FIG. 6 is a flowchart showing a process performed by the AV control unit 21 based on the pause request from the host 1. When the pause request is received from the host 1, the time information acquired by the AV control unit 21 from the timer 24 is set to pause_STC_offset.

When the AV control unit 21 receives the pause request from the host 1, the AV control unit 21 waits for a short circuit of the video data frame and detects a short circuit of the video data frame (ST61). Then, the AV controller 21 sets pause_STC_offset as time information from the timer 24. (ST62). Further, the video control unit 26 gives an instruction to stop inputting the video data (ST63), and starts measuring the shift time of the audio data and the video data based on the time information from the timer 24 (ST64).

Next, when waiting for a short circuit of the audio data frame and detecting a short circuit of the audio frame (ST65), the shift time measurement of the audio data and the video data is terminated based on the time information from the timer 24 (ST66). . At the same time, the shift time of the audio data and the video data is stored as the audio delay time a_delay (ST67). Further, an instruction to stop inputting voice data is made (ST68), and the processing of the pause request from the host 1 is terminated.

FIG. 7 is a timing chart showing processing for the pause request shown in FIG.

In FIG. 7, upon receiving a pause request from the host 1, the AV control unit 21 stops input of the video data on the basis of the frame of the video data. The time t71 acquired from the timer 24 at this time is stored as pause_STC_offset. Then, from the time t71 at which the video data input is paused, the time t72 is obtained from the timer 24 when the short circuit of the frame of the audio data is detected next.

In addition, the difference between the time t72 and the time t71 is stored as a_delay, and the input of the audio data is paused.

Next, processing (during processing) after the pause request processing from the host 1 (after time t72 in FIG. 7) will be described with reference to the flowchart shown in FIG. 8.

During the pause, as described below, the frame shift time f_count, which is a shift time of a frame of audio data and video data, is measured.

In Fig. 8, first, a determination is made as to whether a current pause is made (ST81). If a pause is made, a short circuit of a frame of voice data is awaited, and if a short frame of voice is detected (ST82), time information is acquired from the timer 24. And the frame shift time is measured for the audio data and the video data (ST83).

Next, when waiting for a short circuit of the frame of the video data and detecting a short frame of the video (ST84), time information is obtained and stored from the timer 24, and the frame shift time measurement of the audio data and the video data is finished (ST85). ).

The frame shift time f_count is recorded from the shift time measurement start time of the audio data and the video data in ST83 and the shift time measurement end time of the audio data and the video data in ST85 (ST86).

The above process is repeated during the pause, and the measurement of the frame shift time f_count is continued. Since the frame shift time f_count is rewritten in the memory in the AV control unit 21, the frame shift time f_count represents the shift time of the latest audio data and video data during the pause. Here, always updating f_count is because it is impossible to predict when a pause release request will occur, and it is necessary to prepare for the request.

9 illustrates a method of measuring the frame shift time f_count described based on the flowchart of FIG. 8.

Upon detecting the short circuit of the audio data, the AV control unit 21 acquires the time information t91 from the timer 24, and starts the shift time measurement of the audio data and the video data.

Next, when a short circuit of the video data is detected, the AV control unit 21 obtains time information t92 from the timer 24, and performs shift time measurement t92-t91 between the audio data and the video data. Here, the shift time of the measured audio data and the video data is the frame shift time f_count.

During the pause, this control is repeated based on the audio data, and the latest frame shift time f_count is always stored. In FIG. 9, the latest value of the frame shift time f_count is the difference t96-t95 between the time t95 and the time t96.

Next, the processing for the pause release request from the host 1 will be described.

In other words, when there is a pause release request from the host 1, the AV control unit 21 delays the resumption of the input of the audio data based on the deviation of the audio data and the video data measured during the pause request and during the pause. Next, a method of determining whether to delay the resumption of input of the video data or neither of them is delayed, and how to eliminate the deviation of the AV synchronization is described below.

FIG. 10 is a flowchart showing a process performed by the AV control unit 21 when a pause release request is made from the host 1.

In addition, a_diff of the flowchart of FIG. 10 has shown the audio | voice correction time which is a shift | offset | difference of audio | voice data and video data at the time of a pause and a pause release. In addition, total_audio_delay in the flowchart of FIG. 10 is a cumulative audio correction time which is a variable for accumulating deviation of audio data with respect to video data, and is initialized to 0 at system initialization.

In addition, as already described, the frame shift time f_count is updated at the timing of the paragraph of the video data frame. Therefore, the time from the paragraph of the frame of audio data to the paragraph of the frame of video data on the time axis means that "in the shift time measurement of audio data and video data" in the flowchart of FIG.

For example, the time indicated by the horizontal arrow in Fig. 9 means that the shift time of the audio data and the video data is being measured, and the time other than that is not during the shift time measurement of the audio data and the video data. It means.

In FIG. 10, when the AV control unit 21 receives the pause release request from the host 1, the AV control unit 21 waits for a short circuit of the frame of the video data, and detects a short circuit of the frame of the video data (ST101), and updates the STC_offset ( ST102).

After that, it is determined whether the shift time measurement of the audio data and the video data is being performed (ST103), and if the shift time measurement of the audio data and the video data is being measured, the pause time and the pause release time are based on Equation (1) described later. The audio correction time a_diff, which is the shift time between the audio data and the video data, is obtained (ST104). If it is not measuring, the voice correction time a_diff is obtained based on equation (2) described later (ST105).

The details of the voice correction time a_diff will be described later, but based on the voice delay time a_delay and the frame shift time f_count during the pause, the voice of the video data to be corrected at the time of pause release in the pause processing is performed. The data is misaligned. When the audio correction time a_diff is a positive value, it means that the audio data is delayed with respect to the video data. When the audio correction time a_diff is negative, the audio data with respect to the video data. Means preceded.

Next, the speech correction time a_diff obtained in step ST104 or ST105 is added to the cumulative speech correction time total_audio_delay (ST106).

Therefore, the cumulative speech correction time (total_audio_delay), which is the initial value 0 at the time of system startup, is cumulatively added in step ST106 for a plurality of pause processes while the system is in operation. Since the speech correction time a_diff is a deviation of the speech data to be corrected in each pose processing, the cumulative speech correction time total_audio_delay is a cumulative value obtained by adding each speech correction time a_diff. The audio data is corrected for the video data to be corrected.

After step ST107, on the basis of the accumulated voice correction time (total_audio_delay) value accumulated for each pause processing during system operation, how to control the deviation of the AV synchronization, specifically, how to correct the deviation of the audio data with respect to the video data. Is corrected, and the process of determining which of the audio data and the video data is delayed.

First, in step ST107, when the cumulative speech correction time (total_audio_delay) is negative, i.e., audio data is preceded, the cumulative speech correction time (total_audio_delay) is added to the cumulative speech correction time (total_audio_delay) by one frame length of time (ST108). A process of delaying the resumption of the video data by one frame is performed. The process of delaying the resumption of the video data by one frame is realized by waiting for resumption of input of the video data until a short circuit of the frame of the video data is detected (ST109).

Upon detecting a short circuit in the video frame, input of video data is resumed (ST110).

In step ST107, if the cumulative audio correction time total_audio_delay is not negative, i.e., the audio data is the same or delayed, the input of the video data is resumed without delay (ST110), and the process goes to step ST111.

In step ST111, when the positive cumulative speech correction time total_audio_delay is equal to or greater than one audio data frame audio_frame_time, it is necessary to delay the resumption of the audio data.

If the cumulative speech correction time total_audio_delay is positive but less than one audio data frame, the speech data is resumed without delay (ST114).

In step ST112, the cumulative speech correction time (total_audio_delay) is subtracted from the time of one frame length of speech data (ST112), and then the processing for actually delaying the resume of speech data by one frame is performed. The process of delaying the resumption of the audio data by one frame is realized by waiting for the input of the audio data to be resumed until a short circuit of a frame of the audio data is detected (ST113).

Upon detecting a short circuit in the audio frame, input of video data is resumed (ST114).

Next, the calculation method of the audio | voice correction time a_diff in step ST104 and ST105 of FIG. 10 is demonstrated with reference to FIG. 11 and FIG.

Fig. 11 is a timing chart illustrating a method of calculating the voice correction time a_diff when the shift time is measured during a pause.

In the timing chart shown in Fig. 11, during the f_count measurement, that is, the pause release request (P_RL) from the host 1 to the AV control unit 21 is performed between the paragraph of the frame of the audio data and the paragraph of the frame of the video data. In this case, the voice correction time a_diff is calculated using the value of the frame shift time f_count obtained after the pause release request.

Hereinafter, the procedure performed in step ST104 of FIG. 10 will be described in order to calculate the speech correction time a_diff according to FIG. 11.

When the AV control unit 21 receives the pause release request from the host 1, the AV control unit 21 obtains the time t111 from the timer 24 in accordance with the frame period of the video, and based on the pause_STC_offset stored at the time of the pause request. Reset STC_offset.

In addition, the frame shift time f_count is also measured at the timing t111.

Here, a_delay is the frame shift time of the audio data and the video data at the time of pause as described above, and is the data calculated and stored at the time of pause. In addition, audio_frame_time is a frame period of audio data.

As is apparent from Fig. 11, the speech correction time a_diff can be obtained by the following equation (1).

a_diff = a_delay + f_count-audio_frame_time... (One)

Fig. 12 is a timing chart illustrating a method of calculating the voice correction time a_diff when the shift time during pause is not measured.

In the flow chart shown in Fig. 12, during the f_count measurement, that is, a pause release request from the host 1 to the AV control unit 21 is made between a paragraph of a frame of video data and a paragraph of a frame of audio data. Therefore, the speech correction time a_diff is calculated using the frame shift time f_count obtained before the pause release request.

Hereinafter, the procedure performed in step ST105 of FIG. 10 will be described in order to calculate the speech correction time a_diff according to FIG. 12.

When the AV control unit 21 receives the pause release request from the host 1, it acquires the time t121 from the timer 24 in accordance with the frame period of the video, and based on the pause_STC_offset stored at the time of the pause request. Reset STC_offset.

Here, a_delay is the pause audio delay time which is the audio data at the time of a pause and the video data shift time, as previously described, and is data which was calculated and stored at the time of pause.

In addition, audio_frame_time is a frame period of audio data.

video_frame_time is a frame period of video data.

As is apparent from Fig. 12, the speech correction time a_diff can be obtained by the following equation (2).

a_diff = a_delay + f_count-audio_frame_time + video_frame_time... (2)

Next, a process for eliminating the deviation of the AV synchronization at the time of pause release will be described in detail with reference to FIGS. 13 and 14.

FIG. 13 is a diagram for explaining a process of eliminating AV synchronization deviation by a process of delaying input resume of video data by one frame.

The control for delaying the resumption of the input of the video data is performed by correcting the AV synchronization shift because the cumulative voice correction time (total_audio_delay) is negative (ST107), as described above according to the flowchart at the time of release of the pause of FIG. (ST108) This is realized by performing a process of delaying the resumption of the video data by one frame until a short circuit of the frame of the video data is found (ST109).

13, when the AV control unit 21 receives the pause release request from the host 1, it waits for a short circuit of the frame of the video data and detects a short circuit of the frame of the video data (time t131). Based on the processing flow of 10, the cumulative speech correction time total_audio_delay is calculated. Since this is negative, input of video data is resumed after waiting for one video frame (time t132).

14 is a diagram for explaining a process of eliminating AV synchronization deviation by a process of delaying input resume of audio data by one frame.

The control for delaying the resumption of the input of the audio data is performed by correcting the AV synchronization deviation when the cumulative voice correction time (total_audio_delay) is 1 voice frame or more, as described with reference to the flow chart at the time of release of the pause in FIG. (ST112) This is realized by a process (ST113) which performs a process of delaying the resumption of the voice data by one frame until a paragraph of the frame of the voice data is found.

In FIG. 14, when the AV control unit 21 receives the pause release request from the host 1, the AV control unit 21 waits for a short circuit of the frame of the video data and detects a short circuit of the frame of the video data (time t141), and total_audio_delay. Since is over one audio frame, input of audio data is resumed after waiting for one audio frame (time t142).

As apparent from steps ST107 and ST111 in Fig. 10, when total_audio_delay is positive and does not exceed one audio frame, neither input resume of audio data / video data is delayed. In this case, the discrepancy between the audio data and the video data generated by the pause process at this time is accumulated in total_audio_delay.

However, even when AV synchronization misalignment is eliminated by delaying any of the resumption of input of the audio data / video data, the cumulative audio correction time (total_audio_delay) does not become zero by the processing of steps ST108 and ST112 in FIG. Therefore, the AV synchronization misalignment is not completely eliminated.

However, according to the AV recording apparatus according to the present invention, since the cumulative speech correction time (total_audio_delay) is always settled within one audio data frame while the AV recording apparatus is in operation, the difference is not recognized by the viewer, and it is sufficient. AV synchronization shift can be eliminated.

The present invention is applicable to an apparatus for recording or reproducing synchronously with audio data and video data.

Claims (8)

An audio / video synchronization processing apparatus for performing synchronization processing on video data and audio data having different predetermined frame lengths, respectively, Timer means, Storage means for storing a start time of each frame of the video data and audio data timed by the timer means, a time of a pause request, and a time of a pause release request; On the basis of the start time of each frame of the video data and audio data, the time of the pause request, and the time of the pause release request, which one of the video data and the audio data is delayed after the pause release request in units of frames. Or control means for determining whether or not to delay anything. The method of claim 1, The control means, When a pause request is made, an audio delay time which is a delay time of a frame of audio data is calculated on the basis of a paragraph of a frame of video data, After the pause request, each frame start time of the frame of the image data is monitored for a frame shift time which is a difference of the frame start time of the audio data with respect to the image data, Calculate a speech correction time based on the speech delay time and the frame shift time at the time of the pause release request for the pause request, On the basis of the cumulative speech correction time accumulated in the speech correction time calculated for each pause request, it is determined whether to delay one of the video data and the audio data by frame or none after the pause release request. Voice / video synchronization processing device, characterized in that. The method of claim 2, The control means, On the basis of the cumulative speech correction time, when it is determined that the audio data is preceded by the video data, the video data is delayed by one frame with respect to the audio data after the pause release request. Device. The method of claim 2, The control means, On the basis of the cumulative speech correction time, when it is determined that the audio data is delayed by one frame or more with respect to the video data, the audio data is delayed by one frame with respect to the video data after the pause release request. / Video synchronization processing device. In the audio / video synchronization processing method of performing synchronization processing on video data and audio data each having a different predetermined frame length, Calculating a voice delay time which is a delay time of a frame of audio data on the basis of a paragraph of a frame of video data at the time of a pause request; Monitoring the frame shift time which is the difference of the frame start time of the audio data with respect to the video data for each start time of each frame of the video data after the pause request; Calculating a speech correction time based on the speech delay time and the frame shift time when the pause release request is made; And determining whether to delay one of the video data and the audio data on a frame-by-frame basis or none of the video data and the audio data after the pause release request, based on the accumulated voice correction time. Audio / video synchronization processing method. The method of claim 5, On the basis of the cumulative speech correction time, when it is determined that the audio data is preceded by the video data, the video data is delayed by one frame with respect to the audio data after the pause release request. Lee way. The method of claim 5, On the basis of the cumulative speech correction time, when it is determined that the audio data is delayed by one frame or more with respect to the video data, the audio data is delayed by one frame with respect to the video data after the pause release request. / Video synchronization processing method. An audio / video recording apparatus for generating multiplexed data including video data and audio data having different predetermined frame lengths, respectively, Timer means, Storage means for storing a start time of each frame of the video data and audio data, time of a pause request, and time of a pause release request, revealed by said timer means; Synchronization control means for performing synchronization processing of the audio data after the pause release request on a frame-by-frame basis based on a start time of each frame of the video data and audio data, the time of the pause request, and the time of the pause release request; And multiplexed data generating means for generating the multiplexed data by adding time information to the video data and the audio data synchronized by the synchronization control means.

Images