JPH02228891A - Sound synchronizing method with television system conversion - Google Patents

Abstract

PURPOSE:To keep lip synchronization by respectively providing a buffer memory which can execute writing and reading for sound signal mutually independently, A/D converter and D/A converter, for which a sampling frequency can be made variable, on the side of input/output. CONSTITUTION:A buffer memory 10 is prepared to execute writing and reading for sound signal as well mutually independently with video signal processing, for which the deletion or insertion of a picture corresponding to the difference of a field frequency between two television systems is suitably executed, by using a memory unit 2 to execute writing and reading mutually independently for capacity exceeding one frame of the video. The ratio of the sampling frequency between A/D and D/A converters 9 and 11, which are provided on the side of the input/output to the buffer memory, is suitably changed and the lip synchronization with the video is obtained in the conversion of the preceding video. Thus, with following the conversion of the video between the two television systems for which the field frequency is mutually slightly different, the lip synchronization can be obtained between the unnatural video and sound.

Description

DETAILED DESCRIPTION OF THE INVENTION (Field of Industrial Application) The present invention relates to television format conversion, and more particularly to audio synchronization for converted video.

(Summary of the invention) This invention provides two
It concerns a conversion method between two television formats, using a memory device that can independently write and read a capacity of more than one frame of video to compensate for the difference in field frequency between two television formats. Along with video signal processing that appropriately deletes or inserts the corresponding screen, a buffer memory for audio signals that can be written and read independently from each other is prepared, and provided on the input/output side of the buffer memory. The sampling frequency ratio of the A/D, D/A, or D/D converter is appropriately changed to compensate for the difference between the video and the video in the previous video conversion.

In this way, it is possible to follow the conversion of video between two television systems whose field frequencies are slightly different from each other, and to achieve a smooth transition between video and audio without any unnaturalness.

(Prior Art) A method of converting between two television systems with slightly different field frequencies is disclosed in Japanese Patent Application Laid-open No. 59/1983, filed by the applicant.
-104866 ``Television format conversion method'', which allows you to convert video formats without any problems.

In FIG. 2, the configuration diagram of the configuration example of the system conversion device described in the specification of Japanese Patent Application Laid-Open No.
This shows a case where the TV (high-definition) system 1125 lines/60 fields is converted to the NTSC system 525 lines/159.94 fields.

To explain the outline of its operation with reference to FIG. 2, the input H
The DTV 1125/60 video signal is first converted into a 525/60 video signal in the number of scanning lines by an intra-field conversion circuit 15 using a known technique. This converted signal is once l
Buffer memory 16 with a memory capacity of more than a frame
NT
It is converted into a video signal of the SC525159, 94, but the timing of writing and reading is controlled by the write address control circuit 19 and the read address control circuit 2 in the figure.
0 etc.

The problem with this configuration is the 60-field video human power, 5
Since signals are gradually accumulated in the memory due to the video output of 9.94 fields, it is necessary to delete the extra signals in units of one frame at some point. This operation is performed in the scene change detection circuit 1 in this embodiment.
7 and a motion detection circuit 18, the scene change point or the still image point is detected and executed.

A more detailed explanation can be found in the specification of JP-A-Sho, but
In any case, the purpose of video conversion has been fully achieved. However, for audio, only a delay corresponding to the buffer memory (approximately one frame) was added.

In addition, as a conventional example of only audio signals, as a method to synchronize audio signals with slightly different sampling frequencies,
A method using a variable buffer and silent section detection is disclosed in TLSA Patent No. 4716472 "Variable 5peed".
replay of digital audio w
ith constant output sample
ng rate'', but this patent does not mention the connection between images and adhesives.

(Problems to be Solved by the Invention) As mentioned in the section of the prior art, when converting between two television systems with slightly different field frequencies, the video signal is converted to If the method is followed, the system will be converted without any problems, but no method has yet been proposed for audio signals that can effectively eliminate the distortion from the converted video.

SUMMARY OF THE INVENTION Therefore, it is an object of the present invention to provide an audio synchronization method for television format conversion, which can eliminate distortion from the converted video and convert the audio signal format without any unnaturalness. .

(Means for Solving the Problems) In order to achieve this object, the audio synchronization method accompanying television format conversion of the present invention performs format conversion between two television formats whose vertical scanning frequencies are slightly different from each other. A memory device capable of writing and reading a capacity exceeding one frame independently of each other is provided, and a time point at which a screen is deleted or inserted corresponding to the difference in scanning frequency between the two television systems is set as described above. A television in which the screen is arbitrarily set within the memory capacity of the memory device, and the screen is deleted or inserted when the correlation value of images between adjacent frames or fields deviates from a predetermined range. The system conversion method further includes a buffer memory that can write and read audio signals independently of each other, and an A/D converter with a variable sampling frequency on the input/output side to make writing and reading speeds variable. /
Each converter is provided with either one of an A converter or a D/D converter, and when the screen is deleted or inserted, the sampling frequency ratio of the input/output converter is changed correspondingly to maintain lip synchronization. It is characterized by the following.

(Example) The method of the present invention will be described in detail below with reference to the accompanying drawings and examples.

FIG. 1 shows a block diagram of the configuration of an embodiment of an apparatus for carrying out the method of the present invention.
The illustrated conventional configuration and the video processing portion overlap. The video buffer memory 2 shown in the first figure corresponds to the buffer memory 16 shown in the second figure, and in the first figure, an A/D converter 1 and a D/A converter 3 are provided on the input and output sides, respectively. Also, the analog video human input signal to this A/D converter 1 is HDTV.
1125/60 is a signal whose scanning line number has already been converted to 525/60.

It is also assumed that the scene change detection circuit 17, motion detection circuit 18, write address control circuit 19, read address control circuit 20, etc. shown in the second figure are incorporated into the control circuit 5 shown in the first figure together with audio signal control. . For audio signals, an audio buffer memory 10 and an A/D converter 9 and a D/A converter 11 are provided on its input and output sides, respectively.

The field frequency of high-definition signals is 60.00.
It is specified as Hz. Therefore, from high definition to NTS
To convert to C format, change the number of scanning lines from 1125 to 525.
Convert the field frequency to 60,00H
z to 59.94Hz.

To do this, the video signal arriving at 60.0 Hz is once written to the buffer memory 2 according to this synchronization, and then
If this is read out in accordance with 9.94 Hz synchronization, it is possible to perform system conversion. However, in the case of this embodiment, the writing speed is faster and the continuous output speed is slower, so the amount of information stored in the buffer memory 2 gradually increases, and if left as is, the buffer memory 2 will eventually Memory overflows. To this end, the information in the buffer is reduced by finding the point at which the entire screen is still and the point at which the scene changes, and then deleting the frame.

This will be done once every 33 seconds on average. If the material to be converted includes many scene change points, still images, etc., deletion will be performed once every 33 seconds on average, and no problems will occur. However, if the material is a moving game using only one camera, the material contains almost no still images or scene change points, and the buffer memory is easy to store. It overflows. At this time, even if the image contains movement, the frame must be forcibly deleted, and the converted image will have some movement, which will be a nuisance.

Therefore, it is conceivable to increase the capacity of the buffer memory 2. At this time, if scene change points do not appear often, more and more image information is stored in the buffer memory 2, causing a delay in the output image. At this time, if you focus only on the video, there will be no unnatural feeling at all, but if you do not take any measures other than adding a delay amount to the audio that corresponds to the memory capacity described in the previous example section, Since this progresses relative to the image, it gives a very strange feeling. The present invention aims to improve this point.

That is, like the video, the audio signal is converted into a digital signal by the A/D converter 9 and then temporarily stored in the audio buffer memory 0. Then read again D/A converter 1
1 to convert it into an analog output, or connect it as a digital signal to the signal input for a digital VTR. Here we assume that the read clock is constant, but
The same objective can also be achieved by slightly changing the clock on the D/A conversion side.

Regarding audio, the sampling frequency for A/D is also D/A.
Since the sampling frequencies for both are basically the same, in this case, sampling A/D conversion for A/D conversion is performed.

In this case, the number of samples of the audio signal increases, so the amount of audio information stored in the audio buffer memory increases. That is, a delay occurs in the audio signal output after D/A conversion (this degree of change in pitch is below the detection limit and does not give an unnatural feeling). In this way, the audio signal can have a delay commensurate with the delayed video signal, and as a result, lip synchronization can be achieved.

Even if there is no point in the image signal over several minutes at which frame deletion can be performed, such as at a scene change point, it is expected that there will come a point at which frame deletion can be performed at some point thereafter. If frame deletion is performed at this time, the image information will be advanced by one frame, and as a result, the audio signal will be relatively delayed. Therefore, the amount of information in the buffer 10 decreases. In other words, the audio progresses relatively, and lip syncing becomes possible again.

In this way, lip sync can be achieved by slightly changing the sampling frequency applied to the audio A/D converter 9, but if a sudden change is applied to this clock, the final audio output will be affected. Since there is a possibility that a change in the clock may be detected, it is desirable to change this clock over a period of about 1 to 5 seconds.

Control necessary for these is performed by the control circuit 5.

Conversely, when converting from 59.94Hz to 60.00Hz, in terms of video, first input information in advance until the buffer is full, then convert to 60.00Hz.
The video is read out according to Hz synchronization. In this case, by finding the point in the still image and repeating the frame (
It is necessary to increase the number of frames by supplementing frames), but if the timing that meets this purpose does not arrive soon, the information stored in memory will gradually become less and less.

In other words, as the video progresses relative to the audio, the jitters disappear. At this time, similarly, the control circuit 5
By controlling the sampling clock of the /D converter 9 or the D/A converter 11, it is possible to realize a delay of the audio signal commensurate with the amount of delay of the video signal.

Also, the relative delay between audio and video becomes a very noticeable disturbance when audio takes precedence. Also, if the audio is delayed too much, it will feel unnatural. Therefore, in this case, the advance of the audio by one frame is set as the advance limit, and the delay of two frames is set as the delay limit. If the sound is controlled so that it is within this limit, no practical problem will occur.

It is also possible to replace the A/D and D/A converters of this device with an all-digital system.

Although the embodiments of the method of the present invention have been described in detail above,
The present invention is not limited to this embodiment, and changes and modifications within the scope of the claims of the present invention are obvious.

(Effects of the Invention) The audio synchronization method of the present invention has been described in detail above. According to the method of the present invention, the capacity exceeds one frame when converting between two television systems with slightly different vertical scanning frequencies. Even with a conversion method that converts the video signal through the memory of the video and deletes or inserts the image frame using the scene change or still image point in the video, it is possible to convert the video signal into a clear image of the audio signal. Simultaneous format conversion with the signal can be performed, and even if the capacity of the video memory is large, the distortion will not be lost, and the unnaturalness that conventionally existed between the video and audio can be removed.

[Brief explanation of the drawing]

FIGS. 1 and 2 are block diagrams of an embodiment according to the present invention and a conventional example, respectively. 1.9...A/D converter 2...Buffer memory for video 3.11...D/A converter 4,6...PLL circuit 5...Control circuit 7...Audio sampling Frequency converter 8...Countdown circuit 10...Audio buffer memory 15...In-field conversion circuit 16...Buffer memory 17...Scene change detection circuit 18...Motion detection circuit 9...・Write address control circuit 20...Read address control circuit Fig. 1

Claims (1)

[Scope of Claims] 1. A memory device capable of format conversion between two television formats with slightly different vertical scanning frequencies, and writing and reading of a capacity exceeding one frame of video signals independently of each other. and arbitrarily setting a time point at which to delete or insert a screen corresponding to the difference in scanning frequency between the two television systems within the memory capacity of the memory device,
In a television format conversion system in which the screen is deleted or inserted when the correlation value of images between adjacent frames or fields is out of a predetermined range, the writing and reading for audio signals are further mutually performed. A buffer memory that can be operated independently, and either an A/D converter, a D/A converter, or a D/D converter with a variable sampling frequency to make writing and reading speeds variable on the input/output side. A method for converting television formats, characterized in that each converter is provided, and when the screen is deleted or inserted, the sampling frequency ratio of the input/output converter is changed correspondingly to maintain lip sync. Audio sync method. 2. The synchronization method according to claim 1, wherein the time when the correlation value of the images between the adjacent frames or fields is out of a predetermined range is determined as the time when a scene change or a still image state is detected. Audio synchronization method accompanying television format conversion. 3. In the synchronization method according to claim 1 or 2, when changing the sampling frequency ratio, the sampling frequency of the converter on the output side of the buffer memory is fixed, and the sampling frequency of the converter on the input side of the buffer memory is fixed. An audio synchronization method accompanying television format conversion characterized by changing the sampling frequency. 4. In the synchronization method according to any one of claims 1 to 3, when changing the ratio of the sampling frequencies, the change is caused to occur in a time represented by the reciprocal of the frame vertical scanning frequency difference between the two slightly different television systems. A method of audio synchronization associated with television format conversion, characterized in that the synchronization is performed gradually over a period of less than half the time. 5. The synchronization method according to any one of claims 1 to 4, wherein the converter on the input side of the buffer memory is an A/D converter, and the converter on the output side is a D/A converter or a D/D converter. An audio synchronization method accompanying television format conversion, characterized in that: 6. In the synchronization method according to any one of claims 1 to 5, lip synchronization of the audio to the video is performed within one frame of the video when the audio signal advances relative to the video signal; An audio synchronization method associated with television format conversion, characterized in that the audio synchronization is within two frames of video if it is stored.