WO1998011550A1

WO1998011550A1 - Device for mixing an audio sequence with a video sequence

Info

Publication number: WO1998011550A1
Application number: PCT/FR1997/001599
Authority: WO
Inventors: Christophe Gérard Irénée CARNIEL; Pierre Alfred Keiflin
Original assignee: Netia
Priority date: 1996-09-10
Filing date: 1997-09-10
Publication date: 1998-03-19
Also published as: AU4213097A; FR2753329B1; FR2753329A1

Abstract

This device comprises means (A1, A2,.....An) for acquiring in real time said video sequence in digitised and compressed form; at least one computer (S) and a digital data medium (17) for storing said digitised and compressed video sequence, and at least one mixing station (M1, M2,.....Mp) connected to said computer and comprising: means (9) for decompressing and converting to analog said digitised and compressed video sequence, means for displaying said video sequence, means for acquiring said audio sequence, means for digitising and compressing in real time said audio sequence, and means (R2) for transmitting to the computer (S) the digitised and compressed audio sound for storing on said medium (17) associated with said digitised and compressed video sequence.

Description

Device for mixing a sound sequence with a video sequence.

The present invention relates to a device for mixing at least one sound sequence with a video sequence, possibly accompanied by an already recorded sound.

There are a number of areas such as, for example, the design of multilingual television programs or the production of dubbing films for the cinema, in which it is necessary to add a sound sequence, hereinafter called sound commentary, on a video sequence which is possibly accompanied by a sound sequence already recorded, hereinafter called ambient sound.

For example, a television channel which broadcasts a program in several countries with different languages must be able to broadcast in each country this program in the language used there: in Europe, a channel of television may be required to broadcast the same program in English in England, in French in France, in German in Germany, in Italian in Italy, etc. Known mixing devices used today to a source VCR to play a magnetic tape or cassette on which the video sequence and the ambient sound are recorded, a video monitor to visualize the video sequence recorded on the tape, a microphone allowing an operator to comment on the visualized video sequence on the monitor, a two-way audio mixer that mixes the background sound recorded on the tape and the audio commentary from the microphone, an audio speaker to reproduce the mixed sound thus obtained, and a destination VCR containing a magnetic cassette on which the video sequence and the ambient sound mixed with the audio commentary are recorded. With such a known mixing device, when it is necessary to make, from the same original video sequence, several tapes or cassettes with different sound comments, for example in several languages, it is essential to make several copies of the original cassette. Of course, this need to reproduce the original cassette in several copies is detrimental to the speed of the production process, is a source of disorganization in the event of tape loss and puts a strain on operating costs. In addition, when an operator makes an error in his audio commentary, he is obliged to repeat the operation from the start by resetting the source video recorder to the beginning of the cassette. It is impossible for him to resume part of the audio commentary at an intermediate point in the recorded video sequence.

The invention aims to provide a device for mixing at least one sound sequence with a video sequence which overcomes the drawbacks of the prior art mixing devices. To this end, the subject of the invention is a device for mixing at least one sound sequence with a video sequence, characterized in that it comprises:

means of real-time acquisition of said video sequence in digitized and compressed form, at least one computer connected to said acquisition means and adapted to receive in real time said digitized and compressed video sequence,

- at least one digital information medium associated with said computer for storing said digitized and compressed video sequence, and at least one mixing station connected to said computer and comprising: * means for decompressing and converting said digitized and compressed video sequence transmitted by said computer from said information medium into analog form, * display means for producing a visual representation of said video sequence,

* means for acquiring said sound sequence, in synchronism with said visual representation of said video sequence, * means for digitizing and compressing said sound sequence in real time, and

* means for transmitting said digitized and compressed sound sequence to said computer for storage on said information medium in association with said digitized and compressed video sequence.

If said video sequence to be mixed is associated with an audio sequence and with synchronization data between said video and audio sequences, said device comprises means for real-time acquisition of said audio sequence in digitized and compressed form and said synchronization data , said computer and said information medium are adapted to receive and store in real time said digitized and compressed audio sequence and said synchronization data, and said mixing station comprises means for decompressing and converting into analog form said audio sequence transmitted by said computer from said information medium, means for mixing said audio sequence with said sound sequence acquired in synchronism with said visual representation of said video sequence, and means for applying said audio sequence mixed with said sound sequence to said digitizing means e t sound sequence compression.

Preferably, said means for acquiring said sound sequence comprise at least one microphone and said means for mixing said audio and sound sequences comprise a mixer connected at input to said means decompressing and converting said audio sequence and said microphone to analog form and at output to said means for real-time digitization and compression of sound sequence. Preferably, said mixing station comprises an electro-acoustic transducer connected to the output of said mixer. Said mixing station is constituted, for example, by a microcomputer comprising a display screen.

When said video sequence and said associated audio sequence consist of analog signals, said means of acquiring said video and audio sequences comprise at least one acquisition station comprising means of analog / digital conversion and of compression of said video sequence and means for analog / digital conversion and compression of said associated audio sequence. Preferably, said acquisition station comprises a microcomputer comprising a screen for viewing said video sequence and an electro-acoustic transducer for hearing said associated audio sequence. If said video sequence and said associated audio sequence are recorded on a magnetic tape, said acquisition station comprises a video recorder for reading said tape.

According to a characteristic of the invention, the mixing device comprises several acquisition stations connected by a digital computer network to said computer.

According to another characteristic of the invention, the mixing device comprises several mixing stations connected by a digital computer network to said computer.

Preferably, said computer constitutes the server of said network.

Other characteristics and advantages of the invention will emerge from the description which follows of an embodiment given solely by way of example and illustrated by the appended drawings in which:

Figure 1 is a block diagram of a device for mixing according to the invention;

FIG. 2 is a diagram of a video sequence and ambient sound acquisition station;

Figure 3 is a diagram of a sound mixing station; and FIG. 4 is a time diagram illustrating the progress over time of the information transmission processes between the different stations of the device represented in FIG. 1.

The mixing device of FIG. 1 comprises a certain number of stations A _x , A ₂ , A „for acquiring video sequence and ambient sound connected by a digital computer network R-, to a sound and video server S In the example shown, the digital computer network R-, includes a digital switch C-, to which the acquisition stations A _lf A ₂ , A ,, are connected in a star by lines L _l L ₂ , L _n thus that a common line L _C1 between the digital switch C _j and the server S.

The mixing device also includes a certain number of mixing stations M _1; M ₂ , M-, connected by a second digital computer network R ₂ to the server S. In the embodiment of FIG. 1, the network R ₂ has the same structure as the network R _λ and includes a digital switch C ₂ to which the mixing stations M _lf M ₂ , M _p are connected in star by digital data transmission lines l _x , 1 ₂ , l _p , as well as a common digital data transmission line L _C2 connected between the switch digital C ₂ and the server S.

Preferably, the networks R-_ and R ₂ constitute subsets of the same digital computer network managed by the server S.

FIG. 2 illustrates the structure of the stations A _{l #} A ₂ , A „for acquiring video sequence and ambient sound. Such a station A comprises a source 1 adapted to generate on its outputs 2, 3 and 4 respectively the analog audio signal, the analog video signal and the synchronization signal relating to the video sequence and to the ambient sound which should be mix with a comment sound.

As shown in FIG. 2, the source 1 can be constituted, for example, by a source video recorder for reading a magnetic tape or cassette on which the video sequence and the ambient sound are recorded in the form of analog signals. As a variant, if the sound comment is to be added live or in very slight delay to a video sequence with ambient sound received by a television station forming part of an analog transmission network, the source 1 can be constituted by the head of the television station.

Station A also includes a computer 5, such as a microcomputer or a workstation, connected to outputs 2, 3 and 4 of source 1 to receive the analog audio signal, the analog video signal and the signal synchronization.

The computer 5 comprises an electronic card or circuit for digitizing and compressing the analog video signal, an electronic card for digitizing and compressing the analog audio signal and an electronic card for synchronizing the video data and ambient sound from the signal synchronization received from the output 4 of the source 1. The computer 5 also includes a so-called network electronic card allowing the latter to transmit to the server S and to receive digital data therefrom via the digital computer network R _x .

Station A is completed by a computer monitor 6 making it possible to view in a window 7 the images corresponding to the video signal received from output 3 of source 1, as well as an electro-acoustic transducer 8 such as a speaker connected to the output 2 of source 1 to reproduce the ambient sound associated with the video signal present at output 3 of source 1. Of course, computer 5 includes software for indexing the element to be recorded and for scanning the sound and video.

As a variant, the source 1 can be constituted by the headend of a television station forming part of a digital transmission network on which video and audio data are transmitted in digital and compressed form. In this case, the computer 5 of the acquisition station A connected to the digital network head (source 1) is programmed, on the one hand to transmit directly to the server S via the computer network R _λ the video data and its d received in digitized and compressed form and the associated synchronization data, on the other hand to decompress and convert to analog form said video and audio data, view the video in window 7 of the monitor 6, and listen to the ambient sound from enclosure 8. Decompression and digital / analog conversion of video and audio data is ensured by the aforementioned electronic cards which, conventionally, can operate selectively in compression or decompression, as well as in digital / analog conversion or in analog / digital conversion, depending on an appropriate programming of the computer they equip.

The structure of the mixing stations M _lf M ₂ , M _p of the device of FIG. 1 is represented in FIG. 3. The mixing station M comprises a computer 9, such as a microcomputer or a workstation, connected to the server S via the network R ₂ and associated with a computer monitor 10 making it possible to display in a window 11 the images corresponding to the video signal received from the server S. The ambient sound received from the server S is applied by an output 12 of the computer 9 to one of the inputs of a two-way audio mixer 13, the other input of which is connected to a microphone 14 by means of which an operator can comment on the images displayed in the window 11. The output 15 of the mixer 13 is applied, on the one hand to an input of the computer 9 and, on the other hand, to an electro-acoustic transducer such as a pregnant 16.

The computer 9 includes an electronic card called network to ensure the transmission of information to and from the server S via the network R ₂ , a decompression and digital / analog conversion card of the video data coming from the server S in order to display the corresponding video sequence in window 11 of the monitor 10, a decompression and digital / analog audio conversion card, in order to apply the ambient sound to one of the inputs of the mixer 13, and of digitization and audio compression, in order to transmit to the server S in the form digitized and compressed the ambient sound mixed with the commentary produced at the output of the two-way mixer 13, and a synchronization card in order to ensure synchronization between the digitized and compressed video and audio data transmitted by the computer 9 to the server S via the R ₂ network.

Finally, the computer 9 comprises inter alia software for viewing the video, listening to the ambient sound and simultaneously recording the ambient sound mixed with the commentary. In operation, if the video and ambient sound sequence to be mixed with a commentary is recorded on a cassette or magnetic tape, an operator places it in the source VCR 1 of one of the stations 'acquisition A _lf A ₂ , A ^ which is then available. Preferably, conventional software installed in the computer 5 makes it possible to lock the cassette at the start of the recording. The computer 5 ensures the real-time digitization and compression of the sound and video information coming from the source video recorder 1 and their real-time transmission in digitized and compressed form on one or more mass media 17 associated with the server S. Simultaneously, during the playback of the video and sound cassette, the video is displayed in window 7 of the monitor 6 and the sound is heard by means of the speaker 8.

To mix a comment with a video sequence and given ambient sound, the operator selects this sequence from among those stored in the server S from one of the M _ιr mixing _stations

M ₂ , M _p "mistletoe is then available. The corresponding video and ambient sound data are transmitted to this station M which ensures their decompression and their conversion into analog form so as to display the video sequence in the window 11 of the monitor 10 and to apply the audio-analog signal to the mixer 13 Simultaneously, via the microphone 14, the operator makes his comment by visually synchronizing himself with the video displayed in the window 11.

The ambient sound from the server S, decompressed and converted into analog form by the computer 9, as well as the sound commentary applied to the microphone 14, are applied to the two respective inputs of the two-way audio mixer 13. The output of the mixer 13

(mixed sound) is audible by means of the enclosure 16 and it is on the other hand digitized and compressed in real time by the computer 9, from which it is transmitted via the network R ₂ to the server S where it is stored in real time on the mass support 17.

Using conventional software and hardware (audio and video compression / decompression cards, digital / analog or analog / digital audio and video conversion cards, network card), the M ₁ mixing stations _; M ₂ , M _p allow simultaneously, in real time:

- to carry out the digital / analog conversion and the decompression of the ambient sound and the video coming from the mass support 17 of the server S; and

- to digitize and compress in real time the mixed sound coming from the voice mixer 13, then to transmit it in real time on the mass support 17 of the server S. Conventionally, the server S distributes the video and ambient sound data in real time via the network R ₂ to the sound mixing stations M-,, M ₂ , M _p , and simultaneously receives from these, via the same network R ₂ the mixed sound data in digitized and compressed form and always stores them in real time on the same information medium 17. The server S can also simultaneously receive accuuisition stations A _lt A ₂ , A „, via the network R _{1 #} of video and ambient sound data in digital and compressed form.

Thus, several of the acquisition stations A _j , A ₂ , A. can operate simultaneously on the digital computer network R _λ and on the computer server S. The synchronization card installed in the acquisition stations A generates a signal standard clock which is used as a reference to control the source video recorder associated with this station, the video stream and the sound stream.

Likewise, several of the mixing stations M _{1 (} M ₂ , M _p can operate simultaneously with other mixing stations M and other acquisition stations A on the same computer network R _t R ₂ and on the same server IT S.

In other words, the same video and surround sound information stored on the server S can be played simultaneously by several mixing stations M and it is not necessary to make several copies during recording. As soon as ambient sound and video information has started to be recorded on the server S, it is accessible by all the mixing stations M _{1 #} M ₂ , Mp and there is no need to wait for the end of recording to start mixing. This process is illustrated in the time diagram of FIG. 4 in which the references F designate video and ambient sound data streams directed from the acquisition stations A-,, A ₂ , A „to the server S, thus as data streams comprising simultaneously a stream of video and ambient sound data directed from the server S to the mixing stations M _{1 #} M ₂ , M _p and a data stream of its mix directed from the mixing stations M _lf M ₂ , M _p to the server S.

More specifically, the reference F _: designates a data stream initiated at time t ₀ and corresponding to the digitization and compression on an acquisition station A _α of a first video sequence and ambient sound VS _α , thus cgu 'to the transmission of this data on the network R _: and to their storage on the information medium of the server S.

The flow F ₂ initiated at time t-. corresponds, on the one hand, to reception from the server S via the network R ₂ , to decompression and digital / analog conversion on a mixing station M _κ of the sequence VS ₁ , on the other hand to digitization , compression and transmission to the server S via the network R ₂ of a first mixed sound element SM _n associated with the video element of the sequence VS-,.

The streams F ₃ and F ₄ initiated at times t ₃ and t ₆ respectively, are similar to the stream F ₂ and also relate to the ambient video and sound sequence VSj _. : the stream F ₃ relates to a mixed sound element SM ₁₂ generated on a mixing station M _L , and the stream F ₄ to a mixed sound element SM ₁₃ generated on a mixing station M _M.

The flow F _s initiated at time t ₂ is similar to the flow F-, and relates to the acquisition from an acquisition station A _j of a second video sequence and ambient sound VS ₂ . Finally, the flows F ₆ and F ₇ , initiated at times t ₄ and t _s respectively, are similar to the flows F ₂ , F ₃ and F ₄ , but relate to the video sequence and ambient sound VS ₂ : the flow F ₆ a relates to a mixed sound element SM ₂₁ generated by a mixing station M _N , and the stream F ₇ to a mixed sound element SM ₂₂ generated by a mixing station.

In FIG. 4, t _m represents the minimum time, typically two seconds, which necessarily separates the start of the recording by the server S of a video sequence and ambient sound from the start of the reading of this sequence by a station. mixer M.

The mixing device according to the invention, which ensures fully digital processing of video and sound data makes it possible, notably in the case of multilingual television channels, to significantly reduce the production costs associated with mixing commentary in the different languages and to significantly reduce the time required for these mixing operations. In addition, the result of the mixing operation is a computer file and it is therefore possible for an operator to make repeats of his comment when he has made an error, without having to start the complete operation from the beginning of the sequence to comment.

By way of example, the software and electronic cards for digitization and sound and video compression currently existing make it possible to ensure a compressed stereo sound stream with a bit rate of 32 kilobytes / second and a compressed video stream with a bit rate of 420 kilobytes / second. At the same time, the throughput of a digital computer network is typically 100 megabits / second theoretical and the throughput of a computer mass support (magnetic disk for example) of 5 megabytes theoretical.

Claims

1. Device for mixing at least one sound sequence with a video sequence, characterized in that it comprises:

means (A _{1 (} A ₂ A of real-time acquisition of said video sequence in digitized and compressed form,

- at least one computer (S) connected to said acquisition means and adapted to receive in real time said digitized and compressed video sequence, - at least one digital information medium (17) associated with said computer for storing said digitized video sequence and compressed, and

- at least one mixing station (M _{1 (} M ₂ , M _p ) connected to said computer and comprising: * means (9) for decompressing and converting said digitized and compressed video sequence transmitted by said computer from said support into analog form information,

* display means (10) for producing a visual representation of said video sequence,

* means (14) for acquiring said sound sequence, in synchronism with said visual representation of said video sequence,

* means (9) for digitizing and compressing said sound sequence in real time, and

* means (9, R ₂ ) for transmitting to said computer (S) of said digitized and compressed sound sequence for its storage on said information medium (17) in association with said digitized and compressed video sequence.

2. Device according to claim 1, in which said video sequence to be mixed is associated with an audio sequence and with synchronization data between said video and audio sequences, characterized in that said device comprises means (A _{1 #} A ₂ , A real-time acquisition of said audio sequence under digitized and compressed form and said synchronization data, said computer (S) and said information medium (17) are adapted to receive and store in real time said digitized and compressed audio sequence and said synchronization data, and said mixing station (M _{1 #} M ₂ , M _p ) comprises means (2) for decompressing and converting into analog form said audio sequence transmitted by said computer from said information medium, means for mixing said audio sequence with said acquired sound sequence in synchronism with said visual representation of said video sequence, and means for applying said audio sequence mixed with said sound sequence to said means (9) for digitizing and compressing sound sequence.

3. Device according to claim 2, characterized in that said means for acquiring said sound sequence comprise at least one microphone (14) and said means for mixing said audio and sound sequences comprise a mixer (13) connected at input to said means (9 ) decompressing and converting said audio sequence and said microphone (14) into analog form and at output to said means (9) for real-time digitization and compression of sound sequence.

4. Device according to claim 3, characterized in that said mixing station (M) comprises an electro-acoustic organ transducer (16) connected to the output of said mixer (13).

5. Device according to any one of claims 2 to 4, characterized in that said mixing station (M) consists of a microcomputer (9) comprising a display screen (10).

6. Device according to any one of claims 2 to 5, in which said video sequence and said associated audio sequence consist of analog signals, characterized in that said means for acquiring said video and audio sequences comprise at least one station accguisition (A) comprising means (5) for analog / digital conversion and compression of said video sequence and means (5) for analog / digital conversion and compression of said associated audio sequence.

7. Device according to claim 6, characterized in that said acquisition station (A) comprises a microcomputer (5) comprising a screen (6) for viewing said video sequence and an electro-acoustic hearing transducer (8) of said associated audio sequence.

8. Device according to any one of claims 6 and 7, wherein said video sequence and said associated audio sequence are recorded on a magnetic tape, characterized in that said acquisition station (A) comprises a video recorder (1) of reading of said tape.

9. Device according to any one of claims 6 to 8, characterized in that it comprises several acquisition stations (A ^ A ₂ , A _p ) connected by a digital computer network (R ₂ ) to said computer (S) .

10. Device according to any one of claims l to 9, characterized in that it comprises several mixing stations (MM ₂ , M _n) connected by a digital computer network (R to said computer (S).

11. Device according to any one of claims 9 and 10, characterized in that said computer constitutes the server (S) of said network (R _{1 (} R ₂ ).