FR3111497A1 - A method of managing the reproduction of multimedia content on reproduction devices. - Google Patents

Abstract

Method for managing the reproduction of multimedia content on reproduction devices. The invention relates to a method for managing the reproduction of multimedia content comprising images and sound reproduced on respective reproduction devices (ECR, ENC), characterized in that it comprises a transmission step from a device for reading multimedia content (DEC) of an audio signal with a view to restitution; reception, by the multimedia content playback device, of data related to the duration of transmission of the audio signal from the multimedia content playback device at least to the audio playback device (ENC); obtaining a time lag to be applied between the instants of transmission of images and associated audio signals during subsequent reading of a content, the time lag being a function of the transmission time. Figure 1

Description

Method for managing the reproduction of multimedia content on reproduction devices.

The invention relates to the field of telecommunications.

The invention relates to a method for managing the reproduction of multimedia content on several reproduction devices.

A multimedia content playback device transmits content to the rendering devices. The multimedia content playback device is either a decoder receiving content from a communication network, or a device capable of playing content in a memory internal to the device or on a memory external to the device such as a DVD disc or a memory Flash-like.

The multimedia content comprises at least two interlinked parts, namely images and associated sound, the restitution of which requires synchronization.

The playback devices are arbitrary, namely a smartphone, a tablet, a computer, a television, a speaker, etc.

State of the art

Traditionally, rendering content consists of transmitting content to one or more rendering devices. When multimedia content includes both images and associated sound, traditionally, this content is transmitted from a playback device, such as a decoder, to a playback device equipped with a screen and an integrated loudspeaker. (or internal).

Today, external speakers also called connected speakers are widely used because of their acoustic performance. The connected speakers carry out acoustic processing capable of restoring high quality sound (surround effect, etc.); users increasingly favor the use of connected speakers in addition to or instead of the speaker built into the television.

When a connected speaker is used, the decoder transmits the multimedia content to two separate reproduction devices, namely to an image reproduction device, for example a screen and/or a speaker integrated into the device, and to a separate connected speaker of the restitution device.

The performance of a connected speaker is linked to computer processing carried out by a processor internal to the speaker. Depending on the loudspeaker used, the duration of the processing can exceed 80 milliseconds and therefore leads to a time lag between the restored image and the associated sound which are no longer synchronous.

In addition, to the processing delays is added a signal transmission delay on the communication link which separates the decoder and the enclosure; this is particularly the case when the link is a Bluetooth link.

Ultimately the overall transmission delay of an audio signal is such that the user experience is disastrous; it will then tend to use the connected speakers only to listen to audio content and not audio-video content

The invention improves the situation.

To this end, according to a functional aspect, the invention relates to a method for managing the reproduction of multimedia content comprising images and sound reproduced on respective reproduction devices, characterized in that it comprises the steps following

a step of transmitting from a multimedia content playback device an audio signal for playback;

a step of reception, by the multimedia content playback device, of data related to the duration of transmission of the audio signal from the multimedia content playback device at least to the audio reproduction device;

a step of obtaining a time lag to be applied between the instants of transmission of images and associated audio signals during subsequent reading of a content, the time lag being a function of the transmission time.

The method of the invention comprises a prior step of obtaining an audio signal transmission time from the multimedia content playback device, for example a digital television decoder, and an audio signal receiver, for example a microphone . Once this transmission duration has been obtained, the images associated with the audio signals are transmitted with transmission times delayed with respect to the transmission times of the audio signals by a duration more or less equal to the transmission duration obtained. This results in synchronized reception of images and audio signals despite the longer transmission delay of sound compared to images.

It should be specified here that the expression “at least up to the audio reproduction device” means “up to the audio reproduction device or beyond”. It will indeed be seen, later in the description, that the sound transmission delays in the air are negligible compared to the signal transmission delays from the playback device and the audio reproduction device.

According to a first particular mode of implementation of the invention, the shift duration is calculated again following a hardware and/or software modification of the audio reproduction device. The modification of the audio reproduction device, for example an update of the sound processing software, can have an influence on the duration of transmission of the audio signals between the multimedia content playback device and the receiver device; this mode allows the duration to be readjusted according to the new configuration.

According to yet another second particular mode of implementation of the invention, which may be implemented alternatively or cumulatively with the previous one, the shift duration is calculated again following an addition or deletion of equipment on the link communication of audio signals between the multimedia content playback device and the audio playback device. As for the first mode, this second mode recalculates the transmission time if a device is added or deleted on the communication link between the multimedia content playback device and the audio reproduction device.

According to yet another third particular mode of implementation of the invention, which may be implemented alternatively or cumulatively with the previous modes, the shift duration is stored, and transmitted to a multimedia content playback device as soon as the audio playback connects to the multimedia content playback device.

According to yet another fourth particular mode of implementation of the invention, which may be implemented alternatively or cumulatively with the previous modes, the transmission duration DL is stored and is ideally stored in a read only memory in the restitution device. In this way, as soon as a connection is established with the rendering device, the latter transmits the calculated transmission delay; this prevents the multimedia content playback device from calculating the transmission time. The multimedia content playback device applies the shift as soon as a restitution of a content is requested.

It should be noted here that a lag below a DC duration of around 80 milliseconds between an image and an associated audio signal is imperceptible to a user viewing the content. In this configuration, according to yet another fifth particular mode of implementation of the invention, which may be implemented alternatively or cumulatively with the previous modes, by denoting DC the value of the time lag between two imperceptible restitutions, the delay applied to the instants of transmission of the images is chosen in the interval [t2-t1-80 milliseconds; t2-t1+80milliseconds]. This mode offers a temporal margin which will not affect the quality of the restitution.

According to a material aspect, the invention relates to a management entity, called first entity, able to manage the restitution of multimedia content comprising images and sound reproduced on respective restitution devices, characterized in that the entity includes

a transmission module capable of transmitting from a multimedia content playback device an audio signal for playback;

a reception module capable of receiving data related to the duration of transmission of the audio signal from the multimedia content playback device at least to the audio reproduction device;

a module for obtaining capable of obtaining a time lag to be applied between the instants of transmission of images and associated audio signals during subsequent reading of a content, the time lag being a function of the transmission time .

According to another material aspect, the invention relates to a decoder including a first management entity as defined above.

According to another material aspect, the invention relates to a computer program able to be implemented by the first management entity as defined above, the program comprising code instructions which, when it is executed by a processor, carries out the steps of the method defined above.

Finally, the invention relates to a data medium on which has been stored at least one series of program code instructions for the execution of the method defined above.

According to another functional aspect, the invention relates to a method for managing the reception of audio signals from a multimedia content playback device, characterized in that it comprises

a step of receiving an audio signal;

a time-stamping step of the received audio signal;

a step of transmitting, to the multimedia content playback device, data related to the timestamp produced.

According to another material aspect, the invention relates to a second entity for managing the reception of audio signals from a multimedia content playback device, characterized in that it comprises

a module for receiving an audio signal;

a time-stamping module for the received audio signal;

a transmission module, intended for the multimedia content playback device, of data linked to the timestamp produced.

According to another material aspect, the invention relates to a speaker capable of receiving audio signals to be restored, comprising a second entity as defined above.

According to another material aspect, the invention relates to a computer program capable of being implemented by the second entity as defined above, the program comprising code instructions which, when it is executed by a processor, carries out the steps of the method defined above, namely:

a step of receiving an audio signal;

a time-stamping step of the received audio signal;

a step of transmitting, to the multimedia content playback device, data related to the timestamp produced.

According to another material aspect, the invention relates to a data medium on which has been stored at least one series of program code instructions for the execution of the steps defined above, namely: an audio signal;

a time-stamping step of the received audio signal;

a step of transmitting, to the multimedia content playback device, data related to the timestamp produced.

Note that a data carrier can be any entity or device capable of storing the program. For example, the medium may comprise a storage medium, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or else a magnetic recording medium such as a hard disk. On the other hand, the information medium can be a transmissible medium such as an electrical or optical signal, which can be conveyed via an electrical or optical cable, by radio or by other means. The program according to the invention can in particular be downloaded from an Internet-type network. Alternatively, the information carrier may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of the method in question.

Finally, it should be noted here that, in this text, the term "module" or "entity" can correspond both to a software component and to a hardware component or a set of hardware and software components, a corresponding software component itself to one or more computer programs or sub-programs or more generally to any element of a program capable of implementing a function or a set of functions as described for the modules concerned. In the same way, a hardware component corresponds to any element of a hardware (or hardware) assembly capable of implementing a function or a set of functions for the module concerned (integrated circuit, smart card, memory card, etc. .).

The invention will be better understood on reading the following description, given by way of example and made with reference to the appended drawings in which:

FIG. 1 represents a computer system on which an exemplary embodiment of the invention is illustrated.

FIG. 2 is a schematic view of the architecture of a decoder according to one embodiment of the invention.

FIG. 3 is a schematic view of the architecture of a remote control with integrated microphone according to one embodiment of the invention.

FIG. 4 illustrates, according to a particular embodiment of the invention, the transmission of sound, and various possible operating modes for capturing sound.

FIG. 5 illustrates an operating mode chosen for the capture according to one embodiment.

FIG. 6 illustrates the phase for obtaining the time lag to be applied between the instants of transmission of the images and of the associated sound so as to ensure synchronized video and sound.

FIG. 7 illustrates a variant in which the offset duration is chosen within a time interval without altering the synchronization between the video and the sound.

Detailed description of one (or more) embodiment(s) illustrating the invention

Figure 1 represents a computer system SYS including

• A multimedia content playback device illustrated by means of a decoder DEC capable of communicating with a multimedia content server through a communication network RES. Multimedia contents include images and audio signals;
• A first ECR image restitution device, illustrated by means of a television screen, connected to the decoder;
• A second reproduction device ENC able to reproduce sound. This second device is a connected speaker in our example. This second device could also be a wireless headset;
• A microphone MIC.

In our example the screen is connected in wired mode with the decoder for example via an HDMI connection.
The speaker is connected to the decoder. The invention applies to any kind of connection but is particularly aimed at a wireless connection between the decoder and the ENC enclosure. In our example, the connection is a Bluetooth BT connection.

The remote control also communicates via a Bluetooth BT connection.

The microphone is ideally placed near a user; in our example, the microphone is placed in a TCD remote control.

In our example, the DEC decoder is connected to a GTW home gateway. Recall that a home gateway has the function of interconnecting a first network, for example a local network of the Wi-Fi type, and the network RES defined above which can be for example an extended network of the optical fiber type, ADSL, etc. . Access to the communication network RES is provided by a communication network access provider (not shown in the figures).

FIG. 2 illustrates a decoder DEC configured to implement the management method according to a particular embodiment of the invention.

According to this particular mode, the decoder has the conventional architecture of a computer and notably comprises a processor CPU1 (or microcontroller), a memory MEM1 in which a first entity ENT1 is stored. In our example, the entity ENT1 is a computer program which includes instructions for implementing the steps of the management method which will be described later with reference to FIG. 6, when the program is executed by the processor CPU1.

The decoder DEC further comprises communication modules COM11 and COM12 for communication with communication networks, a network RES and a Bluetooth link BT in our example. The network RES is in our example a wide area network such as an ADSL network, Fiber, etc.

FIG. 3 illustrates a remote control configured to implement the management method according to a particular embodiment of the invention.

According to this particular mode, the remote control has the classic architecture of a computer and notably comprises a processor CPU2 (or microcontroller), a memory MEM2 in which, in which, a first entity ENT2 is stored. In our example, the entity ENT2 is a computer program which includes instructions for implementing the steps of the management method which will be described later with reference to FIG. 5, when the program is executed by the processor CPU1.

The remote control TCD also includes a communication module COM1 for communication with a Bluetooth link BT in our example.

Figure 4 is a schematic view highlighting several possible locations of the microphone for sound capture.

In this figure 4 are represented the decoder DEC, the enclosure ENC and the remote control equipped with the microphone MIC. In this same figure, a time axis AX makes it possible to represent

• an instant t1 of emission of the sound signals
• an instant t2 representative of the restitution of the sound signals by the ENC enclosure
• a moment t2+ representative of the instant of capture of sound signals by the MIC microphone installed in the TCD remote control.

Note here that ∆t represents the diffusion time of the sound signals in the air from the ENC enclosure to the MIC microphone. Assuming that the distance between the ENC enclosure and the MIC microphone is of the order of three meters, with a speed of sound of the order of 300 m/s, the signal diffusion time is around 1/100 second ( 10 milliseconds) between the two devices. This diffusion time is imperceptible and is therefore negligible compared to the transmission time of the sound signals from the decoder DEC and the loudspeaker ENC which is of the order of 200 milliseconds.

It is understood from the above that the location of the microphone is either in the enclosure, in the remote control or any other device located in the room in which the ENC enclosure is located. For this reason, in Figure 5, for reasons of simplification of the presentation of the invention, the microphone is glued to the enclosure. In this figure 5, only the instants t1 and t2 are considered, Δt being negligible.

FIG. 6 illustrates the steps of the method carried out in the remote control TCD in accordance with a first embodiment of the invention.

The entity ENT2 performs a phase of determining a duration associated with the transmission of a sound from the decoder and a receiver device equipped with a microphone or any other type of device capable of picking up sound. As shown above, in our example the microphone is installed in the TCD remote.

During a first step ET1, the decoder DEC transmits a request for access to content. This first step can be carried out by the decoder autonomously without user intervention; the content can for example be stored in the memory MEM2 of the decoder.

The DEC decoder requires the reproduction of video content including images and associated sound both on the TV television speaker and on the wireless speaker connected via Bluetooth.

In our example, the chosen content comprises several image sets IMGn with which respective audio samples ECHn are associated (n is an integer). In this configuration, the first set of images IMG1 corresponds to a first audio sample ECH1, the second set of images IMG2 corresponds to a second audio sample ECH2, and so on.

It is assumed here that the decoder and the microphone agree on an instant tc for the start of monitoring of the transmission of audio signals in the decoder DEC and the start of the monitoring of the reception of audio signals.

The synchronization of the clocks in the decoder and the remote control can be achieved by means of the protocol called “Network Time Protocol” (“network time protocol”) or NTP known to those skilled in the art. Thanks to this protocol, the local clocks of the decoder and the remote control are synchronized on a time reference.

An audio sample ECH1 is transmitted at a time tc+t1 by the decoder DEC. The first entity ENT1 timestamps this transmission:

EXH1 –> tc+t1

The audio sample is received by the microphone at a time tc+DL.

The second entity ENT2 timestamps the receipt.

EXH1 –> tc+t2

In our example, the remote control transmits to the decoder DEC the instant of reception tc+t2.

The decoder receives the instant tc+t2 associated with the set of images IMG1 and deduces that the transmission delay is DL seconds:

DL=t2-t1

"-" is the mathematical subtraction sign

The above steps can be repeated several times (j=1,2,3,…) successively for other sound samples over a given period of time. This results in several delays DLj (j= 1,2,3, etc.) for example during a given duration, for example 10 seconds.

In this configuration, the first entity ENT1 installed in the decoder DEC determines a value of DL for example by calculating an average of all the delays DLj received. The invention is obviously not limited to the calculation of an average; for example only one value of DL can be calculated.

Subsequently, during the restitution of content, a time shift of DL seconds is applied to the instants of transmission of the images in relation to the audio signals. In this case, according to the invention, the instant of transmission of the images of a content is delayed by DL seconds with respect to the associated sound. Ultimately, the images reproduced on the television and on the speaker are synchronized.

The embodiment described above may be the subject of variants described below.

According to a variant, to several restitution devices ENCj corresponds a transmission delay DLj calculated according to the principle of the invention. In this configuration, when the decoder decodes a content with a view to its restitution, the first entity ENT1 detects the enclosure ENCj used, obtains the delay DLj obtained previously and applies a delay delay DLj to the instants of transmission of the images of the content concerned.

As seen previously, the microphone MIC can be that of the TCD remote control. The MIC microphone can also be installed in the ENC enclosure. In fact, the transmission time

In our example, after obtaining the transmission duration DL, the second entity ENT2 stores the duration DL in memory MEM2. In this way, the enclosure ENC can provide this duration DL spontaneously or on request, for example, following a change of decoder DEC by a new decoder. When the ENC enclosure connects to the new decoder, the ENC enclosure detects this new decoder and spontaneously transmits the DL duration. The new decoder then applies a time shift of a DL duration when reproducing content without having to perform the steps described above for determining the DL transmission duration.

Remember that a time lag below a DC duration of around 80 milliseconds between an image and an associated audio signal is imperceptible to a user viewing the content. In this configuration, by noting DC the value of the time lag between two imperceptible restitutions, the delay applied to the instants of transmission of the images is chosen in the interval [DL-80milliseconds; DL+80milliseconds]. This mode offers a temporal margin which will not affect the quality of the restitution.

Claims (13)

Method for managing the reproduction of multimedia content comprising images and sound reproduced on respective reproduction devices (ECR, ENC), characterized in that it comprises the following steps:
a step of transmitting from a multimedia content playback device (DEC) an audio signal for playback;
a step of reception, by the multimedia content playback device (DEC), of a datum (tc+t2) linked to the duration of transmission of the audio signal from the multimedia content playback device at least to the restitution device sound (ENC);
a step of obtaining an offset duration (DL) to be applied between the instants of transmission of images and associated audio signals during subsequent reading of a content, the offset duration being a function of the duration transmission (tc+t2). Management method according to Claim 1, characterized in that the shift duration is calculated again following a hardware and/or software modification of the audio reproduction device. Management method according to one of the preceding claims, characterized in that the time lag is calculated again following the addition or deletion of equipment on the audio signal communication link between the multimedia content playback device and the audio rendering device. Management method according to one of Claims 1 to 3, characterized in that the shift duration is stored and transmitted to a multimedia content playback device as soon as the audio reproduction device connects to the multimedia content playback device. Management entity (ENT1) capable of managing the restitution of multimedia content comprising images and sound reproduced on respective restitution devices, characterized in that the entity comprises
a transmission module capable of transmitting from a multimedia content playback device an audio signal for playback;
a reception module capable of receiving data related to the duration of transmission of the audio signal from the multimedia content playback device at least to the audio reproduction device;
a module for obtaining capable of obtaining a time lag to be applied between the instants of transmission of images and associated audio signals during subsequent reading of a content, the time lag being a function of the transmission time . Decoder (TCD) comprising a management entity (ENT1) as defined in claim 5. A computer program capable of being implemented by a management entity (ENT1) as defined in claim 5, the program comprising code instructions which, when executed by a processor, carry out the steps of the method defined in one of claims 1 to 4. Data medium on which has been stored at least one series of program code instructions for the execution of a method defined in one of Claims 1 to 4. Method for managing the reception of audio signals from a multimedia content playback device, characterized in that it comprises
a step of receiving an audio signal;
a time-stamping step of the received audio signal;
a step of transmitting, to the device for reading multimedia content, data related to the timestamp produced. Entity (ENT2) for managing the reception of audio signals from a multimedia content playback device, characterized in that it comprises
a module for receiving an audio signal;
a time-stamping module for the received audio signal;
a transmission module, intended for the multimedia content playback device, of data related to the timestamp produced. Enclosure capable of receiving audio signals to be reproduced, comprising an entity (ENT2) as defined in claim 10. A computer program capable of being implemented by an entity (ENT2) as defined in claim 10, the program comprising code instructions which, when executed by a processor, carry out the steps of the method defined in the claim 9. Data carrier on which has been stored at least one series of program code instructions for the execution of a method defined in claim 9.