EP4264947A1

EP4264947A1 - Server, method for processing a video by means of the server, terminal and method used by the terminal to augment the video by means of an object

Info

Publication number: EP4264947A1
Application number: EP21848174.5A
Authority: EP
Inventors: Chantal Guionnet; Richard Guignon; Jeremy Lacoche
Original assignee: Orange SA
Current assignee: Orange SA
Priority date: 2020-12-15
Filing date: 2021-12-13
Publication date: 2023-10-25
Also published as: US20240107089A1; WO2022129753A1; FR3117714A1

Abstract

The terminal (Ta) comprises an object adapter (PROCa, PROCb) for augmenting a video which is referred to as the initial video and comprises a succession of images (VSi) acquired by a camera (CAM), the adapter being capable of adapting, at least in images (VSi) following a first image (VSI) of the initial video (V) received by the terminal, at least one characteristic for viewing the augmentation object (Oa, Ob), which characteristic is selected and inserted into the first image (VSI), the adapter being capable of operating according to position and calibration parameters (PPi, PCi) of the camera which are associated with the images (VSI, VSi) of the initial video (V), the parameters (PPi, PCi) having been received by the terminal in association with the initial video.

Description

Title of the invention: Server, method for processing a video by the server, terminal, and method implemented by the terminal to enrich the video with an object

Prior technique

[0001] The invention relates to the general field of enrichment of a video by a user, in a personalized manner.

[0002] In the current state of the art, techniques are known for enriching videos by inserting so-called enrichment objects, for example text, a two-dimensional or three-dimensional image, or another video, before its broadcast. to users.

[0003] This solution does not allow personalized enrichment of the video by the end users, that is to say by inserting enrichment objects chosen by the users.

[0004] To date, there is no solution enabling a user to easily insert objects into a video received via a telecommunications network, which offers a satisfactory rendering from the user's point of view.

Disclosure of Invention

[0005] The invention relates to a method for processing a video with a view to broadcasting. The video, called initial video, comprises a succession of images acquired by a camera. The method is implemented by a server and comprises at least one step of generating an enriched stream, the enriched stream being composed of the initial video and, for at least one image of the initial video, parameters of the position and calibration of the camera associated with this image, the initial video and said parameters being able to be sent to at least one terminal.

[0006] Correlatively, the invention relates to a server configured to process a video with a view to broadcasting, the video, referred to as the initial video, comprising a succession of images acquired by a camera, the server comprising an enriched stream generator, the enriched stream being composed of the initial video and, for at least one image of the initial video, of parameters of the position and of the calibration of the camera associated with this image, the initial video and the said parameters being able to be transmitted to destination of at least one terminal.

The characteristics and advantages of the video processing method according to the invention presented below apply in the same way to the server according to the invention and vice versa.

[0008] The server and the receiving terminals for the video and parameters communicate between them via a communication network, for example an IP network. No limitation is imposed on the nature of this communication network. The proposed server can send the initial video and the parameters of the camera to one or more identified terminals, for example in the context of point-to-point or point-to-multipoint communication. Alternatively, the server can send the initial video and the parameters of the camera in broadcast mode, without identifying a specific destination terminal.

[0009] The camera can be connected to the server via a wired or wireless connection, via a local or long-distance network.

The camera position parameters are the coordinates of the camera lens in a Cartesian frame, for example an orthonormal frame or a spherical frame.

[0011] The camera calibration parameters, also called "calibration parameters", can include at least one coordinate of the optical center of the camera (in the above-mentioned Cartesian frame or in another frame), a focal length, a magnification factor and/or distortion parameters.

[0012] The proposed server can obtain the parameters of the camera from various sensors associated with or included in the camera, for example a coordinate sensor of the camera along a given axis, or a module for determining the focal distance of the camera, or a module for determining a magnification factor.

In particular, the proposed server can obtain these parameters continuously, as the initial video is acquired by the camera or as the initial video is received by the server.

[0013] The server and the processing method proposed make it possible to provide the terminals which receive the initial video and the parameters of the camera, means for enriching the video in a personalized way and for obtaining a natural rendering by taking into account the camera parameters by the terminals for the positioning of objects in the initial video. In particular, the server can send the enriched stream to the terminals, the parameters of the position and of the calibration of the camera being integrated into the enriched stream broadcast.

[0014] We designate by a parameter of the camera associated with an image, the parameter of the camera at the time of the acquisition of the image by the camera.

[0015] The invention also relates to a method for enriching a video, called an initial video, by at least one enrichment object, the initial video comprising a succession of images acquired by a camera, the method being implemented implemented by a terminal and comprising at least one step of adaptation, at least in images following a first image of the initial video received by the terminal, of view characteristics of an enrichment object selected and inserted in the first image, the adaptation being carried out as a function of parameters of the position and of the calibration of the camera associated with the images of the initial video, the parameters having been received by the terminal in association with the initial video.

[0016] Correlatively, the invention relates to a terminal comprising a video enrichment object adapter, called initial video and comprising a succession of images acquired by a camera, the adapter being capable of adapting, at least in images following a first image of the initial video received by the terminal, at least one view characteristic of the enrichment object selected and inserted in the first image, the adapter being capable of operating as a function of parameters of the position and calibration of said camera associated with the images of the initial video, the parameters having been received by the terminal in association with the initial video.

The characteristics and advantages of the enrichment method according to the invention presented below apply in the same way to the terminal according to the invention and vice versa.

The characteristics and advantages of the terminal and of the enrichment method according to the invention presented below apply in the same way to the server and to the processing method according to the invention and vice versa.

The first image is an image of the initial video chosen by a user of the terminal to insert the enrichment object therein. The adaptation of the view characteristics of the enrichment object according to the proposed enrichment method is carried out as a function of parameters of the position and of the calibration of the camera associated with the first image, and of the parameters associated with the following images .

[0020] Following the implementation of the enrichment method according to the invention, the terminal obtains an enriched video from the initial video and the enrichment object inserted in the first image and adapted in the following images. .

[0021] In particular, the terminal can insert several enrichment objects in the same first image. In a particular embodiment, the terminal inserts enrichment objects into different images, each of these images being considered as a first image within the meaning of the invention for the enrichment object inserted into this image.

[0022] The view characteristics of the enrichment object can comprise the size and/or the position of this enrichment object, or any other characteristic relating to a shot of the enrichment object as if he was in the actual scene captured by the camera. The position can be defined in particular by the distance, in an image of the video, between the enrichment object and a point representing a position of the camera; the enrichment object then being positioned on a circle of radius equal to said distance. The position can be defined by a view angle, by the camera, of the enrichment object, as if this object is filmed by the camera, the view angle being an angular coordinate defined with respect to a point representing the position of the camera, for example at 30 degrees northeast of the camera.

[0023] When the user inserts the enrichment object in the first image, an original position and orientation are determined in the scene with respect to the position and orientation of the camera at the time of acquisition of this first image.

[0024] Next, the position and orientation of the enrichment object are adapted in the following images to give the illusion that this object is acquired by the camera according to the position and orientation of the camera associated with these following pictures.

The proposed technique offers a natural rendering because the dimensions (size), positions and angles of view in the images of the enriched video are dynamically adapted according to the parameters of the camera at the time of image acquisition. The rendering is natural in that it gives the user the illusion that the enrichment objects have been acquired by the camera.

The proposed terminal can enrich the initial video substantially simultaneously with the reception of the video and the parameters. Indeed, the terminal does not need to carry out complex processing of the content of the initial video to deduce the parameters of the camera.

The proposed technique allows a user of the terminal to enrich the video in a personalized way, by choosing the enrichment objects to be inserted in the first image. An enrichment object can be text or a fixed or animated image, of two or three dimensions. Thus, two terminals which receive the same initial video coming from the server, can obtain and restore two differently enriched videos. In a particular embodiment, the terminal also inserts an enrichment sound object by matching it to the component of the soundtrack of the initial video associated with the first image.

[0028] In a particular embodiment, the terminal receives an enriched stream comprising the initial video and the parameters of the position and of the calibration of the camera associated with the images of the video. In this mode, the proposed terminal further comprises a communication module configured to receive the enriched stream composed of the initial video and parameters of the position and the calibration of the camera associated with the images of the initial video.

In a particular embodiment, the initial video is sent by the server as its images are acquired by the camera (real-time transmission) and enriched in a personalized way by the terminal just before its live restitution.

[0030] In a particular embodiment, the server sends the parameters associated with the images of the initial video and the initial video in a desynchronized manner, that is to say before or after.

In another particular embodiment of the broadcasting method, the server sends the parameters associated with an image and the image synchronously. In particular, the parameters associated with the image can be sent in the header of a packet comprising this image. The server can send a said image and the parameters associated with this image on the same communication channel, or synchronously on separate channels. According to this mode, the proposed terminal receives the initial video and the parameters associated with at least one image synchronized with this image.

This embodiment allows the terminal to enrich the initial video as it is received. In fact, thanks to synchronization, the terminal has the parameters of the camera corresponding to each image received from the initial video.

In a particular embodiment, the processing method according to the invention comprises an insertion of the parameters of the position and of the calibration of the camera associated with the images of the initial video in the control data of the enriched stream.

The initial video is included in the useful data of the enriched stream. We recall that according to the state of the art of communication networks, when a sending device wishes to send a message to a receiving device, it sends the message over the network with other data which allows the routing of the message to to the receiving device. We recall that the useful data, also called the payload, is the part of the transmitted data which is the actual intended message (i.e. the initial video in our invention). The control data comprises headers and metadata which control the transmission of the payload data.

In particular, the proposed server can send the parameters of the camera in a “data” component associated with another “video” component used to broadcast the initial video.

[0036] According to this mode, insofar as the camera parameters are transmitted in an independent component, without any modification of the video components (images and sound), the server can send the initial video and the camera parameters for terminals in accordance with the invention and terminals of the state of the art, within the framework of the same broadcast. Only the receiver terminals in accordance with the invention and which wish to enrich the initial video take the parameters of the camera into consideration.

[0037] The parameters of the camera can also constitute metadata of the initial video images.

In a particular embodiment, the processing method further comprises a step of inserting into the enriched stream at least one shooting parameter of at least one real object of the scene filmed by said camera. . In a particular embodiment, the processing method further comprises a step of sending, in association with the initial video, this shooting parameter of the real object. This shooting parameter is defined, at the time of acquisition of a said image by the camera, in the Cartesian coordinate system in which the parameters of the position of the camera are defined.

[0040] In a particular embodiment of the enrichment method, the adaptation of the view characteristics of the enrichment object takes into account a shooting parameter of a real object of the scene filmed by the camera, the shooting parameter of the real object being received by the terminal in association with the initial video.

[0041] The shooting parameter of the real object comprises the position of the real object relative to the camera, the position being defined for example in terms of distance and/or angle of frontal or lateral view by relative to a point representing a camera position.

These embodiments make it possible to take into account the positions of the real objects filmed with respect to the aforementioned marker, in particular to take account of the displacement of the real objects with respect to this marker. This allows continuous and dynamic adaptation of the parameters of the enrichment objects according to changes in the content of the initial video. We note that the camera and the real object can both be mobile.

[0043] In a particular embodiment, the enrichment method further comprises a step of recording in a non-volatile memory accessible by the terminal, the initial video or the enriched video, the parameters of the position and of the calibration of the camera, and possibly parameters for shooting the real object.

This embodiment allows the terminal to subsequently enrich the recorded video by referring to the parameters of the camera (and possibly the shooting parameters of the real object filmed), to obtain a new enriched video.

[0045] The recording of the initial or enriched video allows a different enrichment from one time to another. Indeed, the enrichment objects inserted, their view characteristics such as their sizes and positions, and the first images in which the objects are inserted may be different.

In a particular embodiment, the processing method comprises an optimization of a transmission of the parameters of the position and of the calibration of the camera associated with the images of the initial video.

In a particular embodiment, the processing method comprises an insertion into the enriched stream of at least one parameter associated with a given image of the initial video, if and only if this parameter is different from the corresponding parameter associated with the previous image. This mode allows you to insert into the enriched flow optimized parameters and therefore to reduce the necessary resources (bandwidth, energy consumption) for the transmission of this flow.

This mode allows optimization of the parameters to be sent by the server to the terminals. In fact, according to this mode, the server only sends changes to the parameters of the camera, for example a change in the position of the camera, a change in an angle taken by the camera and/or a change the focal length of the camera (zoom in or zoom out). This mode makes it possible to optimize the use of the bandwidth between the server and the terminals, and to reduce the energy consumption of the terminals.

In a particular embodiment, the processing method comprises an insertion into the enriched stream of information on a variation of a parameter associated with a given image of the initial video, if the sending of said information consumes less bandwidth than sending the new parameter after said variation.

In a particular embodiment of the processing method according to the invention, a transmission of the parameters of the position and of the calibration of the camera associated with the images of the initial video in the control data of the enriched stream is repeated. In particular, the transmission of the parameters can be repeated on a regular basis, for example every 15 or 30 minutes. Alternatively, the transmission of parameters can be repeated at predefined times.

The fact of retransmitting the parameters allows each terminal which accesses a current broadcast of the initial video to recover the parameters of the camera and thus be able to carry out the enrichment by taking these parameters into account. This mode allows the terminal to receive the camera parameters even if the camera has not moved. Indeed, the sending of the parameters is not carried out only after a change of one of these parameters.

In a particular embodiment, the processing method includes obtaining by the server at least two initial videos acquired by two cameras and which correspond to the same filmed scene. The proposed server sends the two initial videos and the parameters associated with the images of each of the two videos to the terminal.

In a particular embodiment, the enrichment method includes receiving at least two initial videos acquired by two cameras and corresponding to the same filmed scene, and parameters of these cameras. The insertion of the enrichment object and the adaptation of its view characteristics take into account the parameters of each of the cameras.

According to this mode, the enriched video is obtained from the two initial videos.

[0055] In particular, the enriched video can be obtained by a selection at a time given, of the video to be enriched among the initial videos. The enriched video sometimes corresponds to the first initial enriched video, and sometimes to the second initial enriched video.

[0056] Alternatively, the enriched video can be obtained by a combination of the two initial videos, for example by stereoscopy, followed by an insertion of an enrichment object with a two-dimensional or three-dimensional effect and an adaptation of the characteristics of this object.

The video enriched according to the method of the invention can be a so-called augmented reality video, that is to say obtained by superimposing an initial video representing a real scene and enrichment objects calculated by the terminal when the initial video is received.

The invention also relates to a video broadcasting system, the system comprising at least one camera, a server in accordance with the invention and at least one terminal in accordance with the invention.

The invention also relates to a device for acquiring the initial video with a view to its broadcasting, this device comprising an enriched stream generator, the enriched stream being composed of the initial video and, for at least one image of the initial video, of parameters of the position and of the calibration of the camera associated with this image, the initial video and the parameters being capable of being sent to at least one terminal. This device can be connected to a recorder connectable to a transmitter, for example the proposed server. Alternatively, this device can be connected directly to a transmitter such as the proposed server.

The invention also relates to a computer program on a recording medium, this program being capable of being implemented in a computer or a server in accordance with the invention. This program includes instructions adapted to the implementation of a processing method, as described above.

The invention also relates to a computer program on a recording medium, this program being capable of being implemented in a computer or a terminal in accordance with the invention. This program includes instructions adapted to the implementation of an enrichment process, as described above.

[0062] Each of these programs may use any programming language, and be in the form of source code, object code, or intermediate code between source code and object code, such as in partially compiled form, or in any other desirable shape.

[0063] The invention also relates to an information medium or a recording medium readable by a computer, and comprising instructions of a computer program as mentioned above.

[0064] The information or recording media can be any entity or device capable of storing the programs. For example, the media may comprise a storage medium, such as a ROM, for example a CD ROM or a microelectronic circuit ROM, or even a magnetic recording medium, for example a floppy disk or a disk. hard, or flash memory.

On the other hand, the information or recording media can be transmissible media such as an electrical or optical signal, which can be routed via an electrical or optical cable, by radio link, by optical link without wire or by other means.

The programs according to the invention can in particular be downloaded from an Internet-type network.

Alternatively, each information or recording medium may be an integrated circuit in which the program is incorporated, the circuit being adapted to execute or to be used in the execution of a method in accordance with the invention. Brief description of the drawings

Other characteristics and advantages of the present invention will become apparent from the description given below, with reference to the appended drawings which illustrate an example of embodiment devoid of any limiting character. In the figures:

[fig.l] the [fig.l] is a flowchart representing the steps of the proposed methods, implemented according to a first particular embodiment;

[fig.2] the [fig.2] is a flowchart representing the steps of the proposed methods, implemented according to another particular embodiment;

[fig.3] [fig.3] illustrates functional architectures of a server and proposed terminals, according to a particular embodiment;

[fig.4] [fig.4] presents a hardware architecture of a proposed terminal, according to a particular embodiment; and

[fig.5] [fig.5] presents a hardware architecture of a proposed server, according to a particular embodiment.

Description of embodiments

[0069] The [fig.1] is a flowchart representing:

- steps, numbered Exxx, of a video processing method with a view to broadcasting, implemented according to a particular embodiment by an SRV server; and

- Steps, numbered Fxxx, of a method for enriching a video, implemented according to a particular embodiment by a terminal Ta; the server SRV, the terminal Ta and the methods being in accordance with the invention. A CAM camera is connected to the SRV server, for example via a wired connection. No limitation is imposed on the nature of the connection of the camera to the server.

The server SRV and the terminal Ta can communicate with each other via a communication network, for example a television broadcasting network or an IP network. No limitation is imposed on the nature of this communication network.

During a step El 00, the server SRV obtains a video, called initial video V, acquired by the camera CAM. In the mode described here, the server SRV obtains the video V as it is acquired by the camera CAM. The SRV server is configured to broadcast this video live, as it is acquired.

The initial video V is a sequence of several images VSi. To each VSi image, correspond parameters of the PPi position and of the PCi calibration of the camera when this VSi image was acquired. The parameters of the position PPi comprise for example the coordinates of the camera in a Cartesian coordinate system. The PCi calibration parameters of the camera comprise for example the focal length, a coordinate of the optical center and the distortion parameters of the CAM camera.

During step E100, the server SRV obtains the images VSi and the parameters PPi, PCi of the camera associated with these images VSi. In the mode described here, the SRV server obtains the PPi and PCi parameters synchronous with the associated VSi images.

In the mode described here, and during an optional step E200, the server SRV obtains at least one parameter p_ORi for shooting a real object of the scene filmed by the camera CAM in the Cartesian coordinate system at time of VSi image acquisition. The shooting parameter p_ORi can correspond to a position and/or to a frontal or lateral angle of view of the real object with respect to the camera. The real object can be fixed or mobile.

During a step E300, the server SRV sends to the terminal Ta the initial video V as it is acquired and the parameters PPi and PCi associated with the images VSi of the initial video V.

In the example described here, we assume that the initial video V comprises images VSI to VS5. The first VSI image is acquired by the camera CAM whose parameters, during acquisition, are referenced by PPI and PCI. The server SRV sends during step E300 the VSI image in association with the PPI and PCI parameters. These position and PPI and PCI calibration parameters are not changed during the acquisition of the second VS2 image. To optimize the bandwidth, the server SRV sends during the step E300, the VS2 image without returning the PPI and PCI parameters, these parameters not being changed and also being associated with the VS2 image. Assuming that the third VS3 image corresponds to new camera parameters, PP3 and PC3, the SRV server sends during step E300 the VS3 image in conjunction with the new PP3 and PC3 settings.

In the mode described here, the SRV server only sends the PPi and PCi parameters when these parameters are different from the last parameters sent. In particular, in the event of changes in certain parameters, the server can send information on the evolution of these parameters (for example variations), if this allows better optimization of the bandwidth compared to sending new parameters. . In another embodiment, the server SRV sends the parameters PPi and PCi associated with each image VSi, even if they are also associated with the previous image VS(i-l).

The server SRV sends (E300) the parameters PPi and PCi synchronously with the associated images VSi, but on a different communication channel.

The SRV server also sends during step E300, the shooting parameter p_ORi of the real object of the scene filmed by the camera, at the time of acquisition of the VSi image. This parameter is sent in association with the corresponding VSi image. If the real object changes position, the SRV server sends a new shot parameter in association with the corresponding new image.

During a step F100 of the enrichment method, the terminal Ta receives the initial video V coming from the server SRV, as well as the position parameters PPi and calibration parameters PCi of the camera CAM during the acquisition VSi images.

In the mode described here, as the server has sent (E300) the shooting parameter p_ORi of the real object filmed by the camera CAM, the terminal Ta receives this parameter p_ORi during step F100.

During a step F200, the terminal Ta obtains at least one enrichment object Oa, such as text, or a fixed or animated image. The terminal Ta can select the object Oa in a library, for example recorded in a memory of the terminal Ta or in a memory of another device to which the terminal can access. The selection by the terminal of the object is made in particular according to one or more parameters for viewing the enhanced video, in particular from among the following: viewing context (time of day, location, number of viewers, etc.), user profile, parameters relating to the user's terminal (screen size, quality of the image and/or sound that can be rendered, etc.), etc. Alternatively, the terminal Ta can receive the enrichment object Oa from the server SRV. A user of the terminal Ta selects the object Oa to insert it into an image of the initial video V, called the “first image”.

During a step F300 of obtaining an enriched video Va-i- from the video V, following the selection of the enrichment object Oa, the terminal Ta inserts this selected object into the first image, for example the VSI image. In one embodiment, the terminal Ta obtains during step F200 a sound-type enrichment object, which it inserts during step F300 by synchronizing it with the sound component associated with the first VS1 picture.

As the initial video V is broadcast live, the terminal Ta continues to receive (F100) images VSi of this video V, and adapts during the step F300 of obtaining the enriched video Va+, characteristics view of the enrichment object Oa as a function of the parameters PP1 and PCI associated with the first image VSI, and of the parameters PPi and PCi associated with each of the following images VSi, i varies from 2 to 5 in our example. The view characteristics of the enrichment object include for example the position and/or the size of this object in a VSi image.

According to the example described here, as the parameters PP3 and PC3 associated with the VS3 image are different from those associated with the VS2 image, the view characteristics such as the size and position of the object of enhancement Oa in the third VS3 image are changed from the view characteristics of the enhancement object Oa in the second VS2 image.

According to an example of use of the methods of the invention, the initial video comprises an acquisition of a scene in a room, for example in a living room. The terminal inserts during a first iteration of step F300, an enrichment object Oa representing a clock or a decorative painting, to decorate a wall of the living room. The view characteristics (position and size) of this clock or table Oa inserted in the first VSI image are selected by the user to match the position and dimensions of the wall in the scene. The terminal Ta saves the parameters of the camera associated with the first VSI image and the parameters of the object Oa inserted in the first VSI image, such as its size, its position and its orientation.

Assuming that in a following image VSn of the initial video V, the parameters of the camera CAM change, for example by enlarging the focal length of the camera (zoom), but that the position of the optical center of the camera n has not changed since the acquisition position of VSI. Under these conditions, the position and orientation of the object in the VSn image are identical to what they were in the VSI image but the size of the object Oa is enlarged during one iteration of the step F300. The new size of the Oa object in the VSn image is determined from the size of the Oa object in the VSI image, the change in camera focal length between the VSI and VSn images, and the position of the object Oa in the VSI image with respect to the position of the optical center of the camera associated with the VSI image.

[0090] Supposing that in another following image VSm, the parameters of the camera CAM change, for example by a rotation of the camera, the position of the object Oa in the image VSm is adapted during an iteration of step F300. The new position of the enrichment object Oa is determined from its position in the previous image VSn and the values of the parameter of the camera which has changed (in particular the coordinates of the optical center), before and after the change.

According to the invention, the adaptation of the position, orientation and size (including the depth) of the object Oa in the images VSn, VSm are such that the object is perceived as if it had been acquired in the real scene by the camera, knowing that the position, the orientation and the size of the object Oa in the real scene are defined by the position, the orientation and the size of the object Oa in the VS1 image at the time of its insertion.

According to another example of use of the methods of the invention, the initial video V relates to a football match. On the first image, the enrichment object Oa inserted is a texture covering the image of the balloon illustrated by the first figure. As the position p_ORi of the real ball changes relative to the camera, the size and the position of the portion of the image covered by the texture Oa are adapted during the iterations of step F300.

During a step F500, the terminal T restores the enhanced video Va+, for example on a screen of the terminal Ta. In the described mode, the restitution F500 is implemented as the view characteristics of the object Oa are adapted and the enriched video Va+ is obtained F300. Alternatively, the restitution can take place after the enrichment of all the VSi images of the initial video V is finished.

During an optional step F 120, the terminal Ta saves in a memory accessible by the latter, the initial video V, the parameters PPi and PCi associated with the images VSi of the initial video V, and possibly the parameter p_ORi shot of the real object with respect to the CAM camera. This recording allows subsequent enhancement of the initial video V. The rendering of this subsequent enhancement may be different from the enhanced video Va+.

During an optional step F400, the terminal Ta saves in a memory accessible by the latter, the enhanced video Va-i-, the parameters PPi and PCi associated with the images VSi of the initial video V, and possibly the parameter p_ORi for shooting the real object with respect to the CAM camera. This recording allows subsequent additional enrichment from the already enriched video Va+.

In the mode described with reference to [fig.l], the server sends the parameters PPi and PCi associated with the VSi images, in synchronization with the sending of the VSi images. In another embodiment, the parameters and the images are sent with a time shift, one before the other, with cross-references (at least one reference from a parameter to an image and at least one reference from an image to a parameter) in order to recreate the synchronization at the terminal level. In a particular embodiment, the terminal comprises two modes of restitution of the video received: a mode of restitution of the initial video and an enriched mode of restitution. The enriched restitution mode comprises the implementation of steps F200 to F500 whereas the mode of restitution of the initial video comprises directly after the reception step F100 of the enriched stream the restitution of the initial video carried by this enriched stream. The restitution mode implemented depends on an action by the user. In particular, the enriched restitution mode is triggered by an enrichment request received by the terminal in particular by means of a user interface.

[0098] [fig.2] is a flowchart representing steps Exxx of the method for processing a video, and steps Fxxx of the method for enriching a video, the methods being implemented according to a second mode of particular achievement.

In the mode described here, and unlike the first mode described with reference to [fig.l], the server SRV obtains during steps ElOOa and ElOOb, two initial videos VI and V2 acquired by two cameras CAM1 and CAM2 to film the same scene, simultaneously. These videos are synchronized. During these steps ElOOa and ElOOb, the server SRV obtains parameters of the position PPli and of the calibration PCli of the camera CAM1, associated with the images of the first video VI, and parameters of the position PP2i and of the calibration PC2i of the camera CAM2, associated with the images of the second video V2.

During a step E300 similar to step E300 described with reference to [fig.1], the server SRV sends the two initial videos VI and V2 and all the parameters to the terminal Ta.

[0101] During a step F100 similar to step F100 described with reference to [fig.l], the terminal Ta receives the two initial videos VI and V2 and all the parameters from the server SRV.

During a step F200 similar to step F200 described with reference to [fig.l], the terminal Ta obtains the enrichment object Oa, for example a three-dimensional modeled object.

During a step F300, the terminal Ta obtains an enriched video Va-i- from the two initial videos VI and V2, for example by combining images from the two videos to have a video with a 3D effect, then by inserting the object Oa and by adapting its view characteristics (position, size, orientation, etc.) taking into account the parameters associated with the images of the first video VI and the parameters associated with the images of the first video V2.

The [fig.3] represents functional architectures, according to a particular embodiment, of the proposed server SRV, of the terminal Ta and of two other terminals Tb and Te in accordance with the invention.

The CAM camera described with reference to [fig.l] is connected to the SRV server. the server SRV, the terminals Ta, Tb and Te and the camera CAM form a system for broadcasting a video V.

[0106] The SRV server comprises:

- an OBTS module for obtaining (El 00) the initial video V and obtaining (El 00), for at least one image VSi of the initial video V, parameters PPi, PCi of the position and the calibration of the CAM camera associated with this VSi image; this OBTS module is an enriched stream generator, the enriched stream being composed of said initial video V and, for at least one image VSi of the initial video V, of parameters PPi, PCi of the position and of the calibration of the associated camera to this picture and

- a COMS communication module configured to send (E300) the initial video V and the parameters PPi, PCi to the terminals Ta, Tb and Te.

The OBTS module can also be configured to obtain (E200) the parameters p_ORi for shooting real objects filmed by the camera CAM. The CMS communication module is further configured to send these parameters p_ORi in association with the corresponding images of the initial video V.

In the mode described here, the SRV server further comprises an optimization module OPTIM configured to optimize the transmission of the parameters of the camera PPi and PCi. For example, the OPTIM module can be configured to determine if the PPi and PCi parameters associated with a VSi image are different from the parameters associated with the previous VS image (i-1). The optimization module OPTIM commands the communication module COM_S so that the latter sends (E300) the parameters associated with the image VSi only if and only these parameters are different from those associated with the image VS(i-l). According to another example, the optimization module OPTIM is configured to determine whether the sending of new camera parameters associated with the VSi image or the sending of variations between the parameters associated with the VSi and VS(i-l) images requires fewer resources in terms of bandwidth and to control the communications module according to a result of this comparison.

[0109] In the mode described here, and as illustrated by [fig.3], the server SRV sends the parameters over a communication channel different from the channel used to send the video V. Alternatively, the parameters and the images VSi of the video V can be sent on the same communication channel, the parameters being sent as control data, for example in packet headers, while the images being sent as useful data, in the bodies of the packages.

The terminal Ta comprises:

- a communication module OBT_Ta configured to receive (El 00) the initial video V, this initial video comprising a succession of images VSi acquired by the camera CAM, the communication module is further configured to receive

(El 00) the PPi, PCi parameters of the position and the calibration of the associated camera to at least one image VSi of the initial video V;

- a selector SLCTa configured to select (F200) at least one enrichment object Oa; and

- a PROCa adapter configured to obtain (F300) a Va+ enriched video by:

- insertion of the enrichment object Oa selected in a first image VSI of the initial video V; and

- adaptation of at least one view characteristic of the enrichment object Oa in the following images (VS2 to VS5) according to the parameters PP1, PCI associated with the first image and the parameters PPi, PCi associated with each of the images

VSi following from the initial video V.

In the mode described here, the terminal Ta further includes a module DISPa for restoring (F500) the enhanced video Va+.

The terminal Ta can comprise a memory in which a library is recorded, the enrichment object Oa is obtained from this library.

The terminal Tb has a functional architecture similar to that of the terminal Ta. The terminal Tb obtains (F200) an enrichment object Ob different from the object Oa. The terminal Tb then obtains (F300), from the same initial video V, an enriched video Vb-i- different from the video Va-i- enriched by the terminal Ta. We note that even if the objects Oa and Ob are identical, the enriched videos Oa and Ob can be different because the choice of the users of the terminals Ta and Tb, in terms of the moment of insertion of the objects Oa and Ob (and therefore of the "first image"), and view characteristics (eg position and size) of the enrichment objects upon initial insertion, may be different.

The terminal Te also has a functional architecture similar to that of the terminal Ta. Suppose that the terminal Te does not wish to enrich the initial video V, it then restores a video Vc identical to the initial video V. The implementation of the processing method by the server SRV has no impact on the operation of the Te terminal. Indeed, the video V and the PPi and PCi parameters of the camera are sent on two different channels. The terminal Te can ignore the parameter transmission channel.

[0115] In the embodiments described here, each of the terminals Ta and Tb has the hardware architecture of a computer, as illustrated in [fig.4],

The architecture of the terminal Ta, for example, comprises in particular a 7T processor, an 8T random access memory, a 9T read only memory, a 10T non-volatile flash memory in a particular embodiment of the invention, as well as means of communication 11T. Such means are known per se and are not described in more detail here.

[0117] The 9T read only memory of the terminal Ta constitutes a recording medium in accordance with the invention, readable by the processor 7T and on which is recorded here a computer program ProgT in accordance with the invention.

The 10T memory of the terminal Ta makes it possible to record variables used for the execution of the steps of the method proposed to enrich a video, such as the initial video V, the images VSi, the parameters PPi and PCi of the position and of the camera calibration, the parameter p_ORi of the shot of the real object, and the enhanced video Va+.

The computer program ProgT defines functional and software modules here, configured to enrich the initial video V by the terminal Ta. These functional modules rely on and/or control the 7T-11T hardware elements of the Ta terminal mentioned above.

[0120] In the embodiments described here, the SRV server has the hardware architecture of a computer, as illustrated in [fig.5].

The architecture of the server SRV comprises in particular a 7S processor, an 8S RAM, a 9S ROM, a 10S non-volatile flash memory in a particular embodiment of the invention, as well as 11S communication means. Such means are known per se and are not described in more detail here.

The 9S ROM of the SRV server constitutes a recording medium in accordance with the invention, readable by the 7S processor and on which is recorded here a ProgS computer program in accordance with the invention.

The 10S memory of the server SRV makes it possible to record variables used for the execution of the steps of the proposed method for processing a video, such as the initial video V, the images VSi, the parameters PPi and PCi of the position and calibration of the camera, and the parameter p_ORi of the shot of the real object.

The computer program ProgS defines functional and software modules here, configured to broadcast the initial video V by the server SRV. These functional modules rely on and/or control the 7S-11S hardware elements of the SRV server mentioned above.

Claims

[Claim 1] Method for enriching a video (V), called initial video, by at least one enrichment object (Oa, Ob), said initial video comprising a succession of images (VSi) acquired by a camera (CAM), the method being implemented by a terminal (Ta, Tb) and comprising at least one adaptation step (F300), at least in images (VSi) following a first image (VSI) of said initial video (V) received (F 100) by the terminal, of view characteristics of a said enrichment object (Oa, Ob) selected (F200) and inserted into said first image (VSI), the adaptation (F300) being performed as a function of parameters (PPi, PCi) of the position and of the calibration of said camera associated with the images (VSI, VSi) of the initial video (V), the parameters (PPi, PCi) having been received (F 100) by the terminal in association with the initial video.

[Claim 2] Method according to Claim 1, in which the adaptation (F300) of the view characteristics of the enrichment object takes account of a view parameter (p_ORi) of a real object of a scene filmed by said camera (CAM), the parameter (p_ORi) for shooting the real object being received (F100) by the terminal in association with the initial video.

[Claim 3] Process for processing a video (V) with a view to broadcasting, the video, called initial video (V), comprising a succession of images (VSi) acquired by a camera (CAM), said process being implemented by a server (SRV) and comprising at least one step of generating an enriched stream, the enriched stream being composed of the initial video (V) and, for at least one image (VSi) of the initial video (V), parameters (PPi, PCi) of the position and calibration of said camera associated with this image (VSi), said initial video (V) and said parameters (PPi, PCi) being able to be transmitted (E300 ) to at least one terminal (Ta, Tb, Te).

[Claim 4] Method according to claim 3 comprising an insertion of the parameters (PPi, PCi) of the position and of the calibration of the said camera associated with the images of the initial video in the control data of the enriched stream.

[Claim 5] Method according to one of Claims 3 or 4, in which the said initial video is sent (E300) as it is acquired (El 00).

[Claim 6] Method according to one of Claims 3 to 5, in which the parameters (PPi, PCi) associated with an image (VSi) and this image (VSi) are synchronized in the enriched stream.

[Claim 7] Method according to one of Claims 3 to 6 comprising an optimization of a transmission of the parameters (PPi, PCi) of the position and of the calibration of the said camera associated with the images of the initial video.

[Claim 8] Method according to claim 7 comprising an insertion into the enriched stream of at least one said parameter (PP3, PC3) associated with a given image (VS3) of the initial video (V), if and only if this parameter (PP3, PC3) is different from the corresponding parameter associated (PPI, PCI) with the previous image (VS2).

[Claim 9] Method according to one of Claims 3 to 8 further comprising a step of inserting into the enriched stream (E200) a parameter (p_ORi) for shooting a real object of the scene filmed by said camera.

[Claim 10] Method according to one of Claims 3 to 9, in which a transmission of the parameters (PPi, PCi) of the position and of the calibration of the said camera associated with the images (VSi) of the initial video in the control data of the enriched stream is repeated.

[Claim 11] Computer program (ProgT, ProgS) comprising instructions for carrying out the steps of a method according to Claims 1 to 2, or of a method according to any one of Claims 3 to 10, when said program is executed by a computer.

[Claim 12] A computer-readable recording medium (7T, 7S) on which a computer program according to claim 11 is recorded.

[Claim 13] Terminal (Ta, Tb, Te) comprising an adapter (PROCa, PROCb) of an object for enriching a video, referred to as the initial video and comprising a succession of images (VSi) acquired by a camera (CAM ), the adapter being capable of adapting, at least in images (VSi) following a first image (VSI) of the initial video (V) received by the terminal, at least one view characteristic of the enrichment object (Oa, Ob) selected and inserted into the first image (VSI), the adapter being capable of operating as a function of parameters (PPi, PCi) of the position and calibration of said camera associated with the images (VSI, VSi) of the initial video (V), the parameters (PPi, PCi) having been received by the terminal in association with the initial video.

[Claim 14] Terminal according to claim 13 further comprising a communication module (OBT_Ta, OBT_Tb) configured to receive an enriched stream composed of the initial video (V) and said parameters (PPi, PCi) associated with the images (VSi) of the initial video (V). [Claim 15] Server (SRV) configured to process a video with a view to broadcasting, the video, referred to as the initial video, comprising a succession of images (VSi) acquired by a camera (CAM), the server (SRV) comprising an enriched stream generator (OBTS), the enriched stream being composed of said initial video (V) and, for at least one image (VSi) of the initial video (V), of parameters (PPi, PCi) of the position and calibration of said camera associated with this image (VSi), the initial video (V) and said parameters (PPi, PCi) being able to be sent to at least one terminal (Ta, Tb, Te).