FR3109686A1 - Transposition process of an audiovisual stream - Google Patents

Abstract

Method for transposing an audiovisual stream The invention relates to a method for transposing (100) a source stream of visual data shaped according to a first display format into a transposed stream of visual data shaped according to a second display format. , the method (100) comprising at least the following steps performed: a. Identifying (110) a first formatting model; b. Detection (120) of at least one area of interest in at least one image; vs. Selecting (130) at least one second formatting model; d. Composition (140) of at least one reformatted image according to the second display format and according to the second formatting model, from at least part of the area of interest and from at least part of said image; summer. Repeating (150) steps a through d so as to form said transposed stream of visual data. Figure for abstract: Fig.2

Description

Process for transposing an audiovisual stream

The present invention relates to the field of audiovisual methods. It finds a particularly advantageous application in the field of audiovisual distribution of content on different types of mobile display devices.

STATE OF THE ART

In the history of the audiovisual industry, with the development of terminals, in particular mobile devices integrating a display device, such as smart phones (usually designated by the English term smartphone), there have never been so many different types of display devices and therefore screen formats. In general, these screens extend along a main direction and generally have a rectangular shape.

When considering a smartphone, for example, with respect to audiovisual content in a 16:9 format, a horizontal screen orientation (length of the screen in the horizontal direction) is advantageous for comfort. user visual.

However, some users prefer to use their screen vertically rather than horizontally, which gives degraded comfort because the content reaching the display device is suitable for horizontal viewing.

In addition, the content becomes difficult to view on media where the orientation is not easily adjustable, hence the emergence of new technologies for changing the orientation of televisions, for example.

Thus, the currently most widespread techniques for best accommodating the display of an audiovisual stream as a function of the display device consist of hardware solutions for modifying the orientation of the display device to improve user comfort. user.

These hardware solutions therefore have limitations and obligations to change orientation for the user.

A non-limiting object of the present invention is therefore to propose a solution.

The other objects, features and advantages of the present invention will become apparent from a review of the following description and the accompanying drawings. It is understood that other benefits may be incorporated.

SUMMARY

1. Identification du premier modèle de mise en forme parmi au moins une première base de données de modèles de mise en forme ;
2. Détection d’au moins une première zone d’intérêt dans au moins une première image prise parmi la première pluralité d’images ;
3. Sélection d’au moins un deuxième modèle de mise en forme pris parmi au moins une deuxième base de données de modèles de mises en forme, ladite sélection dudit deuxième modèle de mise en forme étant fonction d’au moins l’un parmi : le premier modèle de mise en forme, ladite première zone d’intérêt, le deuxième format d’affichage ;
4. Composition d’au moins une image reformatée selon le deuxième format d’affichage et selon le deuxième modèle de mise en forme, l’étape de composition comprenant au moins une étape de disposition, en fonction dudit deuxième modèle de mise en forme, d’une première zone reformatée correspondant à une partie au moins de la première zone d’intérêt et d’une deuxième zone reformatée correspondant à une partie au moins de ladite première image ; et,
5. Répétition des étapes a à d pour une pluralité d’images de la première pluralité d’image dudit flux source de sorte à former ledit flux transposé de données visuelles comprenant une deuxième pluralité d’images conformée selon le deuxième format d’affichage.

To achieve this objective, according to one embodiment, a method is provided for transposing at least one source stream of visual data, preferably audiovisual, shaped according to a first display format into a transposed stream of visual data, preferably audiovisual , shaped according to a second display format, the second display format being different from the first display format, the source stream comprising a first plurality of images, each image of the first plurality of images being shaped according to the first display format and according to a first formatting model, the method comprising at least the following steps carried out by at least one computer data processing device:

1. identifying the first shaping model from at least a first database of shaping models;
2. detection of at least a first area of interest in at least a first image taken from among the first plurality of images;
3. Selection of at least one second layout model taken from at least one second database of layout models, said selection of said second layout model being a function of at least one of: the first layout model, said first area of interest, the second display format;
4. Composition of at least one reformatted image according to the second display format and according to the second formatting model, the step of composition comprising at least one step of arranging, according to said second formatting model, a first reformatted zone corresponding to at least part of the first zone of interest and a second reformatted zone corresponding to at least part of said first image; And,
5. Repetition of steps a to d for a plurality of images of the first plurality of images of said source stream so as to form said transposed stream of visual data comprising a second plurality of images shaped according to the second display format.

This allows adaptation of a video stream to the display format of the display device considered for the display of said video stream.

This makes it possible to transpose a video stream formatted for example for a television into a stream formatted for a mobile device whose display dimensions are different from those of the television and this with better viewing comfort and reduced or even zero loss of information. contained in the original stream.

This invention proves to be particularly advantageous for producers and broadcasters of traditional television content, for example in a 16:9 display format, so that they can automatically and preferably in real time transpose their content from the 16:9 format :9 to vertical video formats (9:16, 4:5 or 2:3) used for mobile phones and some social networks.

This makes it possible to intelligently select and position areas of interest relative to each other by adapting to the display format of the display device.

This invention is particularly suitable for television news whose content must be delivered live and for which the "verticalization" of the image could not be done manually in real time. The vertical content produced, i.e. the transposed flow in this case, is published on digital platforms such as social networks or mobile applications.

According to a non-limiting example, the first display format is intended to be displayed by a device, preferably mobile, arranged in a first spatial orientation, and the second display format is intended to be displayed by a device, preferably mobile, arranged in a second spatial orientation different from the first spatial orientation, preferably the first display format being mainly horizontal and the second display format being mainly vertical.

According to one example, the display device can be a mobile device. It can for example be a smart phone.

According to an alternative example, the device can be a fixed device. For example, it can be configured to be attached to a wall. It may for example be an information screen in a public reception area, such as a shopping center or an airport. Often, in these places, fixed screens have shapes such that their main dimension extends vertically.

According to one example, the method comprises a step of displaying on a display device said transposed stream of visual data comprising the second plurality of images shaped according to the second display format.

Another aspect relates to a visual data stream transposed from a source stream of visual data by the method according to the present invention, the visual data of the source stream being shaped according to a first display format and the visual data of the transposed stream being shaped according to a second display format, different from the first display format, preferably the transposed stream being intended to be viewed on a mobile display device.

Another aspect relates to a computer program product, preferably recorded on a non-transitory medium, comprising instructions, which when performed by at least one of a processor and a computer, cause the at least one among the processor and the computer, executes the method according to the present invention.

Another aspect relates to a computer server comprising at least the computer program product according to the present invention and being configured to receive at least one visual data source stream and to transmit at least one transposed stream according to the present invention.

Another aspect relates to a system comprising at least one computer server according to the present invention and at least one display device configured to display the transposed stream according to the present invention.

1. Réception, par un serveur informatique selon la présente invention, d’au moins un flux source de données visuelles conformées selon un premier format d’affichage ;
2. Transposition du flux source de sorte à obtenir ledit flux transposé, cette étape de transposition étant effectuée en mettant en œuvre les étapes a à e du procédé selon la présente invention ;
3. Emission par le serveur informatique dudit flux transposé à destination d’au moins un dispositif d’affichage.

Another aspect relates to a method for displaying at least one transposed stream according to the present invention on a display device, the method comprising at least the following steps:

1. Reception, by a computer server according to the present invention, of at least one source stream of visual data shaped according to a first display format;
2. Transposition of the source stream so as to obtain said transposed stream, this transposition step being carried out by implementing steps a to e of the method according to the present invention;
3. Emission by the computer server of said transposed stream intended for at least one display device.

BRIEF DESCRIPTION OF FIGURES

The aims, objects, as well as the characteristics and advantages of the invention will emerge better from the detailed description of an embodiment of the latter which is illustrated by the following accompanying drawings in which:

Figure 1 shows a system according to one embodiment of the present invention.

FIG. 2 represents certain steps of a method for transposing a stream of visual data according to one embodiment.

Figure 3 illustrates steps 120 and 140 of the method of Figure 2 according to one embodiment of the present invention.

Figure 4 illustrates the transposition of an image from a first display format to a second display format according to one embodiment of the present invention.

FIG. 5 illustrates the transposition of an image from a first display format to a second display format according to an embodiment of the present invention.

Figure 6 illustrates the transposition of an image from a first display format to a second display format according to one embodiment of the present invention.

Figure 7 illustrates an unconformed data stream and a conformed data stream.

FIG. 8 illustrates a step for centering a reformatted zone.

The drawings are given by way of examples and do not limit the invention. They constitute schematic representations of principle intended to facilitate understanding of the invention and are not necessarily scaled to practical applications. In particular the dimensions are not representative of reality.

DETAILED DESCRIPTION

Before starting a detailed review of embodiments of the invention, optional characteristics are set out below which may possibly be used in combination or alternatively:

According to one example, said step of detecting the first zone of interest comprises a step of extracting the first zone of interest from the first image.

According to one example, said step of selecting said second layout model is a function of at least two and preferably three among: the first layout model, said first area of interest, the second display format.

According to one example, the visual data of the source stream is intended to be displayed by a first type of display device and the visual data of the transposed stream is intended to be displayed by a second type of display device different from the first type of display device.

According to one example, the first and the second type of display devices differ by their orientation or more precisely by the direction in which they mainly extend.

According to one example, the source stream comprises a stream, called clean stream and another stream, called dressed stream, the clean stream comprising a plurality of images called raw images and the dressed stream comprising a plurality of images called dressed images, the plurality of dressed images comprising the plurality of raw images and at least one element for dressing at least one raw image.

According to an example, the dressing element comprises at least one of: a timestamp, a title, a subtitle, an image, a graphic animation.

According to one example, the first area of interest is extracted from an image of the plurality of raw images corresponding to the first image.

According to one example, the first area of interest is extracted from an image of the plurality of assembled images corresponding to the first image.

According to one example, the source stream comprises a plurality of input streams comprising a plurality of images, at least one input stream is the clean stream, at least one other input stream is the dressed stream.

According to one example, at least one input stream comprises said at least one skin element.

According to one example, at least one input stream comprises a plurality of images obtained from a first camera filming a scene according to a first angle of view, and at least one other input stream comprises a plurality of images obtained from a second camera filming said scene according to a second angle of view different from the first angle of view.

According to one example, the step of detecting the first zone of interest comprises at least one step of detecting contours in the first image, a zone delimited by at least one contour forming a zone of interest.

According to an example, the step of detecting the first zone of interest comprises at least one step of identifying a type of scene from among a plurality of types of predetermined scenes, each type of scene comprising at least one zone of interest.

According to one example, the step of detecting the first zone of interest comprises at least one step of detecting at least one element of interest taken from at least: a face, a body, an object, a vehicle, a geographical map, a graphic element such as a logo, a title, a clock.

According to an example, the first zone of interest comprises at least one of: a face, a body, an object, a vehicle, a geographical map, a weather map.

According to one example, the step of detecting the first zone of interest further comprises the detection of a second zone of interest in the first image and the second reformatted zone corresponds to at least part of said second zone of interest.

According to one example, the step of detecting the first zone of interest is carried out by an artificial intelligence trained to identify different types of zones of interest and/or elements of interest.

According to one example, the first layout model comprises a first arrangement of a first plurality of zones intended to include a plurality of visual data and the second layout model comprises a second arrangement of a second plurality of zones intended comprising a plurality of visual data, the second arrangement being different from the first arrangement and the second plurality of visual data being at least partly included in the first plurality of visual data.

According to one example, the step of identifying the first formatting model comprises at least one step of receiving an identifier of said first formatting model, this identifier being associated with a formatting model contained in the first formatting model database, preferably this identifier being transmitted by the source stream.

According to one example, the step of identifying the first shaping model comprises at least one step of determining by an artificial intelligence the first shaping model by comparison with at least one shaping model of the first base formatting model data.

According to one example, the composition step further comprises the formatting of at least part of the first zone of interest so as to generate the first reformatted zone.

According to one example, the formatting of at least part of the first zone of interest comprises a step of cutting out at least part of the first zone of interest.

According to one example, the composition step comprises at least one step of extracting at least part of the first zone of interest so as to generate the first reformatted zone.

According to one example, the composition step comprises at least one step of detecting at least one element of interest from the first zone of interest.

According to one example, the step of composition comprises a step of determining a center of the first zone of interest and a step of centering the first reformatted zone according to said center of the first zone of interest.

According to one example, said center of the first zone of interest is taken from at least: a geometric center according to the geometry of the first zone of interest, a colorimetric center, a geometric center according to the morphology of an element detected in the first zone of interest, a center according to a detection of protruding elements in the first zone of interest, a center according to a detection of a displacement of the first zone of interest through several consecutive images.

According to one example, the first display format is shaped so that the first plurality of images is displayed on a display device having a main spatial extension extending in a horizontal direction, and the second display format is shaped so that the second plurality of images is displayed on a display device having a main spatial extension extending in a vertical direction.

According to one example, the first display format has a length dimension L1 and a height dimension H1, and the second display format has a length dimension L2 and a height dimension H2, and L1 is different, preferably higher, than L2, and H1 is different, preferably lower, than H2.

According to one example, the first display format is a 16:9 format and the second display format is a 9:16 format.

According to one example, steps a to e are performed by said computer data processing device in real time, preferably the source stream being broadcast live, and preferably intended to be displayed live by a display device.

According to one example, the visual data includes audiovisual data.

According to one example, the method comprises, before the step of identifying the first formatting model, at least one step of receiving said source stream by the computer data processing device.

According to one example, the method comprises, after the step of composing at least one reformatted image, a step of broadcasting said transposed stream to at least one of: a communication network, a display device .

According to one example, steps a to d are performed on each of the images of the first plurality of images or at least on at least X images per second of the first plurality of images of the source stream, with X≥24.

According to one example, steps b and c are reversed.

According to an example, a plurality of areas of interest is identified during step b.

According to an example, several zones of interest of the plurality of zones of interest have different spatial extension dimensions from each other.

The present invention relates to a clever method allowing the transposition of a stream of visual data shaped according to a first display format into a transposed stream of visual data shaped into a second display format, different from the first.

The first display format is shaped to allow optimum display on first display devices having for example an orientation and a conformation in which the height of the display surface is less than the length of the display surface, for example a 16:9 conformation common today.

The second display format is shaped to allow optimum display on display devices having for example an orientation and a conformation in which the height of the display surface is greater than the length of the display surface, for example a conformation in 9:16 current today at the level of the smartphone when they are held vertically.

Advantageously, the second display format is itself shaped to allow optimum display on second display devices having for example an orientation and a conformation in which the ratio between the height of the display surface and the width of the surface The display surface differs from the ratio between the height of the display surface and the width of the display surface of the first display devices.

Also, it is common today for multimedia content of the video type to be formatted to be displayed on 16:9 screens and for reasons of practicality or viewing locations, the user decides to hold his smartphone according to a vertical orientation, so that the content displayed there, that is to say a video formatted in 16: 9, retains its display format thus reducing the display area of the video on the phone of the 'user.

More generally, it is common for the content of a visual data stream to be shaped according to any first display format and for this format to not be best suited to the display device of said stream. The present invention makes it possible to transpose a source stream of visual data from any display format to a display format optimized for display on a display device different from the first. By optimized display format is meant a display format allowing the use of at least 60, preferably 80 and advantageously 90% of the display surface of a display device considered. For example, a 16:9 display format on a 9:16 screen covers only 30% of its surface. The present invention tends to significantly increase this coverage by transposing a video stream shaped according to a 16 format: 9 into a video stream shaped according to a 9:16 format for example.

The problem, previously presented, is at least partly solved by the present invention which allows a decomposition then a recomposition of the source video stream so that the essential of the visual information can be distributed on the display surface of the telephone for example, and this independently of the orientation chosen by the user.

The present invention also finds application in the transposition of visual data streams displayed on advertising display devices having a screen whose format is different from that of the visual data of the source stream.

In a particularly clever way, the present invention makes it possible to transpose a video formatted according to a first display format into a video formatted according to a second display format. The present invention thus allows the adaptation of a visual content to the display device on which it is broadcast.

According to a preferred embodiment, the present invention makes it possible to transpose, in an automated manner and in real time, a video, formatted according to a first display format, into a video, formatted according to a second display format, different from the first display size.

A classic example is newscasts designed to be viewed in a TV format, while many users use a vertically placed mobile phone to play this content. The present invention, by recomposing the visual content, allows the user better viewing comfort of the video.

As discussed below, according to one embodiment, the source stream can comprise a plurality of visual data streams, called input streams. For example, the source stream can include between 1 and N input streams. As also specified later, areas of interest can be extracted from one or more of these input streams. For example, an area of interest including a clock display or a scrolling banner can be extracted from a different input stream than the one used to extract a person's face, for example. The present invention thus makes it possible to use a plurality of input streams to compose a reformatted image and thus produce said transposed stream. For example, the different input streams may correspond for some to the visual data streams from different cameras.

We will now describe the invention according to an embodiment illustrated through Figures 1 to 8.

FIG. 1 represents a system in which a computer server 300, comprising a computer program product, preferably recorded on a non-transitory medium, comprising instructions, which when carried out by at least one of a processor and a computer, cause the at least one of the processor and the computer to execute said method 100 according to the present invention. The computer server 300 is configured to receive 310 from a transmission station 200 of a source stream. This source stream comprises visual data shaped according to a first display format 30. It will be noted that in the present invention visual data can also contain audio-visual data. This source stream is intended to be displayed by a display device 400 for example of the television type. The computer server 300 then receives this source stream and transposes it into a transposed stream of visual data, which can also include audiovisual data. The visual data of the transposed stream is shaped according to a second display format 40. This second display format 40 is different from the first display format 30.

The computer server 300 is configured to transmit 320 this transposed stream to a display device 500, for example a smart telephone.

According to one embodiment, the computer server 300 is located at a distance, preferably at more than 1 kilometer, from the place of production, of design of the source stream.

According to one embodiment, the display device 500 is a mobile device, such as for example a smart phone.

According to another embodiment, the display device 500 is a fixed device, such as for example an advertising or information display screen in a public place, and of which for example the main extension is vertical.

FIG. 2 represents, according to one embodiment, the transposition method 100.

Advantageously, the method 100 according to an embodiment of the present invention ensures the transposition of a source stream of visual data shaped according to a first display format 30 into a transposed stream of visual data shaped according to a second display format. display 40.

Preferably, the first display format 30 is intended to be displayed by a device 400 whose display surface is arranged in a first spatial orientation, in particular along a main direction of horizontal extension.

Preferably, the second display format 40 is intended to be displayed by a device 500 whose display surface is arranged in a second spatial orientation, in particular along a main direction of vertical extension.

It will be noted that the second display format 40 is advantageously different from the first display format 30.

As previously indicated, the source stream includes a first plurality of images. Each image of the first plurality of images is shaped according to the first display format 30. Advantageously, each image of the first plurality of images is shaped according to a first formatting model 10. a particular arrangement of elements in an image, we will present examples of this later.

1. Identification 110 du premier modèle de mise en forme 10 ;
2. Détection 120 d’au moins une première zone d’intérêt 1a dans une première image prise parmi la première pluralité d’images ;
3. Sélection 130 d’un deuxième modèle de mise en forme 20 en fonction du premier modèle de mise en forme 10 et/ou de ladite première zone d’intérêt 1a et/ou du deuxième format d’affichage 40, le deuxième modèle de mise en forme 20 étant avantageusement différent du premier modèle de mise en forme 10 ;
4. Composition 140 d’une image reformatée selon le deuxième format d’affichage 40 et selon le deuxième modèle de mise en forme 20 à partir d’une partie au moins de la première zone d’intérêt 1a et d’une partie au moins de la première image ; Puis,
5. Répétition 150 des étapes 110 à 140 pour une pluralité d’images de la première pluralité d’image de sorte à former ledit flux transposé de données visuelles comprenant une deuxième pluralité d’images conformée selon le deuxième format d’affichage 40.

In a particularly advantageous manner, the method 100, according to one embodiment, comprises the following steps which are carried out by a computer data processing device, for example by the computer server 300 mentioned earlier:

1. Identification 110 of the first formatting model 10;
2. Detection 120 of at least one first area of interest 1a in a first image taken from among the first plurality of images;
3. Selection 130 of a second layout model 20 as a function of the first layout model 10 and/or of said first area of interest 1a and/or of the second display format 40, the second layout model form 20 being advantageously different from the first formatting model 10;
4. Composition 140 of a reformatted image according to the second display format 40 and according to the second formatting model 20 from at least part of the first area of interest 1a and from at least part of the first image ; Then,
5. Repetition 150 of steps 110 to 140 for a plurality of images of the first plurality of images so as to form said transposed stream of visual data comprising a second plurality of images shaped according to the second display format 40.

According to one embodiment, the method comprises, after the step of composition 140, a step of displaying the transposed stream, that is to say the second plurality of shaped images according to the second display format 40 , on the display device 500.

It will be noted that in the case where the source stream comprises a plurality of input streams, the following steps are carried out on at least 2, preferably on each input stream: the identification 110 of a first layout model form 10, the detection 120 of at least a first zone of interest 1a in a first image taken from among a plurality of images. According to this embodiment, the selection 130 of a second formatting model 20 is performed as a function of the first formatting model 10 specific to each input stream considered and/or of said first zone of interest 1a specific to each stream considered and/or of the second display format 40.

In the present invention, an area is defined as part of an image, part of visual data, so that an area comprises at least part of an image, at least part of visual data. For example, an area of interest 1a in an image comprises a portion of said image, and a reformatted area 1b from at least part of an area of interest 1a comprises at least a portion of the image that includes said area of interest 1a. It will be noted that according to a preferred embodiment, the reformatted zone 1b is an extraction of a zone from the zone of interest 1a, for example by extracting the salient elements from 1a, such as, for example, a face, or from the movement . The reformatted zone 1b is then defined as a part of the zone of interest 1a defined to maximize the transcription of these salient elements .

Advantageously, the identification step 110 of the first shaping model 10 is carried out by comparing the first shaping model 10 with a first database of shaping models. This comparison can be made in several ways. For example, the source stream can include an identifier specific to the formatting model 10 used in the image considered. According to another preferred method, the present invention uses the resources of an artificial intelligence, previously trained in the classification of layout models, so as to identify the first layout model 10 among the plurality of layout models. form of the first database.

Advantageously, the detection step 120 of a first zone of interest 1a can be carried out in various ways and using various techniques. It will be noted that detection of a first zone of interest in a first image taken from among the first plurality of images can also mean the fact of extracting said first zone of interest from the first image. Also, the detection step 120 can include the extraction of the first area of interest (1a) from the first image.

Advantageously, and according to one embodiment, the method comprises, after the step of detecting the first zone of interest 1a in the first image, a step of extracting at least part of the first zone of interest from an image of the source stream.

It will be noted that an area of interest 1a, 2a, 3a, 4a, 5a, 6a can comprise an element of interest such as a face for example. According to one embodiment, the step of detecting a zone of interest 1a, 2a, 3a, 4a, 5a, 6a can comprise the detection of an element of interest. An element of interest can for example be a person, a face, an object, a vehicle, a ball, a map, a title, a clock, a logo, etc…

According to one embodiment, an area of interest 1a, 2a, 3a, 4a, 5a, 6a can be detected through the first shaping model 10. Indeed, it may be that this first shaping model 10 includes an arrangement of visual data such that the areas of interest 1a, 2a, 3a, 4a, 5a, 6a are known as soon as the first layout model 10 is identified. For example, the first database may include information indicating that such or such location of the shaping model is an area of interest 1a, 2a, 3a, 4a, 5a, 6a. According to another example, the source stream can include data indicating a positioning of the areas of interest 1a, 2a, 3a, 4a, 5a, 6a.

According to an example not illustrated, at least part of the areas of interest 1a, 2a, 3a, 4a, 5a, 6a can come from a first input stream and another part from a second input stream, different of the first, the first input stream and the second input stream being comprised by the source stream .

In another example, a zone of interest 1a, 2a, 3a, 4a, 5a, 6a can also be detected by an analysis of contours 11, by searching for contours 11, for example closed, which therefore delimit a zone of the image, zone which can be a zone of interest 1a, 2a, 3a, 4a, 5a, 6a.

According to another example, an artificial intelligence, of the neural network type for example, can be used. Preferably, this artificial intelligence is previously and/or regularly trained to identify areas of interest, for example particular elements on an image such as for example objects, people, faces, a weather map, etc. The areas of the image where these elements are found can therefore be considered as areas of interest 1a, 2a, 3a, 4a, 5a, 6a. Advantageously, these areas of interest 1a, 2a, 3a, 4a, 5a, 6a can come from various input streams.

According to another example, the type of scene can also be identified by an artificial intelligence and therefore predetermined zones can be considered as zones of interest 1a, 2a, 3a, 4a, 5a, 6a.

As we will see later, for each identified area, artificial intelligence can calculate image by image what is the optimized center point of the area considered in order to optimize the quality of the area of interest when the area is transposed. according to the second display format 40. This makes it possible, for example, to center on faces, or on salient and/or moving objects.

A salient object is understood to mean an object which is easily noticed in an image, which stands out from this image in a clear manner, which may present a different contrast for example compared to the rest of the image, or a different colorimetry. For example, a face can be a protruding object against the background of an image.

In a particularly advantageous manner, the second formatting model 20 can be selected from a second database of formatting models.

According to one embodiment, this selection takes place according to pre-established editorial rules. For example, these rules assign the first formatting template 10, and preferably all information in areas of interest, a second formatting template 20.

Advantageously, this second database is different from the first database of formatting models. In a manner similar to the first formatting model 10, the second formatting model comprises rules for positioning the area(s) of interest 1a, 2a, 3a, 4a, 5a, 6a. We will illustrate this later.

1. Une étape de disposition, en fonction dudit deuxième modèle de mise en forme 20, d’une première zone reformatée 1b correspondant à une partie au moins de la première zone d’intérêt 1a ;
2. Une étape de disposition, en fonction dudit deuxième modèle de mise en forme 20, d’une deuxième zone reformatée 2b correspondant à une partie au moins de la première image ;
3. Une étape de découpage d’une partie au moins de la première zone d’intérêt 1a de sorte à générer la première zone reformatée 1b ;
4. Une étape de découpage d’une partie au moins de la première image de sorte à générer la deuxième zone reformatée 2b ;
5. Une étape de cadrage d’une partie au moins de la première zone d’intérêt 1a de sorte à générer la première zone reformatée 1b ;
6. Une étape de cadrage d’une partie au moins de la première image de sorte à générer la deuxième zone reformatée 2b ;
7. Une étape de détermination d’un centre 1c de la première zone d’intérêt 1b et une étape de cadrage selon ce centre 1c pour générer la première zone reformatée 1b ;
8. Une étape de détermination d’un centre d’une partie au moins de l’image et une étape de cadrage selon ce centre pour générer la deuxième zone reformatée 2b.

In a particularly advantageous manner, the step 140 of composing the reformatted image according to the second display format 40 and according to the second formatting model 20 can comprise at least one of the following steps, for example:

1. A step of arranging, according to said second formatting model 20, a first reformatted zone 1b corresponding to at least part of the first zone of interest 1a;
2. A step of arranging, according to said second formatting model 20, a second reformatted zone 2b corresponding to at least part of the first image;
3. A step of cutting out at least part of the first zone of interest 1a so as to generate the first reformatted zone 1b;
4. A step of cutting out at least part of the first image so as to generate the second reformatted zone 2b;
5. A step of framing at least part of the first zone of interest 1a so as to generate the first reformatted zone 1b;
6. A step of framing at least part of the first image so as to generate the second reformatted zone 2b;
7. A step of determining a center 1c of the first zone of interest 1b and a step of framing according to this center 1c to generate the first reformatted zone 1b;
8. A step of determining a center of at least part of the image and a step of framing according to this center to generate the second reformatted zone 2b.

According to one embodiment, the framing step makes it possible to define a reformatted zone comprising one or more elements of interest. According to this embodiment, and for example non-limiting, the framing step makes it possible to cut out certain parts such as hands, parts of the body or other unnecessary elements relative to the element of interest considered, here a face for example non-limiting.

According to one embodiment, the second reformatted zone 2b can be obtained from at least part of a second zone of interest 2a. Indeed, said part at least of the first image can be a second zone of interest 2b.

According to one embodiment, the center of an image or an area can be a geometric center depending on the shape of the area or the image, a colorimetric center depending on a color surface in the area or image, a morphological center according to a shape present in the area or the image.

Preferably, it will be noted that during a step of centering an area, this step can correspond to centering on a moving element so as to take account of the movement of an element of interest and thus to follow it by one image to another.

Advantageously, by repeating the steps previously described, the present invention allows the generation of a transposed stream of visual data comprising a second plurality of images shaped according to the second display format 40. Preferably, each image comprises at least two zones reformatted 1b and 2b arranged according to the second formatting model 20. According to one embodiment, the second reformatted zone may correspond to the whole of the considered image of the source stream. And according to another example, the first reformatted zone 1b can correspond to the whole of the considered image of the source stream, in this case the second reformatted zone 2b can correspond to a predetermined visual content such as advertising banners for example or a personalized background according to the considered source stream. According to one embodiment, the first zone of interest 1a can correspond to the entire considered image of the source stream. The second part of the reformatted image then comes either from the source stream or from another stream or from predetermined visual elements.

According to one embodiment, during the composition of the reformatted image, the reformatted zones 1b and 2b can be at least partly in contact at the level of at least one of their sides, one of their contours.

According to another embodiment, during the composition of the reformatted image, the reformatted zones 1b and 2b can be at least partly superimposed with each other.

FIG. 3 illustrates in particular the step 120 of detecting areas of interest (there are 6 in this figure) and the step 140 of composing a reformatted image. In particular, this reformatted image includes at least some of the areas of interest detected during step 120.

It will be noted that according to one embodiment, the number of areas of interest resulting from step 120 may be different from the number of reformatted areas forming at least in part the reformatted image composed in step 140.

In this figure, an image of the first plurality of images of the visual data source stream is illustrated. This image is shaped according to a first display format 30. This first display format 30 can be defined by its height extension dimension H1 and by its length extension dimension L1. This image is shaped according to a first formatting model 10. For example, this model comprises a layout in which the image is divided into two parts, an upper part and a lower part. The upper part and the lower part are each divided into 3 zones. For example, in the upper part one of the 3 zones can be larger 2a than the other two 1a and 3a. And still according to this embodiment illustrated in FIG. 3, the three zones 4a, 5a and 6a of the lower part can be of identical dimensions to one another.

It will be noted that advantageously, each zone is delimited by at least one outline 11.

During the step 120 of detecting areas of interest, and according to this example, the invention allows the detection of six areas of interest 1a, 2a, 3a, 4a, 5a, 6a.

Then, during the step 140 of composition of the reformatted image, at least part of each of these six areas of interest 1a, 2a, 3a, 4a, 5a, 6a is processed to obtain six reformatted areas 1b, 2b , 3b, 4b, 5b, 6b. These six reformatted areas 1b, 2b, 3b, 4b, 5b, 6b are then arranged according to a second formatting model 20. These six reformatted areas 1b, 2b, 3b, 4b, 5b, 6b then form a reformatted and shaped image according to a second display format 40.

It will be noted that advantageously, each zone of interest 1a, 2a, 3a, 4a, 5a, 6a can be classified automatically, and advantageously via artificial intelligence. Once classified, an area of interest, or at least the reformatted area 1b, 2b, 3b, 4b, 5b, 6b resulting from this area of interest 1a, 2a, 3a, 4a, 5a, 6a, is positioned according to the second formatting model 20 according to its class, and preferably according to predefined editorial choices. For example, the second shaping model 20 can provide that the faces are arranged in particular areas of the image and that the decor or other elements of the scene are arranged in other areas. In figure 3, the reformatted zones 1b, 3b, 4b and 6b can correspond to faces and the zones 2b and 5b can correspond to shots of a larger scene for example.

In figure 4, the principle is the same as in figure 3. Areas of interest 1a, 2a, 3a and 4a are detected, then a reformatted image is composed.

According to one embodiment, the areas of interest 3a and 4a can also include timestamps, titles or subtitles embedded in the image for example. Consequently, the present invention is designed to allow their identification and their processing as areas of interest and their arrangement in the reformatted image, in the case where the second formatting model 20 so provides.

It will be noted here that the first formatting model 10 can change over time. This may or may not lead to modifications of the second formatting model 20. Indeed, the present invention has the advantage of being able to carry out this transposition method 100 in real time, that is to say from a source stream corresponding to a live broadcast of live generated content.

In addition, the present invention also makes it possible to apply this transposition method 100 to content recorded on a data medium, for example so as to broadcast according to the second display format 40 a content originally created according to the first format. display 30. Let us take here the example of video on demand, and the case of a film, the present invention allows a transposition of the film from a first display format 30, generally 16:9, into a second display format 40, for example 9:16 in the case where the user would watch the film on his mobile telephone by holding it vertically for example.

Let us also take the example of video on demand here, and the case of a sports match, the present invention allows a transposition of the video of the match from a first display format 30, generally 16:9, into a second display format 40, for example 9:16 in the case where the user would watch the match on his mobile telephone by holding it vertically for example. The present invention makes it possible to highlight certain areas of interest in the images of the match, such as, for example, the position of a player, the score or even a particular action.

Advantageously, the present invention applies both to a live source stream and to a source stream comprising or originating from a computer file.

According to one embodiment, the source stream can comprise or be a computer file.

Figure 5 illustrates another first shaping model 10 as well as another second shaping model 20.

According to one embodiment, for each first formatting model 10 corresponds one or more second formatting models 20.

FIG. 6 illustrates an embodiment in which an image of the source stream comprises a single zone of interest 1a, for example, without limitation, a face, or a weather map. In this case, and preferably, the method 100 detects this first zone of interest 1a and then composes the reformatted image by arranging the first reformatted zone 1b, originating from a part of the first zone of interest 1a, and by arranging the second reformatted zone 2b corresponding for example to a part of the image of the source stream, or even to the whole image of the source stream. The method 100 thus allows highlighting of the area of interest 1a while keeping the entire scene displayed. This embodiment can be advantageous for scenes of the football match type for example or for the presentation of an element on the screen such as a weather map.

FIG. 7 illustrates an embodiment in which the source stream comprises a first stream called clean stream and a second stream called dressed stream.

Preferably, the clean stream comprises visual data comprising so-called raw images 50 shaped according to the first display mode 30 and according to the first formatting model 10.

Preferably, the dressed stream comprises the same images as the clean stream, shaped in the same way, and further comprises one or more dressing elements 61 configured to dress, decorate, at least one image of the clean stream. In this way, the dressed stream comprises so-called dressed images 60. These dressing elements 61 can be a title, a subtitle, an insert, a scrolling banner, an advertisement for an upcoming program, etc. These elements of skin 61 include various graphic elements generally superimposed on the raw images 50 of the clean stream. The dressed images 60 include the raw images 50 and at least one skin element 61.

According to one embodiment, as mentioned earlier, the source stream can comprise from 1 to N input streams. Each input stream including visual data. This set of visual data from the different input streams is advantageously used to compose the transposed stream. One or more of these input streams may be dressed streams. For example, a first input stream may correspond to the clean stream, a second input stream may correspond to the dressed stream, and a third input stream may include the dressing elements 61, while other streams of input may include different camera angles of the same scene. In this case, and preferably by means of a previously trained artificial intelligence, and/or on the basis of pre-established rules, the transposed flow is produced from at least part of some of these flows. The transposed stream advantageously comprises the reformatted images from compositions of reformatted zones originating from at least part of these various input streams.

According to this embodiment, the method 100 allows the detection of an area of interest 1a, 2a, 3a in a raw image 50 and/or in a dressed image 60.

Then, the reformatted area 1b, 2b or 3b can be obtained from the raw image 50 or from the dressed image 60 depending on the circumstances, such as depending on the part to be extracted from the area of interest 1a, 2a or 3a to form the reformatted area 1b, 2b or 3b. Indeed, if for example, the reformatted zone 1b, 2b or 3b is contained in the dressed image 60, that is to say that the dressing element(s) 61 do not partially cover the zone intended to form the reformatted zone 1b, 2b or 3b, then the dressed flux is sufficient to obtain the desired reformatted zone 1b, 2b, 3b. On the other hand, if the area intended to form the reformatted area 1b, 2b or 3b is covered at least in part by a dressing element 61 in the dressed image 60, the method 100 can use the corresponding raw image 50 of the stream clean to generate the reformatted zone 1b, 2b, 3b, so as not to be bothered by the trim element 61 in question. It will be noted that generally, the dressed stream corresponds to the stream displayed by a display device 400 having a horizontal main extension dimension, such as a television for example, advantageously in a 16:9 format.

According to one embodiment, the method 100 uses an artificial intelligence to virtually remove the dressing elements 61 from a dressed image 60 and thus partially reconstitute at least one artificial raw image 50 from a dressed image 60. This situation can be considered for example when a clean flow is not available.

According to a preferred embodiment, the areas of interest are defined so that they do not present any overlap with the trim elements 61; that is to say so that the areas of interest which do not fully include at least one trim element 61, do not include a part or a portion of a trim element 61. This situation can be considered for example when a clean flow is not available.

FIG. 8 illustrates two steps 141 and 142 for obtaining a reformatted zone 1b from a zone of interest 1a. For example, once the zone of interest 1a is detected, a center 1c is defined there 141, this center 1c has been explained previously. Then, a cropping step 142 can be performed so as to form the reformatted zone 1b from the cropping of the zone of interest 1a.

According to one embodiment, the present invention provides centering 141, preferably image by image, of each reformatted zone so as to be able to keep the element of interest identified in the zone of interest at the center of the reformatted zone.

According to one embodiment, the present invention provides cropping 142 of the area of interest to form the reformatted area most suited to both the element of interest of the area of interest and the second layout model. form 20. This ensures a framing of the elements of interest.

The present invention thus allows a transposition of a source stream intended to be displayed according to a first display format into a transposed stream intended to be displayed according to a second display format different from the first.

The present invention provides, preferably image by image, an identification of a first display format, also called input display format, the identification of at least one zone of interest comprising for example an element of interest, the selection of a second formatting model, also called output display format, then the composition of a reformatted image according to this second formatting model and according to the second display format. This reformatted image comprises at least two parts, a first part corresponding to the first reformatted zone and a second part corresponding to at least part of the considered image of the source stream. This part of the considered image of the source stream can be a second area of interest. According to an example, the second part of the reformatted image may correspond to the entire considered image of the source stream.

Finally, it will be noted that, according to a particular embodiment, when the source stream includes a mention of the conservation of the first display format, each image of the first plurality of images comprising this mention is arranged according to a second layout format. form in which the images of the source stream considered are arranged while retaining their formatting format and the rest of the second formatting model is filled with additional visual elements such as, for example, advertising elements from another stream or from the source stream itself.

The invention is not limited to the embodiments described above and extends to all the embodiments covered by the claims.

List of reference numbers:
1a First area of interest
1b First zone reformatted
1c Center of the first area of interest
2a Second area of interest
2b Second zone reformatted
3a Third area of interest
3b Reformatted Third Zone
4a Fourth area of interest
4b Reformatted Fourth Zone
5a Fifth area of interest
5b Fifth reformatted zone
6a Sixth area of interest
6b Sixth zone reformatted
10 First formatting template
11 Outlines
20 Second formatting template
30 First display format
40 Second display format
50 Raw Image
60 Dressed Image
61 Trim element
100 Transposition process
110 Identifying the first formatting model
120 Detection of at least a first area of interest
130 Selecting a second formatting model
140 Composition of at least one reformatted image
141 Centering the first region of interest
142 Framing the first area of interest
150 Repeat steps 110, 120, 130 and 140
200 Source Stream Transmission System
210 Transmit source stream to TV
300 computer server
310 Reception of the source stream by the computer server from the transmission system
320 Transmission of transposed stream
400 Display device according to the first display format
500 Display device according to the second display format
H1 Dimension of extension in height of an image shaped according to the first display format
L1 Length extension dimension of an image shaped according to the first display format
H2 Dimension of extension in height of an image shaped according to the second display format
L2 Dimension of extension in length of an image shaped according to the second display format

Claims (39)

Method for transposition (100) of at least one source stream of visual data shaped according to a first display format (30) into a transposed stream of visual data shaped according to a second display format (40), the second format display (40) being different from the first display format (30), the source stream comprising a first plurality of images, each image of the first plurality of images being shaped according to the first display format and according to a first formatting model (10), the method (100) comprising at least the following steps performed by at least one computer data processing device:

1. identifying (110) the first shaping model (10) from among at least a first database of shaping models;
2. Detection (120) of at least a first area of interest (1a) in at least a first image taken from among the first plurality of images;
3. Selection (130) of at least one second shaping model (20) taken from at least one second database of shaping models, said selection (130) of said second shaping model (20) being depending on at least one of: the first layout model (10), said first region of interest (1a), the second display format (40);
4. Composition (140) of at least one reformatted image according to the second display format (40) and according to the second formatting model (20), the composition step (140) comprising at least one arrangement step, based on said second shaping model (20), a first reformatted area (1b) corresponding to at least part of the first area of interest (1a) and a second reformatted area (2b) corresponding to at least part of said first image; And,
5. Repetition (150) of steps a to d for a plurality of images of the first plurality of images of said source stream so as to form said transposed stream of visual data comprising a second plurality of images shaped according to the second display format (40).

Method (100) according to the preceding claim, in which the source stream comprises a stream, called clean stream and another stream, called dressed stream, the clean stream comprising a plurality of images called raw images (50) and the dressed stream comprising a plurality of images called dressed images (60), the plurality of dressed images (60) comprising the plurality of raw images (50) and at least one dressing element (61) of at least one raw image (50 ). Method (100) according to the preceding claim, in which the skin element (61) comprises at least one of: a timestamp, a title, a subtitle, an image, a graphic animation. A method (100) according to any one of the two preceding claims wherein the first area of interest (1a) is extracted from an image of the plurality of raw images (50) corresponding to the first image. A method (100) according to any of claims 2 or 3 wherein the first area of interest (1a) is extracted from an image of the plurality of stitched images (60) corresponding to the first image. A method according to any one of the four preceding claims wherein the source stream comprises a plurality of input streams comprising a plurality of images, at least one input stream is the proper stream, at least one other input stream is the dressed flux. Method according to the preceding claim, in which at least one input stream comprises the said at least one dressing element (61). A method according to any preceding claim wherein at least one input stream comprises a plurality of images obtained from a first camera filming a scene from a first viewing angle, and wherein at least one further input comprises a plurality of images obtained from a second camera filming said scene according to a second angle of view different from the first angle of view. Method (100) according to any one of the preceding claims, in which the step of detecting (120) the first zone of interest (1a) comprises at least one step of detecting contours (11) in the first image, a zone delimited by at least one contour (11) forming a zone of interest. Method (100) according to any one of the preceding claims, in which the step of detecting (120) the first zone of interest (1a) comprises at least one step of identifying a type of scene among a plurality of types of predetermined scenes, each type of scene comprising at least one area of interest (1a). Method (100) according to any one of the preceding claims, in which the step of detecting (120) the first zone of interest (1a) comprises at least one step of detecting at least one element of interest chosen from among at least: a face, a body, an object, a vehicle, a geographical map, a graphic element such as a logo, a title, a clock. A method (100) according to any preceding claim wherein the first area of interest (1a) comprises at least one of: a face, a body, an object, a vehicle, a geographical map, a weather map . A method (100) according to any preceding claim wherein the step of detecting the first region of interest (1a) further comprises detecting a second region of interest (2a) in the first image and wherein the second reformatted area (2b) corresponds to at least part of said second area of interest (2a). Method (100) according to any one of the preceding claims, in which the step of detecting (120) the first zone of interest (1a) is carried out by an artificial intelligence trained to identify different types of zones of interest and/ or items of interest. A method (100) according to any preceding claim wherein the first layout model (10) comprises a first arrangement of a first plurality of areas for comprising a plurality of visual data and wherein the second layout model layout (20) comprises a second arrangement of a second plurality of areas intended to include a plurality of visual data, the second arrangement being different from the first arrangement and the second plurality of visual data being at least partly included in the first plurality of visual data. A method (100) according to any preceding claim wherein the step of identifying the first formatting model (10) comprises at least one step of receiving an identifier of said first formatting model (10 ), this identifier being associated with a formatting model contained in the first database of formatting models, preferably this identifier being transmitted by the source stream. A method (100) according to any one of claims 1 to 15 wherein the step of identifying the first fitness model (10) comprises at least one step of determining by an artificial intelligence the first fitness model (10) by comparison with at least one shaping model from the first database of shaping models. A method (100) according to any preceding claim wherein the step of compositing (140) further comprises formatting at least a portion of the first area of interest (1a) so as to generate the first area reformatted (1b). Method (100) according to the preceding claim, in which the formatting of at least part of the first zone of interest (1a) comprises a step of cutting out at least part of the first zone of interest (1a). Method (100) according to any one of the preceding claims, in which the composition step (140) comprises at least one step of extracting at least part of the first zone of interest (1a) so as to generate the first reformatted area (1b). Method (100) according to any one of the preceding claims, in which the step of composition (140) comprises at least one step of detecting at least one element of interest from the first zone of interest (1a). Method (100) according to any one of the preceding claims, in which the step of composition (140) comprises a step of determining a center (1c) of the first zone of interest (1a) and a step of centering the first reformatted zone (1b) according to said center (1c) of the first zone of interest (1a). Method (100) according to the preceding claim, in which the said center (1c) of the first zone of interest (1a) is taken from at least: a geometric center according to the geometry of the first zone of interest (1a), a center colorimetric, a geometric center according to the morphology of an element detected in the first zone of interest (1a), a center according to a detection of protruding elements in the first zone of interest (1a), a center according to a detection a displacement of the first zone of interest (1a) through several images in succession. A method (100) according to any preceding claim wherein the first display format (30) is shaped such that the first plurality of images are displayed on a display device (400) having a primary spatial extent extending in a horizontal direction, and in which the second display format (40) is shaped so that the second plurality of images is displayed on a display device (500) having a main spatial extension extending in a vertical direction. A method (100) according to any preceding claim wherein the first display format (30) has a length dimension L1 and a height dimension H1, and wherein the second display format (40) has a length dimension L2 and a height dimension H2, and wherein L1 is different, preferably greater, than L2, and H1 is different, preferably less, than H2. A method (100) according to any preceding claim wherein the first display format (30) is a 16:9 format and wherein the second display format (40) is a 9:16 format. Method (100) according to any one of the preceding claims, in which steps a to e are carried out by said computer data processing device in real time, preferably the source stream being broadcast live, and preferably intended to be displayed live through a display device. A method (100) according to any preceding claim wherein the visual data includes audiovisual data. Method (100) according to any one of the preceding claims comprising, before the step of identifying (110) the first formatting model (10), at least one step of receiving said source stream by the processing device of computer data. Method (100) according to any one of the preceding claims comprising, after the step of composing (140) at least one reformatted image, a step of broadcasting said transposed stream to at least one of: a communication network, a display device (500). A method (100) according to any preceding claim wherein steps a through d are performed on each of the images of the first plurality of images or at least on at least X images per second of the first plurality of images of the source stream, with X≥24. A method (100) according to any preceding claim wherein steps b and c are interchanged. A method (100) according to any preceding claim wherein a plurality of areas of interest are identified in step b. Method (100) according to the preceding claim, in which several zones of interest of the plurality of zones of interest have different spatial extension dimensions from each other. A visual data stream transposed from a source visual data stream by the method (100) of any preceding claim, the visual data of the source stream being shaped into a first display format (30) and the visual data of the transposed stream being shaped according to a second display format (40), different from the first display format (30), preferably the transposed stream being intended to be viewed on a mobile display device (500). Computer program product, preferably recorded on a non-transitory medium, comprising instructions, which when performed by at least one of a processor and a computer, cause the at least one of the processor and the computer, executes the method (100) according to any one of claims 1 to 34. Computer server (300) comprising at least the computer program product according to the preceding claim and being configured to receive at least one source stream of visual data and to transmit at least one transposed stream according to Claim 35. System comprising at least one computer server (300) according to the preceding claim and at least one display device (500) configured to display the transposed stream according to claim 35. Method for displaying at least one transposed stream according to claim 35 on a display device (500), the method comprising at least the following steps:

1. Reception (310), by a computer server (300) according to claim 37, of at least one source stream of visual data shaped according to a first display format (30);
2. Transposition of the source stream so as to obtain said transposed stream;
3. Transmission (320) by the computer server (300) of said transposed stream to at least one display device (400).