WO2021214097A1

WO2021214097A1 - Method for transposing an audiovisual stream

Info

Publication number: WO2021214097A1
Application number: PCT/EP2021/060307
Authority: WO
Inventors: Thomas Menguy
Original assignee: Wildmoka
Priority date: 2020-04-22
Filing date: 2021-04-21
Publication date: 2021-10-28
Also published as: FR3109686B1; FR3109686A1

Abstract

The invention relates to a method (100) for transposing a source stream of visual data shaped according to a first display format into a transposed stream of visual data shaped according to a second display format, the method (100) comprising at least the following steps: a. identifying (110) a first shaping model; b. detecting (120) at least one area of interest in at least one image; c. selecting (130) at least one second shaping model; d. composing (140) at least one image reformatted according to the second display format and according to the second shaping model, from at least one portion of the area of interest and from at least one portion of the image; and e. repeating (150) steps a to d so as to form the transposed stream of visual data.

Description

"Method of transposing an audiovisual stream" TECHNICAL FIELD OF THE INVENTION

The present invention relates to the field of audiovisual processes. It finds a particularly advantageous application in the field of audiovisual broadcasting of content on various types of mobile display devices.

STATE OF THE ART In the history of the audiovisual industry, with the development of particularly mobile terminals incorporating a display device, such as smart phones (usually referred to as smartphone), there has never been so much different types of display devices and therefore screen formats. In general, these screens extend in a main direction and generally have a rectangular shape. When considering a smartphone, for example, with respect to audiovisual content in a 16: 9 aspect ratio, a horizontal screen orientation (length of the screen in the horizontal direction) is advantageous for comfort. visual of the user.

However, some users prefer to use their screen vertically rather than horizontally, which results in degraded comfort because the content to be displayed is suitable for horizontal viewing.

In addition, the content becomes difficult to view on media where the orientation is not easily adjustable, hence the emergence of new technologies for modifying the orientation of televisions, for example; hence also the emergence of techniques such as that implemented by the video editing software called BeeCut making it possible in particular to transform visual content from a 16: 9 format to a 9:16 format and providing means

editing to add overlays, transitions, or filters to reformatted visual content, so you can customize it.

Thus, the techniques most widely used at present for best accommodating the display of an audiovisual stream as a function of the display device consist of hardware solutions for modifying the orientation of the display device to improve the comfort of the display. user or in software solutions requiring their implementation by an informed person to create a reformatted video montage relative to a source visual content.

These hardware or software solutions therefore have limitations and obligations to change orientation or implementation for the user.

A non-limiting object of the present invention is therefore to provide a solution to at least one of these limitations and / or obligations.

The other objects, features and advantages of the present invention will become apparent on examination of the following description and the accompanying drawings. It is understood that other advantages can be incorporated.

SUMMARY OF THE INVENTION

To achieve this objective, according to one embodiment there is provided a method of transposing at least one source stream of visual data, preferably audiovisual, shaped according to a first display format into a transposed stream of visual data, preferably audiovisual. , conformed according to a second display format, the second display format being different from the first display format, the source stream comprising a first plurality of images, each image of the first plurality of images being conformed according to the first display format and according to a first formatting model, the method comprising at least the following steps carried out by at least one computer data processing device: a. Identifying the first shaping model among at least a first shaping model database; b. Detection of at least a first area of interest in at least a first image taken from among the first plurality of images; vs. Selection of at least one second shaping model taken from at least a second database of shaping models, said selection of said second shaping model being a function of at least one of: the first layout model, said first area of interest, the second display format; d. Composition of at least one image reformatted according to the second display format and according to the second formatting model, the composition step comprising at least one step of arranging, as a function of said second formatting model, of a first reformatted area corresponding to at least part of the first area of interest and a second reformatted area corresponding to at least part of said first image; and, e. Repeating steps a to d for a plurality of images of the first plurality of images of said source stream so as to form said transposed stream of visual data comprising a second plurality of images shaped according to the second display format.

The source stream includes one stream, called clean stream, and another stream, called dressed stream. The clean stream includes a plurality of so called raw image images and the dressed stream includes a plurality of so called dressed images. The plurality of wrapped images includes the plurality of raw images and at least one wrapping element of at least one raw image.

This allows adaptation of a video stream to the display format of the display device considered for displaying said video stream.

This makes it possible to transpose a video stream formatted for example for a television set into a stream formatted for a mobile device whose display dimensions are different from those of the television set and this with better viewing comfort and reduced or even zero loss of information. contained in the initial stream.

In addition, if the area intended to form the reformatted area is at least partially covered by a dressing element in the dressed image, the method allows the use of the corresponding raw image of the clean stream to generate the reformatted area. , so as not to be hampered by the covering element in question.

This invention is particularly advantageous for producers and broadcasters of traditional television content, for example in a 16: 9 display format, so that they can automatically and preferably in real time transpose their content from the 16 format. : 9 to vertical video formats (9:16, 4: 5 or 2: 3) used for mobile phones and some social networks.

This allows the areas of interest to be intelligently selected and positioned relative to each other by adapting to the display format of the display device.

This invention is particularly suitable for news broadcasts whose content must be delivered live and for which the "verticalization" of the image could not be done manually in real time. The vertical content produced, that is to say the transposed flow in this case, is published on digital platforms such as social networks or mobile applications.

According to a non-limiting example, the first display format is intended to be displayed by a device, preferably mobile, arranged in a first spatial orientation, and the second display format is intended to be displayed by a device, preferably mobile, arranged in a second spatial orientation different from the first spatial orientation, preferably the first display format being mainly horizontal and the second display format being mainly vertical.

In one example, the display device can be a mobile device. For example, it could be a smart phone.

According to an alternative example, the device can be a fixed device. For example, it can be configured to be mounted on a wall. It may for example be an information screen in a public reception area, such as a shopping center or an airport. Often in these Places, fixed screens have shapes such that their main dimension extends vertically.

According to one example, the method comprises a step of displaying on a display device said transposed stream of visual data comprising the second plurality of images conformed according to the second display format.

Another aspect relates to a visual data stream transposed from a visual data source stream by the method according to the present invention, the visual data of the source stream being conformed according to a first display format and the visual data of the transposed flow being conformed according to the present invention. a second display format, different from the first display format, preferably the transposed stream being intended to be viewed on a mobile display device.

Another aspect relates to a computer program product, preferably recorded on a non-transient medium, comprising instructions, which when performed by at least one of a processor and a computer, cause the at least one among the processor and the computer, executes the method according to the present invention.

Another aspect relates to a computer server comprising at least the computer program product according to the present invention and being configured to receive at least one source stream of visual data and to transmit at least one transposed stream according to the present invention.

Another aspect relates to a system comprising at least one computer server according to the present invention and at least one display device configured to display the transposed stream according to the present invention.

Another aspect relates to a method for displaying at least one stream transposed according to the present invention on a display device, the method comprising at least the following steps: a. Reception, by a computer server according to the present invention, of at least one source stream of visual data conforming to a first display format; b. Transposition of the source stream so as to obtain said transposed stream, this transposition step being carried out by implementing steps a to e of the method according to the present invention; vs. Transmission by the computer server of said transposed stream to at least one display device.

BRIEF DESCRIPTION OF THE FIGURES

The aims, objects, as well as the characteristics and advantages of the invention will become more apparent from the detailed description of an embodiment thereof which is illustrated by the following accompanying drawings in which:

Figure 1 shows a system according to one embodiment of the present invention.

FIG. 2 represents certain steps of a method of transposing a visual data stream according to one embodiment. FIG. 3 illustrates steps 120 and 140 of the method of FIG. 2 according to an embodiment of the present invention.

Figure 4 illustrates the transposition of an image from a first display format to a second display format according to one embodiment of the present invention.

Figure 5 illustrates the transposition of an image from a first display format to a second display format according to one embodiment of the present invention.

Figure 6 illustrates the transposition of an image from a first display format to a second display format according to one embodiment of the present invention.

Figure 7 illustrates a nonconformed data flow and a conformed data flow.

FIG. 8 illustrates a step of centering a reformatted zone.

The drawings are given by way of example and are not limiting of the invention. They constitute schematic representations of principle intended to facilitate understanding of the invention and are not necessarily on the scale of practical applications. In particular, the dimensions are not representative of reality.

DETAILED DESCRIPTION

Before embarking on a detailed review of embodiments of the invention, optional features are listed below which may optionally be used in combination or alternatively:

According to one example, said step of detecting the first area of interest comprises a step of extracting the first area of interest from the first image.

According to one example, said step of selecting said second formatting model is a function of at least two and preferably three among: the first formatting model, said first area of interest, the second display format.

According to one example, the visual data of the source stream are intended to be displayed by a first type of display device and the visual data of the transposed stream are intended to be displayed by a second type of display device different from the first type of display. display device.

According to one example, the first and the second type of display devices differ by their orientation or more precisely by the direction in which they mainly extend.

According to one example, the dressing element comprises at least one of: a timestamp, a title, a subtitle, an image, a graphic animation.

In one example, the first area of interest is extracted from one image of the plurality of raw images corresponding to the first image.

In one example, the first area of interest is extracted from one image of the plurality of dressed images corresponding to the first image.

According to one example, the source stream comprises a plurality of input streams comprising a plurality of images, at least one input stream is the proper stream, at least one other input stream is the dressed stream. According to one example, at least one inlet flow comprises said at least one covering element.

According to one example, at least one input stream comprises a plurality of images obtained from a first camera filming a scene from a first viewing angle, and at least one other input stream comprises a plurality of images obtained from a first viewing angle. second camera filming said scene from a second viewing angle different from the first viewing angle.

According to one example, the step of detecting the first area of interest comprises at least one step of detecting contours in the first image, an area delimited by at least one contour forming an area of interest.

According to one example, the step of detecting the first zone of interest comprises at least one step of identifying a type of scene from among a plurality of types of predetermined scenes, each type of scene comprising at least one zone of interest.

According to one example, the step of detecting the first area of interest comprises at least one step of detecting at least one element of interest taken from at least: a face, a body, an object, a vehicle, a geographical map, a graphic element such as a logo, a title, a clock.

According to one example, the first area of interest includes at least one of: a face, a body, an object, a vehicle, a geographic map, a weather map.

According to one example, the step of detecting the first area of interest further comprises the detection of a second area of interest in the first image and the reformatted second area corresponds to at least part of said second area. interest.

According to one example, the step of detecting the first area of interest is performed by artificial intelligence trained to identify different types of areas of interest and / or elements of interest.

According to one example, the first shaping model comprises a first arrangement of a first plurality of areas intended to include a plurality of visual data and the second shaping model comprises a second arrangement of a second plurality of areas intended comprising a plurality of visual data, the second arrangement being different from the first arrangement and the second plurality of visual data being at least in part included in the first plurality of visual data.

According to one example, the step of identifying the first shaping model comprises at least one step of receiving an identifier of said first shaping model, this identifier being associated with a shaping model contained in the first formatting model database, preferably this identifier being transmitted by the source stream.

According to one example, the step of identifying the first shaping model comprises at least one step of determining by an artificial intelligence the first shaping model by comparison with at least one shaping model of the first base formatting model data. According to one example, the composition step further comprises formatting at least part of the first area of interest so as to generate the reformatted first area.

According to one example, the formatting of at least part of the first area of interest comprises a step of cutting out at least part of the first area of interest.

According to one example, the composition step comprises at least one step of extracting at least part of the first area of interest so as to generate the first reformatted area.

According to one example, the composition step comprises at least one step of detecting at least one element of interest from the first area of interest.

According to one example, the composition step comprises a step of determining a center of the first area of interest and a step of centering the first reformatted area according to said center of the first area of interest.

According to one example, said center of the first zone of interest is taken from at least: a geometric center according to the geometry of the first zone of interest, a colorimetric center, a geometric center according to the morphology of an element detected in the first zone of interest, a center according to a detection of elements in salience in the first zone of interest, a center according to a detection of a displacement of the first zone of interest through several images in succession.

According to one example, the first display format is shaped so that the first plurality of images are displayed on a display device having a main spatial extent extending in a horizontal direction, and the second display format is shaped so that the second plurality of images are displayed on a display device having a main spatial extent extending in a vertical direction.

According to one example, the first display format has a length dimension L1 and a height dimension H1, and the second display format has a length dimension L2 and a height dimension H2, and L1 is preferably different. greater than L2, and H1 is different, preferably less, than H2.

In one example, the first display format is a 16: 9 format and the second display format is a 9:16 format.

According to one example, steps a to e are performed by said device for processing computer data in real time, preferably the source stream being broadcast live, and preferably intended to be displayed live by a display device.

According to one example, the source stream is received continuously by the computer data processing device, and steps a to e are performed by said computer data processing device in real time, that is to say as and when as the source stream is received. Preferably the source stream is received “live”. Preferably, the transposed stream is broadcast live.

According to one example, the visual data comprises audiovisual data. According to one example, the method comprises, before the step of identifying the first formatting model, at least one step of receiving said source stream by the computer data processing device.

According to one example, the method comprises, after the step of composing at least one reformatted image, a step of broadcasting said transposed stream to at least one of: a communication network, a display device .

According to one example, steps a to d are performed on each of the images of the first plurality of images or at least on at least X images per second of the first plurality of images of the source stream, with X> 24.

According to one example, steps b and c are inverted.

According to one example, a plurality of areas of interest are identified in step b.

According to one example, several areas of interest of the plurality of areas of interest have dimensions of spatial extension that are different from each other.

The present invention relates to a clever method for transposing a stream of visual data conformed in a first display format into a transposed flow of visual data conformed in a second display format, different from the first.

The first display format is shaped to allow optimum display on first display devices having, for example, an orientation and a conformation in which the height of the display surface is less than the length of the display surface, for example a 16: 9 conformation common today.

The second display format is itself shaped to allow optimum display on display devices having for example an orientation and a conformation in which the height of the display surface is greater than the length of the display surface, for example a 9:16 conformation common today in smartphones when held vertically.

Advantageously, the second display format is itself shaped to allow optimum display on second display devices having for example an orientation and a conformation in which the ratio between the height of the display surface and the width of the surface display is differ from the ratio between the height of the display surface and the width of the display surface of the first display devices.

Also, it is common today that video-type multimedia content is formatted to be displayed on screens in 16: 9 and that for questions of convenience or viewing locations, the user decides to hold his smartphone according to a vertical orientation, so that the content displayed there, i.e. a video conformed to 16: 9, retains its display aspect ratio thus reducing the display surface of the video on the phone of the 'user.

More generally, it is common for the content of a visual data stream to be shaped according to any first display format and for this format not to be the best suited to the display device of said stream. The present invention makes it possible to transpose a source stream of visual data from any display format to a display format optimized for display on a display device other than the first. The expression “optimized display format” is understood to mean a display format allowing the use of at least 60, preferably 80 and advantageously 90% of the display surface of a display device in question. For example, a 16: 9 display format on a 9:16 screen only covers 30% of the surface thereof. The present invention tends to significantly increase this coverage by transposing a video stream conformed to a 16: format: 9 in a video stream conformed to a 9:16 format for example.

The problem, presented above, is at least partly solved by the present invention which allows a decomposition then a recomposition of the source video stream so that most of the visual information can be distributed over the display surface of the device. telephone for example, and this regardless of the orientation chosen by the user.

The present invention also finds application in the transposition of visual data streams displayed on advertising display devices having a screen whose format is different from that of the visual data of the source stream.

In a particularly clever manner, the present invention makes it possible to transpose a video formatted according to a first display format into a video formatted according to a second display format. The present invention thus makes it possible to adapt visual content to the display device on which it is broadcast.

According to a preferred embodiment, the present invention makes it possible to transpose, in an automated manner and in real time, a video, formatted according to a first display format, into a video, formatted according to a second display format, different from the first. display format.

A classic example is news broadcasts designed to be displayed in a television format, where many users use a mobile phone arranged vertically to play this content. The present invention, by recomposing the visual content allows the user a better comfort in viewing the video.

As discussed below, according to one embodiment, the source stream may include a plurality of visual data streams, referred to as input streams. For example, the source stream can include between 1 and N input streams. As also explained below, areas of interest can be extracted from one or more of these input streams. For example, an area of interest including the display of a clock or a scrolling banner can be extracted from a different input stream than that used to extract a person's face, for example. The present invention thus makes it possible to use a plurality of input streams to compose a reformatted image and thus produce said transposed stream. For example, the different input streams may correspond to some of the visual data streams from different cameras.

We will now describe the invention according to an embodiment illustrated through Figures 1 to 8. Figure 1 shows a system in which a computer server 300, comprising a computer program product, preferably recorded on a non-transient medium, comprises instructions, which when performed by at least one of a processor and a computer, cause at least one of the processor and the computer to execute the transposition method 100 according to the present invention. The computer server 300 is configured to receive 310 from a transmission station 200 of a source stream. This source stream comprises visual data conformed according to a first display format 30. It will be noted that in the present invention visual data can also contain audio-visual data. This source stream is intended to be displayed by a display device 400, for example of the television type. The computer server 300 then receives this source stream and transposes it into a transposed stream of visual data, which can also include audiovisual data. The visual data of the transposed stream are shaped according to a second display format 40. This second display format 40 is different from the first display format 30.

The computer server 300 is configured to transmit 320 this transposed stream to a display device 500, for example a smart phone.

According to one embodiment, the computer server 300 is located at a distance, preferably more than 1 kilometer, from the place of realization, from the design of the source stream.

According to one embodiment, the display device 500 is a mobile device, such as for example a smart phone.

According to another embodiment, the display device 500 is a fixed device, such as for example an advertising or information display screen in a public place, and of which for example the main extension is vertical.

FIG. 2 represents, according to one embodiment, the transposition method 100.

Advantageously, the method 100 according to an embodiment of the present invention ensures the transposition of a source stream of visual data conformed according to a first display format 30 into a transposed flow of visual data conformed according to a second format. display 40.

Preferably, the first display format 30 is intended to be displayed by a device 400 whose display surface is arranged in a first spatial orientation, in particular in a main direction of horizontal extension.

Preferably, the second display format 40 is intended to be displayed by a device 500 whose display surface is arranged in a second spatial orientation, in particular in a main direction of vertical extension.

Note that the second display format 40 is advantageously different from the first display format 30.

As indicated above, the source stream comprises a first plurality of images. Each image of the first plurality of images is conformed according to the first display format 30. Advantageously, each image of the first plurality of images is conformed according to a first shaping model 10. This is called a shaping model. particular arrangement of elements in an image, we will present examples later.

In a particularly advantageous manner, the method 100, according to one embodiment, comprises the following steps which are carried out entirely by a computer data processing device, for example by the computer server 300 mentioned above: a. Identification 110 of the first shaping model 10; b. Detection 120 of at least a first area of interest 1a in a first image taken from among the first plurality of images; vs. Selection 130 of a second shaping model 20 as a function of the first shaping model 10 and / or of said first area of interest 1a and / or of the second display format 40, the second shaping model shape 20 being advantageously different from the first shaping model 10; d. Composition 140 of an image reformatted according to the second display format 40 and according to the second formatting model 20 from at least part of the first area of interest 1a and at least part of the first image ; Then, e. Repeating steps 110 to 140 150 for a plurality of images of the first plurality of images so as to form said transposed stream of visual data comprising a second plurality of images shaped according to the second display format 40.

According to one embodiment, the method comprises, after the composition step 140, a step of displaying the transposed stream, that is to say of the second plurality of images shaped according to the second display format 40. , on the display device 500.

It will be noted that in the case where the source stream comprises a plurality of input streams, the following steps are performed on at least 2, preferably on each input stream: the identification 110 of a first setting model form 10, the detection 120 of at least a first zone of interest 1a in a first image taken from among a plurality of images. According to this embodiment, the selection 130 of a second shaping model 20 is carried out as a function of the first shaping model 10 specific to each input stream considered and / or of said first zone of interest 1a. specific to each stream considered and / or of the second display format 40.

In the present invention, an area is defined as part of an image, part of visual data, so that an area includes at least part of an image, at least part of visual data. For example, a zone of interest 1a in an image comprises a portion of said image, and a reformatted zone 1b from at least part of a zone of interest 1a comprises at least a portion of the image that includes said area of interest 1a. It will be noted that according to a preferred embodiment, the reformatted zone 1b is an extraction of a zone of the zone of interest 1a, for example by extracting the protruding elements of 1a, such as, for example, a face, or of movement. . The area reformatted 1b is then defined as part of the area of interest 1a defined to maximize the transcription of these salient elements.

Advantageously, the step 110 of identifying the first shaping model 10 is performed by comparing the first shaping model 10 with a first database of shaping models. This comparison can be made in several ways. For example, the source stream may include an identifier specific to the formatting model 10 used in the image considered. According to another preferred method, the present invention uses the resources of an artificial intelligence, previously trained in the classification of shaping models, so as to identify the first shaping model 10 among the plurality of shaping models. form the first database.

Advantageously, the step 120 of detecting a first area of interest 1a can be performed, by the computer data processing device, in various ways and using various techniques. It will be noted that the detection of a first area of interest in a first image taken from among the first plurality of images can also be understood to mean extracting said first area of interest from the first image. Also, the detection step 120 may include extracting the first area of interest (1a) from the first image.

Advantageously, and according to one embodiment, the method comprises, after the step of detecting the first area of interest 1a in the first image, a step of extracting at least part of the first area of interest. from an image of the source stream.

It will be noted that an area of interest 1a, 2a, 3a, 4a, 5a, 6a can include an element of interest such as a face for example. According to one embodiment, the step of detecting an area of interest 1a, 2a, 3a, 4a, 5a, 6a can comprise the detection of an element of interest. For example, an item of interest could be a person, face, object, vehicle, balloon, map, title, clock, logo, etc.

According to one embodiment, an area of interest 1a, 2a, 3a, 4a, 5a, 6a can be detected through the first shaping model 10. Indeed, it may be that this first shaping model 10 includes an arrangement of visual data such that areas of interest 1a, 2a, 3a, 4a, 5a, 6a are known once the first shaping pattern 10 is identified. For example, the first database may include information indicating that such or such location of the shaping model is an area of interest 1a, 2a, 3a, 4a, 5a, 6a. According to another example, the source stream can include data indicating a positioning of the areas of interest 1a, 2a, 3a, 4a, 5a, 6a.

According to an example not illustrated, at least part of the areas of interest 1a, 2a, 3a, 4a, 5a, 6a can come from a first input flow and another part from a second input flow, different of the first, the first input stream and the second input stream being understood by the source stream.

In another example, a zone of interest 1a, 2a, 3a, 4a, 5a, 6a can also be detected by an analysis of contours 11, by looking for contours 11, for example closed, which therefore delimit an area of the image, which area may be an area of interest 1a, 2a, 3a, 4a, 5a, 6a.

According to another example, artificial intelligence, of neural network type for example, can be used. Preferably, this artificial intelligence is previously and / or regularly trained to identify areas of interest, for example particular elements on an image such as for example objects, people, faces, a weather map, etc. areas of the image where these elements are located can therefore be considered as areas of interest 1a, 2a, 3a, 4a, 5a, 6a. Advantageously, these areas of interest 1a, 2a, 3a, 4a, 5a, 6a can come from various input streams.

According to another example, the type of scene can also be identified by artificial intelligence and therefore predetermined areas can be considered as areas of interest 1a, 2a, 3a, 4a, 5a, 6a.

As we will see later, for each identified area, artificial intelligence can calculate image by image what is the optimized centering point of the considered area in order to optimize the quality of the area of interest when the area will be transposed. according to the second display format 40. This makes it possible, for example, to center on faces, or on protruding and / or moving objects.

By a protruding object is meant an object which is easily noticed in an image, which emerges from this image clearly, which may present a different contrast, for example compared to the rest of the image, or a different colorimetry. For example, a face can be an object that protrudes from the background of an image.

Particularly advantageously, the second shaping model 20 can be selected from a second shaping model database.

According to one embodiment, this selection takes place according to pre-established editorial rules. For example, these rules assign to the first shaping model 10, and preferably to all the information of the areas of interest, a second shaping model 20.

Advantageously, this second database can be different from the first database of shaping models. Similar to the first shaping model 10, the second shaping model includes rules for positioning the area or areas of interest 1a, 2a, 3a, 4a, 5a, 6a. We will illustrate this later.

In a particularly advantageous manner, the step 140 of composing the reformatted image according to the second display format 40 and according to the second formatting model 20 can comprise at least one of the following steps, for example: i. A step of arranging, as a function of said second shaping model 20, of a first reformatted zone 1b corresponding to at least part of the first zone of interest 1a; ii. A step of arranging, as a function of said second formatting model 20, of a second reformatted zone 2b corresponding to at least part of the first image; iii. A step of cutting out at least part of the first zone of interest 1a so as to generate the first reformatted zone 1b; iv. A cutting step of at least part of the first image so as to generate the second reformatted area 2b; v. A step of framing at least part of the first zone of interest 1a so as to generate the first reformatted zone 1b; vi. A step of framing at least part of the first image so as to generate the second reformatted area 2b; vii. A step of determining a center 1c of the first zone of interest 1b and a step of framing according to this center 1c to generate the first reformatted zone 1b; viii. A step of determining a center of at least part of the image and a step of framing along this center to generate the second reformatted zone 2b.

According to one embodiment, the framing step makes it possible to define a reformatted zone comprising one or more elements of interest. According to this embodiment, and for example without limitation, the framing step makes it possible to cut certain parts such as hands, parts of the body or other elements not necessary in relation to the element of interest considered, here a face. for example not limiting.

According to one embodiment, the reformatted second zone 2b can be obtained from at least part of a second zone of interest 2a. Indeed, said part at least of the first image can be a second area of interest 2b.

According to one embodiment, the center of an image or of an area may be a geometric center depending on the shape of the area or the image, a colorimetric center depending on a color surface in the area or the image. image, a morphological center according to a shape present in the area or the image.

Preferably, it will be noted that during a step of centering a zone, this step may correspond to centering on a moving element so as to take account of the movement of an element of interest and thus to follow it by. one image to another.

Advantageously, by repeating the steps described above, the present invention allows the generation of a transposed stream of visual data comprising a second plurality of images conformed according to the second display format 40. Preferably, each image comprises at least two zones. reformatted 1 b and 2b arranged according to the second formatting model 20. According to one embodiment, the second reformatted zone can correspond to the whole of the considered image of the source stream. And according to another example, the first reformatted zone 1b can correspond to the whole of the considered image of the source stream, in this case the second reformatted zone 2b can correspond to predetermined visual content such as advertising banners for example or a personalized background depending on the source stream considered. According to one embodiment, the first zone of interest 1a can correspond to the whole of the image considered of the source stream. The second part of the reformatted image therefore comes either from the source stream or from another stream or even from predetermined visual elements.

According to one embodiment, during the composition of the reformatted image, the reformatted zones 1b and 2b can be at least partially in contact at the level of at least one of their sides, one of their contours.

According to another embodiment, when composing the reformatted image, the reformatted areas 1b and 2b can be at least partly superimposed with one another.

FIG. 3 illustrates in particular the step 120 of detecting areas of interest (6 in number in this figure) and the step 140 of composing a reformatted image. In particular, this reformatted image comprises at least part of the areas of interest detected during step 120.

It will be noted that according to one embodiment, the number of areas of interest resulting from step 120 may be different from the number of reformatted areas forming at least part of the reformatted image composed in composition step 140.

In this figure, an image of the first plurality of images of the visual data source stream is shown. This image is conformed according to a first display format 30. This first display format 30 can be defined by its dimension of extension in height H1 and by its dimension of extension in length L1. This image is conformed according to a first shaping model 10. For example, this model comprises an arrangement in which the image is divided into two parts, an upper part and a lower part. The upper part and the lower part are each divided into 3 zones. For example, in the upper part one of the 3 zones may be larger 2a than the other two 1a and 3a. And still according to this embodiment illustrated in FIG. 3, the three zones 4a, 5a and 6a of the lower part can be of identical dimensions to each other.

It will be noted that advantageously, each zone is delimited by at least one contour 11.

During the step 120 of detecting areas of interest, and according to this example, the invention allows the detection of six areas of interest 1a, 2a, 3a, 4a, 5a, 6a.

Then, during the step of composing the reformatted image 140, at least part of each of these six areas of interest 1a, 2a, 3a, 4a, 5a, 6a is processed to obtain six reformatted areas 1b, 2b , 3b, 4b, 5b, 6b. These six reformatted areas 1b, 2b, 3b, 4b, 5b, 6b are then arranged according to a second shaping model 20. These six reformatted areas 1b, 2b, 3b, 4b, 5b, 6b then form a reformatted image and conformed according to a second display format 40.

It will be noted that advantageously, each area of interest 1a, 2a, 3a, 4a, 5a, 6a can be classified automatically, and advantageously via artificial intelligence. Once classified, an area of interest, or at least the reformatted area 1b, 2b, 3b, 4b, 5b, 6b from this area of interest 1a, 2a, 3a, 4a, 5a, 6a, is positioned according to the second formatting model 20 according to its class, and preferably according to predefined editorial choices . For example, the second shaping model 20 can provide that the faces are arranged in particular areas of the image and that the decor or other elements of the scene are arranged in other areas. In FIG. 3, the reformatted zones 1b, 3b, 4b and 6b can correspond to faces and the zones 2b and 5b can correspond to shots of a larger scene, for example.

In Figure 4, the principle is the same as that presented above with reference to Figure 3. Areas of interest 1a, 2a, 3a and 4a are detected, then a reformatted image is composed.

According to one embodiment, the areas of interest 3a and 4a can also include time stamps, titles or subtitles embedded in the image, for example. Therefore, the present invention is designed to allow their identification and treatment as an area of interest and their arrangement in the reformatted image, in the event that the second formatting model 20 provides for it.

It will be noted here that the first shaping model 10 can change over time. This may or may not lead to modifications of the second shaping model 20. Indeed, the present invention has the advantage of being able to carry out this transposition method 100 in real time, that is to say from a source stream corresponding to a live broadcast of live generated content.

In addition, the present invention also makes it possible to apply this transposition method 100 to a content recorded on a data medium for example so as to broadcast according to the second display format 40 a content originally created according to the first format. display 30. Let us take here the example of video on demand, and the case of a film, the present invention allows a transposition of the film from a first display format 30, generally of 16: 9, into a second display format 40, for example 9:16 in the case where the user would watch the film on his mobile telephone while holding it vertically for example.

Let us also take the example of video on demand here, and the case of a sports match, the present invention allows a transposition of the video of the match from a first display format 30, generally of 16: 9, into a second display format 40, for example 9:16 in the event that the user watches the match on his mobile phone while holding it vertically, for example. The present invention makes it possible to highlight certain areas of interest in the images of the match, such as, for example, the position of a player, the score or even a particular action.

Advantageously, the present invention applies both to a live source stream and to a source stream comprising or originating from a computer file.

According to one embodiment, the source stream can comprise or be a computer file.

FIG. 5 illustrates another first shaping model 10 as well as another second shaping model 20. According to one embodiment, for each first shaping model 10 corresponds one or more second shaping models 20.

FIG. 6 illustrates an embodiment in which an image of the source stream comprises a single area of interest 1a, for example, without limitation, a face, or a weather map. In this case, and preferably, the method 100 detects this first zone of interest 1a and then composes the reformatted image by placing the first reformatted zone 1b, originating from part of the first zone of interest 1a, and by arranging the second reformatted zone 2b corresponding for example to part of the image of the source stream, or even to the entire image of the source stream. The method 100 thus makes it possible to highlight the area of interest 1a while keeping the entire scene displayed. This embodiment may be advantageous for scenes such as a football match, for example, or for the presentation of an element on the screen such as a weather map.

FIG. 7 illustrates an embodiment in which the source stream comprises a first stream called clean stream and a second stream called dressed stream.

Preferably, the clean stream comprises visual data comprising so-called raw images 50 conformed according to the first display mode 30 and according to the first shaping model 10.

Preferably, the dressed stream includes the same images as the clean stream, shaped in the same manner, and further comprises one or more trim elements 61 configured to dress, decorate, at least one image of the clean stream. In this way, the dressed stream comprises so-called dressed images 60. These dressing elements 61 can be a title, a subtitle, an insert, a scrolling banner, an advertisement for an upcoming program, etc. These elements cover 61 include various graphic elements generally superimposed on the raw images 50 of the clean stream. The wrapped images 60 include the raw images 50 and at least one wrapping element 61.

According to one embodiment, as mentioned earlier, the source stream can comprise from 1 to N input streams. Each input stream includes visual data. This set of visual data from the various input streams is advantageously used to compose the transposed stream. One or more of these input streams can be dressed streams. For example, a first input flow can correspond to the clean flow, a second input flow can correspond to the dressed flow, and a third input flow can include the skin elements 61, while other flows d The input can include different camera angles of the same scene. In this case, and preferably by means of previously trained artificial intelligence, and / or on the basis of pre-established rules, the transposed flow is produced from at least part of some of these flows. The transposed stream advantageously comprises the images reformatted from compositions of reformatted areas originating from at least part of these various input streams.

According to this embodiment, the method 100 allows the detection of an area of interest 1a, 2a, 3a in a raw image 50 and / or in a dressed image 60. Then, the reformatted area 1b, 2b or 3b can be obtained from the raw image 50 or from the dressed image 60 depending on the circumstances, such as for example depending on the part to be extracted from the area of interest 1a, 2a. or 3a to form reformatted zone 1b, 2b or 3b. Indeed, if for example, the reformatted zone 1b, 2b or 3b is contained in the dressed image 60, that is to say that the covering element or elements 61 do not partially cover the zone intended for form the reformatted zone 1b, 2b or 3b, then the dressed flow may be sufficient to obtain the desired reformatted zone 1b, 2b, 3b. On the other hand, if the zone intended to form the reformatted zone 1b, 2b or 3b is covered in part at least by a covering element 61 in the dressed image 60, the method 100 can use the corresponding raw image 50 of the stream clean to generate the reformatted zone 1b, 2b, 3b, so as not to be hampered by the covering element 61 in question. It will be noted that generally, the dressed stream corresponds to the stream displayed by a display device 400 having a horizontal main extension dimension, such as a television for example, advantageously in a 16: 9 format.

According to one embodiment, the method 100 uses artificial intelligence to virtually remove the skin elements 61 from a wrapped image 60 and thus partially reconstruct at least one artificial raw image 50 from a wrapped image 60. This situation can be. considered for example when a clean flow is not available.

According to a preferred embodiment, the areas of interest are defined so that they do not overlap with the covering elements 61; that is to say so that the areas of interest which do not fully include at least one covering element 61, do not include a part or a portion of a covering element 61. This situation can be considered for example when a clean flow is not available.

Figure 8 illustrates two steps 141 and 142 of obtaining a reformatted area 1b from an area of interest 1a. For example, once the area of interest 1a is detected, a center 1c is defined there 141, this center 1c has been previously explained. Then, a cropping step 142 can be performed so as to form the reformatted area 1b from the cropping of the area of interest 1a.

According to one embodiment, the present invention ensures centering 141, preferably image by image, of each reformatted zone so as to be able to keep in the center of the reformatted zone the element of interest identified in the zone of interest.

According to one embodiment, the present invention provides a cropping 142 of the area of interest to form the reformatted area most suited to both the element of interest of the area of interest and to the second layout model. form 20. This ensures that the elements of interest are framed.

The present invention thus allows a transposition of a source stream intended to be displayed according to a first display format into a transposed stream intended to be displayed according to a second display format different from the first.

The present invention provides, preferably image by image, an identification of a first display format, also called an input display format, the identification of at least one area of interest comprising for example an element d. 'interest, the selection of a second formatting model, also called output display format, then the composition of an image reformatted according to this second formatting model and according to the second display format. This reformatted image comprises at least two parts, a first part corresponding to the first reformatted zone and a second part corresponding to at least a part of the considered image of the source stream. This part of the considered image of the source stream can be a second area of interest. According to one example, the second part of the reformatted image can correspond to the whole of the considered image of the source stream.

Finally, it will be noted that, according to a particular embodiment, when the source stream includes a mention of the conservation of the first display format, each image of the first plurality of images comprising this mention is arranged according to a second layout format. form in which the images of the source stream considered are arranged while retaining their formatting format and the rest of the second formatting model is filled with additional visual elements such as for example advertising elements from another stream or from the source stream itself.

The invention is not limited to the embodiments described above and extends to all the embodiments covered by the claims.

REFERENCES 1a First area of interest

I b First reformatted zone

1c Center of the first area of interest 2a Second area of interest

2b Second reformatted zone

3a Third area of interest

3b Third zone reformatted

4a Fourth area of interest

4b Fourth reformatted zone

5a Fifth area of interest

5b Fifth reformatted zone

6a Sixth area of interest

6b Sixth reformatted zone

10 First fitness model

I I Contours

20 Second shaping model

30 First display format

40 Second display format

50 Raw Image

60 Dressed Image

61 Trim element

100 Transposition process

110 Identification of the first formatting model 120 Detection of at least a first area of interest 130 Selection of a second formatting model

140 Composition of at least one reformatted image

141 Centering of the first area of interest

142 Framing of the first area of interest

150 Repeat steps 110, 120, 130 and 140 200 System for transmitting the source stream 210 Transmitting the source stream to a television set 300 Computer server

310 Receipt of the source stream by the computer server from the transmission system

320 Transmission of the transposed stream

400 Display device according to the first display format 500 Display device according to the second display format H1 Height extension dimension of an image conformed according to the first display format L1 Extension dimension in length of an image conformed according to the first display format

H2 Dimension of extension in height of an image conforming to the second display format

L2 Extension dimension in length of an image conforming to the second display format

Claims

1. A method of transposing (100) at least one source stream of visual data conformed according to a first display format (30) into a transposed flow of visual data conformed according to a second display format (40), the second display format (40) being different from the first display format (30), the source stream comprising a first plurality of images, each image of the first plurality of images being conformed according to the first display format and according to a first shaping model (10), the method (100) comprising at least the following steps performed by at least one computer data processing device: a. Identifying (110) the first shaping model (10) from at least a first shaping model database; b. Detecting (120) at least a first area of interest (1a) in at least a first image taken from the first plurality of images; vs. Selection (130) of at least a second shaping model (20) taken from at least a second shaping model database, said selection (130) of said second shaping model (20) being a function of at least one of: the first shaping model (10), said first area of interest (1a), the second display format (40); d. Composition (140) of at least one image reformatted according to the second display format (40) and according to the second formatting model (20), the composition step (140) comprising at least one disposition step, as a function of said second shaping model (20), of a first reformatted zone (1b) corresponding to at least part of the first zone of interest (1a) and of a second reformatted zone (2b) corresponding to at least part of said first image; summer. Repeating (150) steps a through d for a plurality of images of the first plurality of images of said source stream so as to form said transposed stream of visual data comprising a second plurality of images shaped according to the second display format (40), in which the source stream comprises a stream, called clean stream and another stream, called dressed stream, the clean stream comprising a plurality of images called raw images (50) and the dressed stream comprising a plurality of images said wrapped images (60), the plurality of wrapped images (60) comprising the plurality of raw images (50) and at least one wrapping element (61) of at least one raw image (50).

2. Method (100) according to the preceding claim, wherein the trim element (61) comprises at least one of: a timestamp, a title, a subtitle, an image, a graphic animation.

3. A method (100) according to any one of the preceding two claims, wherein the first area of interest (1a) is extracted from one image of the plurality of raw images (50) corresponding to the first image.

4. A method (100) according to any one of claims 1 or 2, wherein the first area of interest (1a) is extracted from an image of the plurality of dressed images (60) corresponding to the first image.

5. Method according to any one of the preceding claims, in which the source stream comprises a plurality of input streams comprising a plurality of images, at least one input stream is the own stream, at least one other stream of entrance is the dressed flow.

6. The method of the preceding claim, wherein at least one input flow comprises said at least one trim element (61).

7. Method according to any one of the two preceding claims, in which at least one input stream comprises a plurality of images obtained from a first camera filming a scene according to a first viewing angle, and in which at least one other input stream comprises a plurality of images obtained from a second camera filming said scene from a second viewing angle different from the first viewing angle.

8. Method (100) according to any one of the preceding claims, wherein the step of detecting (120) the first area of interest (1a) comprises at least one step of detecting contours (11) in the first. image, an area delimited by at least one contour (11) forming an area of interest.

9. The method (100) according to any one of the preceding claims, wherein the step of detecting (120) the first area of interest (1a) comprises at least one step of identifying a type of scene from among. a plurality of predetermined scene types, each scene type comprising at least one area of interest (1a).

10. The method (100) according to any one of the preceding claims, wherein the step of detecting (120) the first area of interest (1a) comprises at least one step of detecting at least one element of. interest taken from at least: a face, a body, an object, a vehicle, a geographical map, a graphic element such as a logo, a title, a clock.

11. Method (100) according to any one of the preceding claims, in which the first area of interest (1a) comprises at least one of: a face, a body, an object, a vehicle, a geographical map, a weather map.

12. The method (100) according to any one of the preceding claims, wherein the step of detecting the first area of interest (1a) further comprises detecting a second area of interest (2a) in the. first image and in which the second reformatted area (2b) corresponds to at least part of said second area of interest (2a).

13. The method (100) according to any one of the preceding claims, wherein the step of detecting (120) the first area of interest (1a) is performed by an artificial intelligence trained to identify different types of areas of interest. interest and / or elements of interest.

14. A method (100) according to any preceding claim, wherein the first shaping model (10) comprises a first arrangement of a first plurality of areas for comprising a plurality of visual data and wherein the second shaping model (20) includes a second arrangement of a second plurality of areas for comprising a plurality of visual data, the second arrangement being different from the first arrangement and the second plurality of visual data being at least partially included in the first plurality of visual data.

15. The method (100) according to any one of the preceding claims, wherein the step of identifying the first shaping model (10) comprises at least one step of receiving an identifier of said first shaping model. form (10), this identifier being associated with a formatting model contained in the first formatting model database, preferably this identifier being transmitted by the source stream.

16. The method (100) according to any one of claims 1 to 14, wherein the step of identifying the first shaping model (10) comprises at least one step of determining by an artificial intelligence the first shaping model. shaping (10) by comparison with at least one shaping model of the first shaping model database.

17. The method (100) according to any one of the preceding claims, wherein the composing step (140) further comprises formatting at least part of the first area of interest (1a) so as to generate the first reformatted zone (1b).

18. Method (100) according to the preceding claim, in which the formatting of at least part of the first area of interest (1a) comprises a step of cutting out at least part of the first area of interest ( 1a).

19. Method (100) according to any one of the preceding claims, in which the composition step (140) comprises at least one step of extracting at least part of the first zone of interest (1 a). so as to generate the first reformatted zone (1 b).

20. The method (100) according to any one of the preceding claims, wherein the composing step (140) comprises at least one step of detecting at least one element of interest of the first area of interest (1a. ).

21. The method (100) according to any one of the preceding claims, wherein the composing step (140) comprises a step of determining a center (1c) of the first area of interest (1a) and a step centering the first reformatted zone (1 b) along said center (1 c) of the first zone of interest (1a).

22. Method (100) according to the preceding claim, wherein said center (1c) of the first area of interest (1a) is taken from at least: a geometric center according to the geometry of the first area of interest (1a) , a colorimetric center, a geometric center according to the morphology of an element detected in the first zone of interest (1a), a center according to a detection of elements in salience in the first zone of interest (1a), a center according to one detection of a displacement of the first zone of interest (1 a) through several images in succession.

23. A method (100) according to any preceding claim, wherein the first display format (30) is configured so that the first plurality of images are displayed on a display device (400) having a display. main spatial extension extending in a horizontal direction, and wherein the second display format (40) is shaped so that the second plurality of images are displayed on a display device (500) having a main spatial extension extending in a vertical direction.

24. A method (100) according to any preceding claim, wherein the first display format (30) has a length dimension L1 and a height dimension H1, and wherein the second display format (40) ) has a length dimension L2 and a height dimension H2, and in which L1 is greater than L2, and H1 is less than H2.

25. A method (100) according to any preceding claim, wherein the first display format (30) is a 16: 9 format and wherein the second display format (40) is a 9:16 format. .

26. A method (100) according to any preceding claim, wherein the source stream is received continuously by the computer data processing device, and steps a to e are performed by said computer data processing device in real time, the transposed stream preferably being broadcast live.

27. A method (100) according to any preceding claim, wherein the visual data includes audiovisual data.

28. A method (100) according to any one of the preceding claims comprising, before the step of identifying (110) the first shaping model (10), at least one step of receiving said source stream by the processing device. computer data processing.

29. A method (100) according to any one of the preceding claims comprising, after the step of composing (140) at least one reformatted image, a step of broadcasting said transposed stream to at least one of : a communication network, a display device (500).

30. Method (100) according to any one of the preceding claims, in which steps a to d are performed on each of the images of the first plurality of images or at least on at least X images per second of the first plurality. images of the source stream, with X> 24.

31. A method (100) according to any preceding claim, wherein steps b and c are inverted.

32. A method (100) according to any preceding claim, in which a plurality of areas of interest are identified during step b.

33. The method (100) of the preceding claim, wherein a plurality of areas of interest of the plurality of areas of interest have dimensions of spatial extent different from each other.

34. Visual data stream transposed from a source stream of visual data by the method (100) according to any one of the preceding claims, the visual data of the source stream being conformed according to a first display format (30) and the data visuals of the transposed stream being conformed according to a second display format (40), different from the first display format (30), preferably the transposed stream being intended to be viewed on a mobile display device (500).

35. A computer program product, preferably recorded on a non-transient medium, comprising instructions, which when performed by at least one of a processor and a computer, cause the at least one of the processor and the computer, performs the method (100) according to any one of claims 1 to 33.

36. A computer server (300) comprising at least the computer program product according to the preceding claim and being configured to receive at least one source stream of visual data and to transmit at least one transposed stream according to claim 34.

37. System comprising at least one computer server (300) according to the preceding claim and at least one display device (500) configured to display the transposed stream according to claim 34.

38. A method of displaying at least one transposed stream according to claim 34 on a display device (500), the method comprising at least the following steps: a. Reception (310), by a computer server (300) according to claim 37, of at least one source stream of visual data conforming to a first display format (30); b. Transposition of the source stream so as to obtain said transposed stream; vs. Transmission (320) by the computer server (300) of said transposed stream to at least one display device (400).