Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/222—Studio circuitry; Studio devices; Studio equipment
  • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
  • H04N5/265—Mixing
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
  • G06F3/147—Digital output to display device ; Cooperation and interconnection of the display device with other functional units using display panels
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T11/00—2D [Two Dimensional] image generation
  • G06T11/60—Editing figures and text; Combining figures or text
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N5/00—Details of television systems
  • H04N5/04—Synchronising
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N9/00—Details of colour television systems
  • H04N9/64—Circuits for processing colour signals
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2340/00—Aspects of display data processing
  • G09G2340/10—Mixing of images, i.e. displayed pixel being the result of an operation, e.g. adding, on the corresponding input pixels
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2340/00—Aspects of display data processing
  • G09G2340/12—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels
  • G09G2340/125—Overlay of images, i.e. displayed pixel being the result of switching between the corresponding input pixels wherein one of the images is motion video
• • G—PHYSICS
  • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
  • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
  • G09G2340/00—Aspects of display data processing
  • G09G2340/14—Solving problems related to the presentation of information to be displayed
  • G09G2340/145—Solving problems related to the presentation of information to be displayed related to small screens

Definitions

• the present invention relates to the general field of image processing, in particular video sequences on mobile terminals. It relates more particularly to a method of embedding real-time images within a video sequence representing for example characters in motion.
• the invention relates to a method for processing video sequences on mobile terminals, more specifically, a real-time image inlay in the video stream.
• the video stream is read with the embedded images.
• the image is merged with the video stream, frame by frame. For each frame, the image is positioned at the right place. This method involves the fact that the image undergoes in real time a trapezoidal matrix transformation so that it can adapt to the video stream.
• the invention aims firstly at a method of incrustation of an image to be embedded in a video sequence, for a mobile terminal of the tablet type or smartphone,
• step 100 may also occur after the beginning of step 300 of playing the video.
• the method comprises a step 600 of applying a deformation to the image to be embedded, so as to match this image to be embedded in the shape of the keying area.
• the method comprises a step 750 of tracking the movement of an incrustation zone, by identification of the pixel movements, or in real time using the known motion detection algorithms, shapes, or object recognition by learning, either in pre-production.
• step 500 the identification of the keying area is done by touch input by a user on the display interface of the mobile terminal
• step 500 in the case where the key points are not pre-calculated, keying points defining the keying area are calculated in real time by the mobile terminal, by using keying methods. recognition of images by motion detection or object recognition by learning.
• step 500 in the case of a prior determination of the key points, a file containing the coordinates of the key points in the video sequence is associated with said video sequence, so as to be read (at the latest at the same time) by the mobile terminal.
• the method comprises means for reading a table of coordinates, associated with the video sequence, these coordinates representing, for each frame, the positions of the four extreme points of the incrustation zone, that is to say of the image to be embedded in the video.
• step 700 to insert the image to be embedded, when the video is displayed in real time on the mobile terminal, the method implements a function responsible for seeking the transformation of the image to be embedded with respect to the current frame, said function being called at each frame display.
• step 700 to insert the image to be embedded, the image of the video is merged with the image to be embedded by re-calculating a resulting image by merging the images. raw data of the two images, and this one is displayed.
• the method comprises the following steps:
• the method includes a step of synchronizing the mask with the video sequence.
• the sequence video and the mask are synchronized by a process of double video: the original video sequence being played in a first part, intended to be displayed, and a second part, not displayed, of this video sequence containing only the information allowing the color changes and modifying the opacity of the video sequence, the method comprising, during the display of the video sequence on the mobile terminal, a step of applying the opacity and / or color transformations given by the second part on the first part.
• the opacity or mask information is encoded in a color management format, a color channel managing the opacity, and other channels managing the objects.
• FIG. 1 a flowchart of the steps involved in the present process
• Figure 2 an illustration of a frame of a video sequence in the case of application of an opacity on a part of the image.
• the invention implements a display terminal, here but not limited to smartphone type.
• This display terminal is, in the present non-limiting embodiment, supposed to have means for storing image sequences, calculation means, for example of the microprocessor type, adapted to execute a software application previously loaded in memory, image display means, and advantageously data input means by a user of said terminal.
• the invention relates to a method for processing a video sequence on a mobile terminal, in particular of the smartphone type.
• the video sequence of which he As an example, here are characters or objects moving within the display area during the video sequence.
• the purpose of the method is then to embed an image, called the image to be embedded, on a part of an object (for example the face of a character), called the incrustation zone, this image to be embedded according to the movement of the this object's overlay area during the video sequence, so as to create a sense of realism.
• the keying area may typically be the face of a moving character, said character approaching or moving away from the camera, and the face facing or turning during the sequence.
• the overlay area is a shape that includes the portion of the object to be replaced by the image to be embedded.
• the image to be embedded is trapezoidal, rectangular, polygonal or elliptical.
• the shape of the incrustation zone is, in the present nonlimiting example of implementation of the method, of the same type as the image to be embedded: for example, if the image to be embedded has the shape of a polygon, the area of inlay will be a polygon having the same number of edges, while being possibly deformed (different angles and lengths of the different edges). Similarly, if the image to be embedded has the shape of an ellipse, the key area will also be elliptical.
• the method makes it possible to determine a deformation function of the incrustation zone, then to deform the image to be embedded in a similar way.
• this method includes a step of pre-calculating particular points of the video sequence, called points. incrustation (ie coordinates in time and on a predetermined area of the display area) defining the area of incrustation, so as not to require third intervention during the inlay, and be quite sparing in computing resources for use on mobile devices.
• points. incrustation ie coordinates in time and on a predetermined area of the display area
• key points are calculated in real time by the mobile terminal. This is achieved for example by using image recognition methods by motion detection or object recognition by learning.
• the image to be embedded is merged with the video stream, frame by frame.
• the image to be embedded is positioned at the right place, ie at the location of the overlay area, reproducing the shape.
• Positioning at the location of the keying area requires prior identification of a moving area integrated in the video stream, by identifying pixel movements in real time using known motion detection algorithms, shapes, or recognition object by learning, either in pre-production.
• a file containing the coordinates of the key points in the video sequence is associated with said video sequence, so as to be read (at the latest at the same time) by the terminal mobile.
• each video sequence corresponds to an array of coordinates which represent for each frame, the positions of the four extreme points of the incrustation area, that is to say of the image to be placed in the video.
• the process can use two techniques:
• the image to be embedded is displayed at these coordinates, after having been deformed to be fixed on the corresponding coordinates (four points in the case of a trapezoid) . That is to say that, in a particular mode of implementation, not limiting, the shape of the image to be embedded and its position in the image must correspond exactly to the shape and position of the inlay area to this moment of the video sequence.
• the image of the video is merged with the image to be incrusted by recalculating a resulting image by merging the raw data of the two images, and then this one is displayed.
• This second technique saves the resources of the mobile terminal.
• the pants is highlighted (which corresponds to the area to be embedded). Information about these pants can then be displayed in a new window.
• the method comprises a first step 100 of choosing the image to be embedded.
• a second step 200 the image to be embedded is resized by an adjustment of the user.
• step 300 the video sequence is read. Then, in step 300, the video sequence is read. Then a frame of this video sequence is displayed in step 400.
• step 500 determines whether the frame has a keying area (to check if this image is likely to receive an image to be embedded).
• step 400 If this is not the case, the process returns to step 400, otherwise the step
• a trapezoidal deformation is applied to the image to be embedded in such a way that the shape of the image to be imaged corresponds to the shape of the incrustation zone.
• step 700 this image is displayed in step 700, replacing the overlay area. After this last step, the process returns to step 400.
• a step of the method consists in making the video sequence more or less opaque by locations.
• the image to be embedded can be a color mask, it is necessary to be able to synchronize the mask with the video sequence concerned: the display of the mask on the video sequence must be perfectly synchronized.
• the video sequence and the mask are synchronized by a double video process: the original video sequence (without mask) is played in the visible part, but a non-displayed part of this video sequence is composed by the mask.
• the opacity or mask information is encoded in the RGB (or other color management system) format, a color channel that handles the opacity, and the others. channels managing the objects.
• the object to be embedded is a car 210, and we want to change the color of the head of a pedestrian 220 present on the original video sequence.
• the opacity is coded on the B channel (Blue) and the color change on the R channel (Red).
• the video sequence is here decomposed into two parts: a first part 230, here but not limited to the upper part of the image of the transmitted video file, representing the inlaid object (the car) and the original video sequence, a second part 240 , here but not limited to the lower part of the image of the transmitted video file, displaying only the information allowing the color changes and the modification of the opacity of the video sequence.
• the information is thus encoded in a single video file, and the display is responsible for applying the opacity and / or color transformations given by the lower part on the upper part.
• the process then comprises the following additional steps:
• the opacity is managed on the red channel, in the lower field weft less than one pixel having a color in RGB corresponding to a value FF0000 in hexadecimal. So we recover the R value here FF, to apply it on the opacity of the pixel to be displayed on the main frame.
• the Alpha channel (opacity) of the main frame will therefore have value for the corresponding pixel FF.
• the transformation can be a color change. To can change the color of different objects in real time, you must be able to create matching masks.
• Each mask is encoded in RGB on the second part of the frame. This encoding consists of 2 parts: one channel is used to manage the opacity of the mask, another channel to identify the masks.
• the embedded image gives the impression of blending into the context of the video.
• the calculation and display is done on a mobile terminal.
• the method makes it possible to modify the object of the video by a tactile interaction.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Physics & Mathematics (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Human Computer Interaction (AREA)
• General Engineering & Computer Science (AREA)
• Image Processing (AREA)

Applications Claiming Priority (2)

Publications (1)

Family

ID=47469980

Family Applications (1)

Country Status (4)

Families Citing this family (5)

Family Cites Families (19)

• 2011
  • 2011-12-16 FR FR1161847A patent/FR2984668B3/fr not_active Expired - Fee Related
• 2012
  • 2012-12-17 US US14/364,941 patent/US8866970B1/en not_active Expired - Fee Related
  • 2012-12-17 EP EP12808803.6A patent/EP2791778A1/de not_active Withdrawn
  • 2012-12-17 WO PCT/EP2012/075828 patent/WO2013087935A1/fr active Application Filing

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events