Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/20—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video object coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

• the invention relates to a method and a device for coding and decoding a scene composed of objects whose textures come from different video sources.
• Multimedia broadcasting systems are generally based on the transmission of video information, either via separate elementary streams, or via a transport stream multiplexing the different elementary streams, or a combination of the two.
• This video information is received by a terminal or receiver made up of a set of elementary decoders simultaneously performing the decoding of each of the elementary streams received or demultiplexed.
• the final image is composed from the decoded information. This is for example the case of the transmission of streams of MPEG 4 coded video data.
• This type of advanced multimedia system attempts to offer great flexibility to the end user by offering them possibilities for composing several flows and interactivity at the terminal level.
• the extra processing is actually quite significant if we consider the complete chain, from. the generation of simple flows to the restitution of a final image. It concerns all the levels of the chain: coding, addition of inter-stream synchronization elements and packetization, multiplexing, demultiplexing, taking into account of inter-stream synchronization elements and depacketization, decoding.
• composition system upon reception, which produces the final image of the scene to be viewed according to the information defined by the content creator.
• a great complexity of management at the system level or at the processing level is therefore generated.
• the first systems therefore require the management of numerous data flows both at the transmission and reception levels. It is not possible to achieve in a simple way, a local composition or "scene" from several videos. Expensive devices such as decoders and complex management of these decoders must be put in place for the exploitation of these streams.
• the number of decoders can be a function of the different types of coding used for the data received corresponding to each of the streams, but also the number of video objects that can compose the scene.
• the processing time of the received signals due to centralized management of the decoders, is not optimized. The management and processing of the images obtained, because of their multitude, are complex.
• the invention aims to overcome the aforementioned drawbacks.
• Its subject is a method of coding a scene made up of objects whose textures are defined from images or parts of images from different video sources, characterized in that it comprises the steps:
• auxiliary data comprising information relating to the composition of the composed image and information relating to the textures of the objects.
• the composite image is obtained by spatial multiplexing of the images or parts of images.
• the video sources from which the images or parts of images composing the same composed image are selected have the same coding standards.
• the composite image may also include a still image not from a video source.
• the dimensioning is a reduction in size obtained by subsampling.
• the composed image is coded according to the MPEG 4 standard and the information relating to the composition of the image are the texture coordinates.
• the invention also relates to a method for decoding a scene composed of objects, coded from a composite video image grouping images or parts of images from different video sources and from auxiliary data which is information of composition of the composite video image and of information relating to the textures of the objects, characterized in that it performs the steps of:
• the method is characterized in that the extraction of the textures is carried out by spatial demultiplexing of the decoded image. •.
• the method is characterized in that a texture is processed by oversampling and spatial interpolation to obtain the texture to be displayed in the final image viewing the scene.
• the invention also relates to a device for coding a scene composed of objects whose textures are defined from images or parts of images from different video sources, characterized in that it comprises:
• a video editing circuit receiving the different video sources for dimensioning and positioning on an image, images or parts of images originating from these video sources, for producing a composite image
• an auxiliary data generation circuit connected to the video editing circuit to supply information relating to the composition of the composed image and information relating to the textures of the objects, a coding circuit for the composed image,
• the invention also relates to a device for decoding a scene composed of objects, coded from a composite video image grouping together images or parts of images from different video sources and from auxiliary data which is information. of composition of the composite video image and of information relating to the textures of the objects, characterized in that it comprises:
• auxiliary data a circuit for decoding the auxiliary data - a processing circuit receiving the auxiliary data and the decoded image for extracting textures from the decoded image from the auxiliary data for composing the image and for applying textures to objects of the scene from the auxiliary data relating to the textures.
• the idea of the invention is to group, on an image, elements or texture elements which are images or parts of images coming from: different video sources and necessary for the construction of the scene to be visualized, so to "transport" this video information on a single image or a limited number of images.
• a spatial composition of these elements is therefore produced and it is the overall composite image obtained which is coded instead of coding separate from each video image from video sources.
• a scene . overall, the construction of which usually requires several video streams can be constructed from a more limited number of video streams and even from a single video stream transmitting the composed image.
• the decoding circuits are simplified and the construction of the scene carried out in a more flexible manner .
• QCIF format an English expression Quarter Common Intermediate Format
• GIF Common Intermediate Format
• the image On reception, the image is not simply presented. It is recomposed using transmitted composition information. This makes it possible to present the user with a less frozen image, potentially including an animation resulting from the composition, and to offer him further interactivity, each recomposed object being able to be active.
• Management at the receiver is simplified, the data to be transmitted can be more compressed due to the grouping of video data on an image, the number of circuits necessary for decoding is reduced. Optimizing the number of streams minimizes the resources required in relation to the content transmitted.
• FIG. 1 a coding device according to the invention
• FIG. 1 represents a coding device according to the invention.
• the circuits at 1 n symbolize the generation of: various video signals, available to the encoder for the coding of a scene to be viewed by the receiver. These signals are transmitted to a composition circuit 2 which has the function of composing an overall image from those corresponding to the signals received. The overall image obtained is called the composite image or mosaic.
• This composition is defined on the basis of information exchanged with an auxiliary data generation circuit 4.
• composition information making it possible to define the composed image and thus to extract, at the receiver, the various elements or sub- images composing this image, for example position and shape information in the image such as the coordinates of the vertices of rectangles if the elements constituting the transmitted image are of rectangular shape or shape descriptors.
• This composition information makes it possible to extract textures and it is thus possible to define a library of textures for the composition of the final scene.
• auxiliary data relate to the image composed by the circuit 2 but also to the final image representing the scene to be viewed at the receiver.
• graphic information for example relating to geometric shapes, appearances, the composition of the scene making it possible to configure a scene represented by the final image.
• This information defines the elements to be associated with graphic objects for the mapping of textures. They also define the possible interactivities making it possible to reconfigure the final image from these interactivities ...
• the composition of the image to be transmitted can be optimized according to the textures necessary for the construction of the final scene.
• the composite image generated by the composition circuit 2 is transmitted to a coding circuit 3 which performs coding of this image.
• Auxiliary data 1 from circuit 4. are i transmitted to a coding circuit 5 which realizes, .coding of these data:
• the outputs of coding circuits 3 and 5 are transmitted to the inputs of a multiplexing circuit 6 which multiplexes the received data, ie video data relating to the composed image and auxiliary data
• the output of the multiplexing circuit is transmitted to the input of a transmission circuit 7 for the transmission of the multiplexed data.
• the composite image is produced from images or parts of images of any shape extracted from video sources but may also contain still images or, in general, any type of representation. Depending on the number of sub-images to be transmitted, one or more composed images can be produced for the same instant, that is to say for a final image of the scene. In the case where the video signals use different standards, these signals can be grouped by standard of the same type for the composition of a composite image.
• a first composition is made from all the elements to be coded according to the MPEG-2 standard, a second composition from all the elements to be coded according to the MPEG-4 standard, another from the elements to be coded according to the standard JPEG or GIF images or other, so that a single stream is emitted per type of coding and / or by media type.
• the image composed may be a regular mosaic consisting for example of rectangles or sub-images of the same size or else an irregular mosaic.
• the auxiliary flow transmits the data corresponding to the composition of the mosaic.
• the composition circuit can perform the composition of the overall image 0 from enclosing rectangles or limitation windows defining the elements.
• a choice of the elements necessary for the final scene is made by the composer.
• These elements are extracted from images available to the composer from different video streams.
• a spatial composition is then produced from the selected elements 5 - ; by "placing" them on a global image constituting a single video.
• V The information about the positioning. these various elements, coordinates, dimensions, etc., are transmitted to the auxiliary data generation circuit which processes them to transmit them, on the stream.
• composition circuit is in the known field; This is for example 0 a professional video editing tool, of the "Adobe premiere" type (Adobe
• - .. - is a registered trademark). Thanks to such a circuit, objects can be extracted ... from video sources, for example by selecting parts of images, the images of these objects can be resized and positioned on a global image. A spatial multiplexing is for example carried out to obtain the composite image.
• the means of constructing a scene, from which a part of the auxiliary data is generated, are also in the known field.
• the MPEG4 standard uses the VRML language (Virtual Reality Modeling Language) or more precisely the binary language BIFS 0 (BInary Format for Scenes) which allows to define the presentation of a scene, to change it, to update it .
• the BIFS description of a scene makes it possible to modify the properties of objects and to define their conditional behavior. It follows a hierarchical structure which is a tree description. 5
• the data necessary for the description of a scene concern, among other things, the construction rules, the animation rules for an object, interactivity rules for another object ... They describe the final scenario. Some or all of this data constitutes the auxiliary data for the construction of the scene.
• FIG. 2 represents a receiver for such a coded data stream.
• the signal received at the input of the receiver 8 is transmitted to a demultiplexer 9 which separates the video stream from the auxiliary data.
• the video stream is transmitted to a video decoding circuit 10 which decodes the overall image as it was composed at the level of the coder.
• the auxiliary data at the output of the demultiplexer 9 are transmitted to a decoding circuit 11 which performs decoding of the auxiliary data.
• a processing circuit 12 processes the video data and the auxiliary data coming respectively from the circuits 10 and 11 to extract the elements, the textures necessary for the
• the recomposition information then extracting only these elements from the composed image: - -
• the elements are extracted; - for example, by spatial demultiplexing.
• the construction information therefore makes it possible to select only a part of the elements constituting the composed image. They also allow the user to "navigate" in the constructed scene in order to view objects of interest.
• the navigation information from the user is for example transmitted to an input of the circuit 12 (not shown in the figure) which modifies the composition of the scene accordingly.
• the textures transported by the composed image may not be used directly in the scene. They can, for example, be memorized by the receiver for use in offset time or for the constitution of a library used for the construction of the scene.
• An application of the invention relates to the transmission of video data in MPEG4 standard corresponding to several programs from a single video stream or more generally the optimization of the number of streams in an MPEG4 configuration, for example for a program guide application. If, in a classic MPEG-4 configuration, it is necessary to transmit as many streams as there are videos that can be viewed at the terminal, the method described makes it possible to send a global image containing several videos and to use texture coordinates to build a new scene upon arrival.
• FIG. 3 represents an example of a composite scene constructed from elements of a composite image.
• the global image 14, also called composite texture is composed of several sub-images or elements or sub-textures 15, 16, 17, 18, 19.
• the image 20, at the bottom of the figure, corresponds to the scene at view.
• the positioning of the objects to construct this scene corresponds to the graphic image 21 which represents the graphic objects. . •
• each of MPEG ⁇ 4 coding and according to the prior art each
• video or still image corresponding to elements. 15 to 19 is transmitted in 1 a video or still image stream.
• the graphic data is transmitted in the graphic stream. • ' . • • ; .
• a global image is composed from the images relating to the different videos or still images to " form the composite image 14 represented at the top of the figure. This global image is coded.
• Auxiliary data relating to the composition of the overall image and defining the geometric shapes are transmitted in parallel allowing the elements to be separated. The texture coordinates at the vertices, when these fields are used, allow these shapes to be textured from the composite image.
• Auxiliary data relating to the construction of the scene and defining the graphic image 21 are transmitted.
• the composite texture image is transmitted over the video stream.
• the elements are coded as video objects and their geometric shapes 22, 23 and texture coordinates at the vertices (in the composite image or the composite texture) are transmitted over the graphic stream.
• the texture coordinates are the composition information of the composed image.
• the stream which is transmitted can be coded to the MPEG-2 standard and in this case, it is possible to exploit the functionalities of the circuits of existing platforms integrating the receivers.
• elements supplementing the main programs can be transmitted on an additional video stream
• MPEG-2 or MPEG-4 This flow can contain several visual elements such as logos, advertising banners, animated or not, which can be combined with one or other of the programs broadcast, at the choice of the broadcaster. These items can also be displayed based on user preferences or profile. An associated interaction can be expected.
• Two decoding circuits are used, one for the program, one for the composite image and the auxiliary data. A spatial multiplexing is then possible of the program being broadcast with additional information coming from the composed image. . : '.
• a single annex video stream can be used for a program package, to complete - several programs or several user profiles.

Applications Claiming Priority (3)

Publications (1)

Family

ID=8866006

Family Applications (1)

Country Status (5)

Families Citing this family (9)

Family Cites Families (13)

• 2001
  • 2001-07-27 FR FR0110086A patent/FR2828054B1/fr not_active Expired - Fee Related
• 2002
  • 2002-07-24 JP JP2003518188A patent/JP2004537931A/ja active Pending
  • 2002-07-24 EP EP02791510A patent/EP1433333A1/de not_active Withdrawn
  • 2002-07-24 US US10/484,891 patent/US20040258148A1/en not_active Abandoned
  • 2002-07-24 WO PCT/FR2002/002640 patent/WO2003013146A1/fr active Application Filing

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events