FR2940689A1 - Method for navigating user of mobile terminal e.g. portable computer, on application, involves detecting movement of positioning device, and modulating movement data in audio format for transmitting movement data to server - Google Patents

Abstract

The method involves displaying a user interface on a screen (211) of a terminal based on received display data of a remote server (220), and detecting a movement of a positioning device (250) mounted on a user in response to display content to determine navigation data of the user on the application. The movement data, in audio format, is modulated for transmitting the movement data to a server by an audio channel. The positioning device is arranged for communicating with the terminal by a Bluetoothlink. Independent claims are also included for the following: (1) a method for processing navigation data of the user of mobile terminal on an application (2) a mobile device comprising a positioning device (3) an application server comprising an application module (4) a computer program comprising a method for navigating user of a mobile terminal on an application executing on a remote server.

Description

Method for User Navigation of a Mobile Terminal on an Application Running on a Remote Server The domain of the invention is one in which a mobile terminal uses an application to access a service.

More specifically, the invention relates to a method of user navigation of a mobile terminal on an application running on a remote server.

The invention finds application in a field where a mobile terminal uses an application to access a service. In this context, an application may consist, for example, of consulting information such as the weather, stock prices, the balance of a bank account, a photograph album, or broadcasting multimedia content such as videos.

To provide a service to a user on his mobile phone, there are two solutions - to install an application on the user's terminal or to use the terminal web browser to access a remote application.

In the first case, the user must download a file, install it on his handset, and then ask the operating system to launch it. This process implies that the downloaded executable is compatible at the binary level, and at the level of the resources exploited on the handset of the user.

JAVA technology developed by Sun proposes to use universal binary code for all applications and a "virtual machine" is installed on each terminal. This virtual machine, specific to each type of terminal, translates the universal binary code into binary code directly executable by the terminal. In addition to the additional computing load consumed by this translation, this technology does not solve the problem of the disparity of computing power, memory and screen size of the fleet of deployed mobile terminals. 1 JAVA technology is now deployed on the majority of mobile phones. However, to adapt to the evolution of mobile phones, and to meet the need of manufacturers to differentiate their products, JAVA APIs have evolved and multiplied. JAVA versions therefore differ from one terminal to another according to its date of appearance on the market, and the price range in which it is located. It is therefore very complex for an editor to provide a deployable application over a whole heterogeneous fleet of mobile terminals.

For the user, the steps of finding the version of the application compatible with its terminal, downloading, and installation are boring, and constitute a real obstacle to its use.

In the second case, the application runs on a server accessible through an IP network. The web browser of the terminal is used to display the user interface produced by the application in the form of more or less sophisticated web pages. This is used by Google for various applications: messaging, calendar, word processing, spreadsheet etc ...

The transposition of this model to mobile telephony presents several difficulties.

Indeed, web standards evolve rapidly, driven by the personal computer market: evolution of JavaScript, appearance of Flash technology, etc. On the one hand, the computing power, the amount of memory as well as the computing capacities display evolve faster than for mobile terminals, these being highly subject to the compromise weight, autonomy, bulk. On the other hand, personal computers undergo regular software updates, often automatic, that allow them to be up to date with the latest developments in web standards. This is not the case for mobile phones for two main reasons: each phone having a specific operating system or no operating system, all the firmware must be modified, tested and validated, and this, for each handset model. Only the manufacturer can perform this complex operation and therefore expensive, which in the case of low-priced terminals is not justified. The cost of the 3G data bandwidth remains high and highly dependent on the operator as well as the type of subscription chosen by the user, which makes the automatic update models inapplicable today. 2 In addition, web standards are not suitable for small screens. The choice of fonts, layout of the various graphic elements (titles, texts, images ...), left to the terminal give a heterogeneous graphic rendering from one mobile to another according to the resolution of the screen, and the choice of the organization of the display taken by the embedded web browser.

The definition of WAP has been an attempt to adapt web technologies to mobile telephony. Its failure is probably due to the disparity of page rendering on each terminal, as well as the poor display possibilities and the lack of rich media rendering.

The size of mobile phone screens are sometimes small. US Pat. No. 6,288,704 describes a system making it possible to display on a small screen a larger virtual image than this one by detecting the movement of the latter with the aid of images acquired by an integral camera. In the case of a mobile phone, this system has the disadvantage of having to install an application on the phone requiring substantial computing power necessary for motion detection calculations.

The invention in at least one embodiment, aims to overcome these disadvantages of the state of the art and to make improvements.

A first aspect of the invention proposes a method for exchanging data between an application running on a server and a remote terminal, the method comprising the following steps: reception to a server, via a wireless audiovisual link , a request for data by a remote terminal; data generation based on the request by an application running on the server; constitution of audiovisual data from the data generated by the application; and transmission of audiovisual data to the terminal via the audio-visual link.

The data exchange request or initialization can be in the form of a video call, an audio call, a message, and so on. In the context of the invention, by the term audiovisual link is meant a telecommunication channel between two entities 3 for simultaneously transmitting audio and video content in real time. By the term query is meant an initialization indication of data exchange.

Thus the speed of execution of the application is not limited by the computing power and the memory available on the terminal. The application can consume a large amount of server resources without compromising the terminal's autonomy. Resources consumed, include bandwidth, which can be used to communicate with other services In addition, access to the application on the server is simpler and faster. Just make a video call. No additional software should be downloaded or installed. The broadcasting of audiovisual sequences is simpler and more instantaneous.

The data generated by the application can be converted into audiovisual data.

Audiovisual data can be retrieved from a database from the data generated by the application. The server may generate display data to present a user interface on the terminal screen to enable the terminal user to interact with the application.

An image of the user interface can be transmitted to the terminal via the audio-visual link of the H324m protocol.

The invention also proposes a method of interaction of a terminal with an application running on a remote server, transmission of a terminal to a remote server, via a wireless audiovisual link, an information request; receiving at the terminal, via the audio-visual link, the audiovisual display data according to the request, displaying a user interface on the terminal screen according to the audiovisual data received; using the user interface to interact with the application from the terminal through the audiovisual link to receive the desired information. 4 The terminal can generate DTMF signals to be transmitted to the server to interact with the application. The terminal can send interaction data to the application in the form of audio and / or video data.

The invention also proposes an application server comprising an application module for executing an application accessible by a remote terminal: receiving means for receiving, via a wireless audiovisual link, a request for data by a remote terminal; means for generating data based on the request; means for providing the audiovisual data from the data generated by the application; and transmission means for transmitting the audiovisual data to the terminal via the audiovisual link.

The invention further proposes a mobile terminal able to interact with an application running on a remote server, the terminal comprising transmission means for transmitting to a remote server, via a wireless audiovisual link, a information request; reception means for receiving, via the audio-visual link, audiovisual display data according to the request, a screen for displaying a user interface according to the audiovisual data received, means for using the audiovisual link; user interface to interact with the application.

A second aspect of the invention provides a method of user navigation of a mobile terminal on an application running on a remote server, the method comprising the steps of displaying a user interface on a terminal screen based on display data received from the server; sensing the movement of a user-mounted terminal positioning device in response to the display content to determine user navigation data on the application; and modulating the motion data in audio format for transmission to the server via an audio channel.

This gives the user the ability to interact more finely with the application running on the server. In addition, the method does not require the downloading of additional software to be used since the method uses an existing Bluetooth standard profile.

Advantageously, the positioning device can be arranged to communicate with the terminal via a Bluetooth type link.

The display information is received from the server via an audiovisual link. The link may be an audio-visual link according to the H324m protocol. The modulation step may comprise a step of modulation to Bluetooth transcoding to audio H324m.

The invention also proposes a method of processing navigation data of a user of a mobile terminal on an application on a remote server, the method comprising the following steps: reception to the server via a link of telecommunication of the movement data of a device positioning device movable according to the movements of the user; and processing the motion data to determine navigation data of the application based on the motion data.

The motion data can be received via an audio channel.

Advantageously, the server can detect the presence of the positioning device by predefined carrier or sequence detection via the audio channel.

The invention further proposes a mobile device capable of communicating via a wireless link with an application running on a remote server, the device comprising: a terminal handset comprising a screen for displaying an interface user from the application and a keypad; a positioning device displaceable in accordance with the movement of the user, and being able to communicate with the handset via a wireless intermediate; means for generating motion data as a function of the displacement of the positioning device; and a modulator for modulating the motion data into audio format for transmission to the server via an audio channel.

Preferably the handset may include a Bluetooth interface, and the positioning device may include a Bluetooth radio to communicate with the Bluetooth interface.

The positioning device may comprise an accelerometer or a mouse. The invention further provides an application server comprising an application module for executing at least one application; a demodulator for demodulating the received motion data via an audio channel of a mobile terminal; and a motion data processing module for determining navigation data of the application based on the motion data.

The server may be equipped with means for detecting the presence of a positioning device. A third aspect of the invention provides a method of modeling a display from an application on a server, the display being intended for a remote terminal; characterized in that the display is modeled to compose an image based on a display information from the application, by means of superimposed layers, each layer being composed of pixels and being associated with a first set of data corresponding modification, each pixel being characterized by a transparency factor Ttp of said pixel; the first set of modification data indicating characteristics of said layer related to the modifications made by the application, the resulting image being composed of image pixels and being associated with a second set of modification data, the second set of modification data indicating whether the image has been modified due to the application changing the layers, the image pixels being composed of the corresponding superposed pixels of the layers; the method comprising the steps of: modifying the content of the pixels of at least one layer by at least 7 a primitive according to the display information received from the application; modifying the first set of modification data for at least one layer according to the modifications made by the primitive (s); modifying the second set of data based on the first set of data; composition of an image according to the pixels of the layers; and transmitting to an encoder the image to be encoded and encoding information according to the second set of change data

This modeling method makes it possible to simply describe the elements of a user interface and to be able to generate a compressed video stream from the descriptive elements of the model with few calculations. This method also makes it possible to simplify the composition of the images.

The first set of modification data may indicate the motion vectors of the layer, a transparency factor, and / or for each block of pixels of the layer if said block has been modified by the application, and / or if it is completely transparent, completely opaque or neither.

The second set of modification data may indicate a motion vector for each macroblock of pixels of the image for use in a differential encoding. The pixels of the image are grouped into macroblocks. The pixels of a layer are grouped into blocks.

Preferably, each layer Ci can be characterized by a two-dimensional displacement vector Vd and / or a transparent scalar Stc. defining a transparency for the set of said layer Ci. The step of modifying the second set of modification data can comprise, for each macroblock of pixels of the image to be encoded, a search in the data set for the blocks of pixels layers that affect the macroblock to see if a block affecting the marcobloc has been modified 8 The second set of modification data can indicate for each macroblock of pixels if said macroblock has been modified since the previous image.

If several layers used to compose the image have had a simultaneous translation, the vector Vd for the encoding can be chosen according to a predetermined criterion. The invention further proposes a device for modeling a display from an application on a server, the display being intended for a remote terminal; characterized in that the display is modeled to compose an image based on display information from the application, by means of superimposed layers, each layer being composed of pixels and being associated with a first set of data of corresponding modification, each pixel being characterized by a transparency factor Ttp of said pixel; the first set of modification data indicating characteristics of said layer related to the modifications made by the application, the resulting image being composed of image pixels and being associated with a second set of modification data, the second set of modification data indicating whether the image has been modified due to the application changing the layers, the image pixels being composed of the corresponding superposed pixels of the layers; the method comprising the steps of: means for modifying the content of at least one layer by at least one primitive according to the display information received from the application; means for modifying the first set of modification data for at least one layer according to the modifications made by the one or more primitives; means for modifying the second set of data based on the first set of data; means for composing an image according to the content of the layers; and means for transmitting to an encoder the image to be encoded and encoding information according to the second set of change data.

The invention further provides an encoder arranged to receive an image generated by the method as described above and encoding information generated by the method as described above, the encoder being arranged to 'encode the image using encoding information. A fourth aspect of the invention provides a method of uploading audio and / or video data, the method comprising the steps of: inputting video and / or audio data by a mobile terminal; transmission of video and / or audio data to a remote server via a wireless audiovisual link; recording on the server video and / or audio data; transcoding the video and / or audio data to the server to a format compatible with a personal computer; and transmitting the transcoded data from the server to an on-line audiovisual broadcast service.

This method provides a user of a terminal a quick and easy way to put online audio-visual sequences filmed in a mobility situation.

The method may further include a step of generating a display for the terminal for controlling the start and end of the recording and / or giving a name to the sequence.

The method may further include a step of registering the terminal number to create a link between the recorded sequence and the user who created it.

The invention further proposes a server comprising a recording module for recording video and / or audio data received from a remote mobile terminal via a wireless audiovisual link; a conversion module for converting the video and / or audio data to a reproducer-readable format of the PC type audiovisual data; and means for transmitting transcoded data to an online broadcast service.

The server may be equipped with a display generation module for generating a display for the terminal allowing a user to control the start and end of the recording and / or to give a name to the sequence. The server may further include a database for storing the terminal number to create a link between the recorded sequence and the user who created it.

A fifth aspect of the invention proposes a method of user navigation of a mobile terminal on an application running on a remote server, the method comprising the following steps: receiving the motion vectors of the video data, the vectors of motion having been calculated for compression of a video sequence transmitted from a terminal; estimation of the movement of the terminal as a function of the motion vectors; and constitution of the content to be displayed on the terminal according to the estimated movement to enable navigation on a user interface larger than the size of the terminal screen.

This method makes it possible to display on a small screen a virtual image larger than this one or to go from one image to another without any additional heavy calculations in the handset or on the server. A move to the right of the handset may, for example, mean switching to a next image in the case of viewing a photo album. The movement of the handset can be used to move a pointer on the screen (like a mouse on the computer).

The display content can be transmitted from the server to the terminal via an audiovisual link. The audiovisual link can be a link to the H324m protocol.

The invention further proposes a method of user navigation of a mobile terminal on an application running on a remote server, the method comprising the following steps: moving the terminal to navigate the application calculating the motion vectors video data, the motion vectors being related to the movement of the terminal by the user; transmission of the motion vectors to the server as navigation data of the application; and receiving the content to be displayed on the terminal according to the motion vectors to enable navigation over a user interface image larger than the size of the terminal screen. The invention further provides an application server comprising: an application module for executing at least one application; a decoder for receiving motion vectors of the video data, the motion vectors having been calculated for compressing a video sequence transmitted from a terminal; a processing module for estimating the movement of the terminal according to the motion vectors and for constituting a content to be displayed on the terminal according to the estimated motion to allow navigation on a user interface image larger than the size of the terminal screen.

Each aspect of the present invention provides a computer program for carrying out the corresponding method described above. Such a program may be downloadable via a telecommunication network and / or stored in a memory of a processing device and / or stored on a storage medium intended to cooperate with a processing device.

Other features and advantages of the invention will appear on examining the detailed description below, and the attached drawings in which:

- Figure 1 schematically illustrates a telecommunication system according to at least one embodiment of the invention; FIG. 2A schematically represents a server and a terminal according to a first embodiment of the invention; FIG. 2B illustrates the steps of a method of exchanging data between a terminal and a server according to the first embodiment of the invention; FIG. 3A schematically represents a server and a terminal according to a second embodiment of the invention; FIG. 3B illustrates the terminal and the server of FIG. 3A; FIG. 3C illustrates a pointer according to the second embodiment of the invention; FIG. 4 schematically represents a server and a terminal according to a third embodiment of the invention; - Figure 5 schematically shows modules of the server of Figure 4; FIG. 6A diagrammatically represents modeling layers according to the third embodiment of the invention; FIG. 6B schematically represents the macroblocks of an image according to the third embodiment of the invention; FIGS. 7A to 7C illustrate the steps of a method of modeling a display according to the third embodiment of the invention; FIGS. 8A to 8C show schematically a server and a terminal according to a fourth embodiment of the invention; FIGS. 9A and 9B schematically represent a server and a terminal according to a fifth embodiment of the invention; and - Figure 10 schematically shows a conventional encoder.

The modules shown in the various figures are functional units, which may or may not correspond to physically distinguishable units. For example, these modules or some of them may be grouped into a single component, or be functionalities of the same software. On the other hand, some modules may be composed of separate physical entities.

FIG. 1 schematically illustrates a telecommunication system in which a user terminal 10 in a mobility situation has access via a GSM 3G network 1 (abbreviated from Global System of Mobile Communications Third Generation) to a server remote application 20 on which runs at least one application. Of course, the invention is not limited to such a network. Thus, for example, the network may be any type of telecommunication network making it possible to establish an audiovisual link. The mobile terminal 10 may be a mobile phone, a PDA, a laptop and so on.

The application on the server 20 allows the user to access real-time information (such as weather, stock prices ...), personal (such as the balance of a bank account, a photo album personal ...) and multimedia (such as videos). A user interface produced by the application is displayed in the form of graphic elements (titles, texts, images, etc.) on the screen 11 of the terminal 10 and allows the user to interact with the application. The terminal 10 is equipped with a decoder for decoding the video stream sent by the application of the server 13 to display the user interface on the screen 11 of the terminal 10. The terminal 10 can be equipped with a video encoder for transmitting video data to the server.

Referring now to Figure 2A in which is illustrated a first embodiment of the invention. In this example, a bidirectional multimedia link 101 provided by an audiovisual communication available on the 3G network using the h324m protocol is used by a terminal 110 to interact with the application that runs on the remote server 120 at the terminal 110.

H324m is a protocol for setting up and transporting an audiovisual conference over a bidirectional digital telecommunication channel. It is, in fact, a family of protocols united and harmonized. • The H223 protocol defines the data multiplexing / demultiplexing. • The H245 protocol defines the frame format for negotiating the opening of audio and video channels as well as associated codecs. • The H263 protocol defines the video encoding. • The G723.1 and 3GP-AMR protocols define audio encoding.

In this example, the terminal 110 is a mobile phone type video phone 3G. The terminal 110 comprises a screen 111, on which will be displayed a user interface provided by the server 120, keys of the keyboard 112, a video decoder 113 for decoding video data received from the server 120, an audio decoder 114 for decoding audio data received from the server 120, a video encoder 115 for encoding video data, and an audio encoder 116 for encoding audio data, a camera 117 for capturing image sequences, means 118 for generating an audio stream, for example a microphone or a Bluetooth headset and means 119 for example a speaker to reproduce audio data.

The keyboard keys 112 are capable of generating logical DTMF signals within an H324m session.

The server 120 is equipped with an application module 121 or executes the application, a conversion module 122 comprising a software allowing a processor 14 to generate an audio and / or video stream, in real time, from data generated by the application and an encoder 123 for encoding the audio and / or video stream generated by the conversion module 122. The encoder 123 may be an H.264 type video encoder as illustrated in FIG. 10 for example or other type of encoder presented in the EG Richardson-H.264 and MPEG-4 video compression. John Wiley & Sons Ltd., 2003 ISBN 0-470-84837.5.

An example of a typical video encoder is shown in Figure 10. In this encoding system the data to be encoded arrive at the encoder as a sequence of images evenly spaced in time (step PE1). Each image is in YUV format. The YUV standard is a representation of the color of video data. If the image sequence is in another format, for example RGB, then a transformation to the YUV format should be performed.

The images are then cut into macroblocks during a step PE2 block partitioning. Each standard defines the size and format of these macroblocks. In the case of the H263 standard, for example, each image is first divided into 8x8 pixel blocks. The YUV format sets Y for each pixel in the image, but U and V are averaged over 4 pixels. The parameter Y represents the luminance, while U and V represent the chrominance, that is to say the information on the color. The blocks are grouped into macroblocks comprising 4 Y blocks, 1 U block and 1 V block. Thus, a macroblock contains all the information relating to a 16 × 16 pixel area of the image.

In a step DCT (discrete cosine) transformation step D3, for each macroblock, each block constituting this macroblock is transformed using a discrete 2-dimensional cosine transformation.

The PE3 transformation step DCT is followed by a PE4 quantization step.

This step consists of reducing the number of bits used to code each result coefficient of the DCT transformation. The greater the quantification, the greater the loss of quality of the transmitted image. The entropy PE5 PE5 step is a compression step without loss of information. Each standard defines the chosen algorithm. The two most used entropic coding techniques are modified Huffman variable length code methods, and arithmetic coding. Entropy coding is used to compress quantized coefficients, motion vectors, quantization levels, and various information to describe the format and type of image transmitted.

Then, in a transmission step PE6, the video data stream obtained is transmitted within a previously open channel within an H324m link.

An inverse quantization step PE7 is the inverse operation of PE4 quantization. An inverse DCT PE8 transformation step is the reverse operation of the DCT transformation.

In a differential estimation phase PE9, when a macroblock is encoded in differential mode, a difference is made between the current image and the previously sent image, as it will be decoded. This is why inverse quantization and inverse DCT transformation are performed. To reduce the amount of information to be transmitted, a translation vector is applied to the macroblock of the previous image so that it approaches the best of the current image. The search for this vector is called motion vector search. This difference is then transformed, quantified and encoded.

In a flow measurement phase PE10 for the outgoing flow of the encoder to be transmitted in a limited rate communication channel, the size of each encoded picture is evaluated.

The control of the PE11 encoder is the heart of the encoder. This function controls the parameters of the encoder such as the quantization level, the choice between absolute or differential encoding for the image, and for each of the macroblocks. Alternatively in another embodiment the server 120 may be equipped with a modeling module and an encoder as will be described for the third embodiment of the invention.

The application module 121, the conversion module 122 and / or the encoder 123 are for example made by programs executed by one or more processors. Of course the server modules may be separate modules as illustrated in Figure 2A or may be part of one or more integrated modules.

The server 120 further comprises means for managing the incoming flows 124 for receiving and processing audio data transmitted by the terminal 110 and for receiving and processing video data transmitted by the terminal 110. The means 124 for receiving and processing audio data received from the terminal 110 may comprise a voice recognition device for example and / or an audio recording device. The means 124 for processing video data received from the terminal 110 may include an augmented reality device. By augmented reality device is meant a device that makes it possible to superimpose a 3D or 2D virtual model to the perception that we naturally have of reality and this in real time. This device can be applied to visual perception (superposition of virtual images to real images) as well as to proprioceptive perceptions such as tactile or auditory perceptions. The means 124 for processing video data received from the terminal 110 may also include or alternatively a pattern recognition device and / or a video recording device.

The server 120 has access to an application database 132 and a content database 131. The application module 121 can retrieve video and audio files in the content database according to the application data. The server is also equipped with a voice synthesis module 126 for generating audio data to be transmitted to the terminal according to the data received from the application module 121. a video multiplexing module 125 for managing the video data received from the content database 131 or the encoder 123; and an audio multiplexing module for managing the audio data received from the speech synthesis module 126 or the content database 131.

As illustrated in FIG. 2A, the user of the terminal 110 can interact with the application on the application module 121 of the remote server via the DTMF signals generated by the keys of the telephone keypad 112; the audio stream of the upstream channel of the audio input 118; and / or the video stream of the ascending path of the camera 117.

FIG. 2B shows a flowchart of an example of a method for exchanging data between the terminal 110 and the server 120. In a step E101, the user of the terminal 110 makes an audio or video call to initiate access to the the application on the application module 121 of the server 120 to consult information, such as the weather.

In a step El 02 the application generates data for a first display on the screen 111 of the terminal 110 in response to the call from the terminal 110. In the case of a weather consultation, for example, the display generated by the application could be, a map of France to allow the user to choose a region of interest. The data generated by the application module 121 is then transmitted from the application module 121 to the conversion module 122 to convert them into a video stream in a step E103. Alternatively the application can retrieve the audiovisual data from the content database 131, or can generate audio data from the speech synthesis module in response to the call from the terminal 110.

In a step El 04 the video stream is encoded by the encoder 123 and the encoded stream 30 is sent to the terminal 110 by the audiovisual link. The data stored in the database 131 may already be in encoded format. The terminal 110 receives the video stream encoded in a step El 05 and decodes it with the aid of the decoder 112. In a step E106, an image of the map of France is displayed on the screen 111.

In a step El 07 the user of the terminal 110 uses the keyboard keys 113 to select a region of the map displayed on the screen 111 and thus generates data according to the content of the display. The DTMF signals generated by the keyboard keys 112 are then transmitted by the audiovisual link to the server 120 in an El step 08. The signals are received by the server 120 and processed by a DTMF processing module 126 of the server 120 in a step E109 . The data corresponding to the DTMF signals are then transmitted to the application module 121 in a step E110 in a format usable by the application. The application processes the data and the method can continue with the data exchange via the audiovisual line between the terminal 110 and the server 120.

The user can also provide interface data in audio form (by voice for example) or in video form (by transmission of an image or a sequence of recorded images for example).

The advantages of this solution are many:

Interoperability is guaranteed by the initial application of the H324m protocol: the audio-visual communication between the server 120 and the terminal 110. This imposes a scrupulous respect for the H324m protocol as well as audio and video compression standards. The rendering on the screen 111 of the terminal 110 is guaranteed by the fact that a video image is transmitted "to the nearest pixel", and that the size of the image is negotiated at the initialization of the session H324m. -The speed of execution of the application is not limited by the computing power and the available memory on the terminal 110. The application can consume a large amount of resources of the server 120 without compromising the autonomy of the terminal 10 The resources consumed include the bandwidth that can be used to communicate with other services. -Access to the application on the server 120 is simple and fast. Just have a video call. No additional software should be downloaded or installed. -The broadcasting of audiovisual sequences is simple and instantaneous

A second embodiment of the invention proposes to give a user of a mobile terminal having access to an application on a remote server the ability to interact more finely with the application running on the server.

In this example, the terminal 210 comprises a screen 211, on which will be displayed, as in the previous example, a user interface for interacting with an application on a remote server 220, audio decoders 213 and video decoders 214 for decoding video data and / or audio received from the server 220, keys of the keyboard 212, a speaker 219 for reproducing audio data and a Bluetooth interface 215 for communicating with a positioning device or pointer 250 equipped with a Bluetooth radio for application navigation . As in the previous example, the keyboard keys 212 may be able to generate logical DTMF signals within an H324m session. The terminal 210 may be equipped with means for generating an audio stream and means for generating a video stream as in the terminal of the preceding example. The terminal further comprises an audio transcoder 216 for transcoding audio data for the server.

This second embodiment differs from the first in that the terminal 210 is equipped with a navigation pointer 250 to provide the user's movement data. The pointer 250 is equipped with a processor, a Bluetooth radio 252, an electric accumulator, a positioning module 254 and a modulator 255. The positioning module 254 may be of the mouse type, or accelerometer type, for example. The pointer 250 is integrated with a device positioned in the hand or the index of the user as shown in Figure 3C.

The server 220 is equipped with an application module 221, a conversion module 222 comprising software enabling a processor to generate an audio and / or video stream, in real time, from the data generated by the application; an encoder 223 for encoding the video and / or audio stream generated by the conversion module 222 for transmission to the terminal 210; and a demodulator 226 for demodulating data received by an audio channel. As in the previous embodiment, the server has access to an application database 232 and a content database 231 for retrieving the audio and / or video files for transmission to the terminal 210.

3G phones equipped with Bluetooth support the "Headset profile" profile defined in the Bluetooth standard. In normal use, during a video call, a Bluetooth headset is used to delocalize the audio functionality of the terminal to the user's head. In this second embodiment the pointer 250 is seen by the phone 210 as a Bluetooth headset. The data of the movement of the index of the user and thus of the pointer 250 are modulated in the upstream audio channel transmitted by the pointer 250 to the server 220. A robust modulation to the transcoding CSV (Bluetooth) to AMR (H324m) audio can be performed by the audio transcoder 214 for the transmission of these data.

In the server 220, the audio data received via the audio channel is demodulated by the demodulation module 226 to retrieve the motion data from the pointer 250 and transmit it to the current application in the application module 221. , in the form of events. A predefined carrier or sequence detection may allow the server 220 to detect the presence of the pointer 250.

In use mode the display information is generated by the application module 221 and is converted to audio-visual format for transmission by the conversion module 222. The audiovisual data is encoded by the encoder 223 and sent to the terminal Alternatively, the encoded audiovisual data can be retrieved from the content database 231. The audiovisual data is then decoded by the decoder 213 and or 214 and a user interface is displayed on the screen 211. The user moves his index on which is mounted the pointer 250 in response to the content of the display to navigate the application on the server 220 and thus give instructions to the application. The motion data as a function of the displacement of the pointer 250 is modulated by the audio transcoder 216 to the transcoding CSV (Bluetooth) 21 to AMR (H324m) audio and is then transmitted to the server 210 via an audio channel. The demodulator 226 demodulates these motion data. The motion data is then processed by the application module 221 to retrieve the navigation instructions from the user. Thus the movement of the user's index can be used to retrieve the intentions or the instructions of the user vis-à-vis the application.

For example, in the case of viewing an image larger than the screen of the terminal, the movements of the index may indicate the desired translational movement of the displayed image. Another example is to reproduce on the screen a pointer, and to interpret movements of the index to move this pointer on the screen. A specific movement of the pointer 250 can be used to move from one part of an image to another part, from one image to another, from one list to another list, from one screen to another screen , to answer yes or no etc.

Thus the user can browse the user interface to interact more finely with the application, so the pointer 250 allows a more intuitive navigation of the application in the user interface when the upstream audio channel of the terminal 210 to server 220 is not used.

Advantageously one can equip the index of the user of a button operable with the thumb, and can act as a "click" (case of the mouse) ... the pressure information on this button to be added to the data acceleration before 25 modulation.

An advantage of this embodiment is that it does not require the downloading of any specially designed software or driver on the mobile phone to be used since it can use an existing Bluetooth standard profile. As shown in FIG. 4 in a third embodiment of the invention, the server 320 comprises an application module 321, a modeling module 322 and an encoder 323. The modeling module 321 presents a model for enabling the It is sufficient to simply describe the elements of a user interface to be displayed on the terminal 310, and to be able to generate by the encoder 323 a compressed video stream from the descriptive elements of the model with few computations. This model also simplifies the composition of images.

As illustrated in FIG. 5, the modeling module 322 constitutes an interface between the application module 321 defining what must be displayed on the screen 311, and the encoder 323 that uses the information generated by the modeling module 322 to generating the video stream to be sent to the terminal 310. For this purpose the modeling module 322 receives display information from the application module 321 and processes the display information so as to provide encoding information and the image to encode with the encoder. The encoder 323 can thus encode the image to be encoded using the encoding information. The modeling module 322 is designed to use a model that allows the faith to simply describe complex screens, and the encoder 323 to effectively use the model information during encoding.

The third embodiment of the invention proposes to model the display by superimposed layers (C1 ... Ci ... CN) as illustrated in FIG. 6A. Each layer Ci consists of a set of M pixels (P1..Pi .... PM). Each pixel Pi is characterized by a four-dimensional color vector Vc (R, G, B, A) defining the color of the pixel Pi as well as a transparency factor Ftp. N layers Ci are stacked on top of each other.

This model 3221 in the form of layers (C1 ... Ci ... CN) stacked makes it possible to conveniently define the effects useful for the presentation of information from the application on the server 320 to a user of the terminal 310. For example , you can draw on the lowest CN layer, a background image, and draw text on the top layer C1. Thus, it is possible to modify the text on the layer C1 and keep the background defined on the layer CN without the application needing to draw the pixels of the background image on the layer CN with each modification of the text of the layer C1. By way of example, for an icon to appear progressively on the screen 311 of the terminal 310, it is possible to draw the icon on a layer Ci above a background of the layer CN, for example, and to vary its transparency. F.

When it is desired to make a transition between two display images on the screen 311 of the terminal 310 for example, it is possible to place the two images on two layers Ci and Ci + 1, for example, and to vary the transparency Ftp of the pixels. (P, P .. .... PM). relevant layers to give a crossfade effect.

In addition, each layer Ci has a two-dimensional displacement vector Vd, a scalar Stc defining a transparency for the entire layer as well as a table called a modified block table or in English layer altered block determining, for a block Bi pixels (P, .. Pi .... PM). if the block Bi has been modified by the application, if it is totally opaque, completely transparent or neither. A block Bi can consist for example of 8x8 pixels.

Table 1 is an example of a representation of such a table Block # Modification Transparency 9 1 0 0 2 NX 1 NTNOX Table 1 In the table the first column identifies the blocks, the second indicates whether the block has been modified yes (0) or (N) and the third column indicates whether the block is opaque (0), transparent (T) or neither (X).

A frame buffer 3222 represents the image to be encoded. The image to be encoded is composed of pixels grouped in groups of pixels called macroblocks. A macroblock (or macroblock) can correspond to a group of 1616 pixels for example. In the variants, a macroblock may correspond to 16 * 8, 8 * 16 or 8 * 8 for example. A table called macroblock modification table is added, describing, for a macroblock MBm of the image composed by the model 3221 if the macroblock MBm has been modified compared to the previous image. A macroblock MBm is composed of a number of blocks Bmk as illustrated in FIG. 6B. The number of blocks that constitute a macroblock depends on the video encoding standard used.

Table 2 is an example of such a table Macroblock Modification from Motion vector # the previous image 1 0 V, 2 N V2 i O Vi N O Va Table 2

In the table the first column identifies the macroblocks, the second indicates whether the macroblock has been modified yes (0) or (N) since the previous image and the third column indicates the layers modified by movement. In the image, an image pixel consists of a mixture of the superimposed pixels of the layers by a process of mixing the pixels with alpha or pixel alpha blending in English.

An example of the composition of a macroblock is shown in Figure 6B. MBm represents the macroblock m of the frame buffer. For simplicity, only Y blocks of brightness are represented. MBm is made up of four blocks. Bm3 is the third block of the macroblock MBm of the frame buffer. In this example, C; represents the layer i, Ci represents the layer j, V; represents the translation vector of the layer i with respect to o, V; represents the translation vector of the layer j with respect to o, o represents a fixed origin with respect to the frame buffer, Bc ;, represents the block I of the layer i, and Bck represents the block k of the layer j.

Zm3iljk represents the zone of block 3 of the macroblock m affected by the blocks i of the layer I and j of the layer k. Suppose in this example that the layer i is above the layer j, that is to say that an opaque area of the layer i can hide an area of the layer i.

If, for example, the block Bc ;, is marked as opaque and modified then the macroblock MBm will be marked as modified. If, on the other hand, the block Bc ;, is marked as transparent, in motion, and the block Bck is marked as neither opaque nor transparent, unmodified, then the macroblock MBm will not be marked as modified.

For all zones Zm3xxxx that affect the block Bm3, the indications of the tables of modification of the blocks of layers as well as the descriptions of movement and transparency of the layers make it possible to evaluate its contribution to the modification of the buffer of frame. The motion vector of the macroblock MBm will be chosen from the motion vectors of the layers.

An example of the modeling process according to one embodiment of the invention will be described with reference to FIGS. 7a to 7c. In a first step E31 of the method, the module of the application 321 sends a display information to the modeling module 322. The display information can be in the form of the primitives. In a step E32, the modeling module 322 can modify the contents of the layers Ci as a function of display information received using primitives. Primitives allowing for example to copy an image to one or more layers from a file in png or jpeg format, to draw a rectangle, or to add text from a font. By way of example, a select layer primitive makes it possible to select a current layer C; that primitives called later will affect. Alternatively, we can add to each primitive the layer C; to which it applies.

For example, the rectangle primitive will use the following parameters:. x, y: position of the upper left corner. w, h: width and height of the rectangle. c: fill color of the rectangle This primitive will draw on the current layer C;, at the position defined by the coordinates (x, y), a rectangle of width w and height h. The interior of this rectangle will be filled with color c.

In a step E33, layer data (transparency factor, motion vector, and modified block table are modified in accordance with the implementation of the primitive or primitives invoked by the application.) For this purpose, primitives modify the layer Ci to which they apply, and modify the modified block table according to the modifications made by the primitives, for example, the primitive rectangle invoked with the operational parameters x = 0, y = 0, w = 8, h = 8, c = black, modifies the current layer Ci, and indicates in the table of modified blocks that the first block B1 of the layer Ci has been modified.

Moreover, it is possible, at any time and for each layer, to define a translation movement by the displacement vector Vd, as well as a variation of the transparency factor by means of the transparency scalar Stc. For example, we can define a move to or in English move to primitive to which the following parameters are passed:. x, y: coordinates of the final position of the layer. This primitive moves the current Ci layer from its current position to the position defined by the coordinates (x, y) in a time t defined in milliseconds, for example. For each image to be encoded, for each macroblock MBm of the frame buffer representing the image, in a step E34, the blocks Bi of the layers Ci which have a non-empty intersection with this macroblock MBm are explored. Since the layers are stacked, each macroblock MB m of the frame buffer is constituted from, potentially, at least part of the pixels of several blocks Bmk of each layer Ci as illustrated in FIG. 6B. In order to detect whether an MBmi macroblock has undergone modifications, as well as the nature of these modifications, it is therefore necessary to analyze the blocks Bi of the layers Ci that affect this macroblock MBm.

If the corresponding Bi blocks are marked in the macroblock modification table as having been modified by the application, or if they are not totally transparent and if the Ci layer to which they belong is animated by a movement, or if layer Ci undergoes a change of transparency, then the macroblock MBm of the buffer is recalculated. Thus, with this modeling, the step E35 of composition of the display image from the layers Ci is also simplified. In step E35, an image is composed by means of the layers Ci as a function of display information. transmitted by the application module 321.

In the step E34, the data relating to the macroblock Mbi in the table "of the macroblock modifications are modified as a result of the modifications of the block Bi: in the table it is indicated whether the macroblock MB has been modified from the previous image, as well as the vectors of motion corresponding to the layers Ci which induced an alteration of the macroblock MBm by their movement Thus one can constitute encoding information for the encoding step An arbitration is necessary if several layers Ci have a different simultaneous translation. in this case, a criterion will be used to use the most judicious vector Vd.In the encoding step one can, for example, encode the macroblock MBmi with the displacement vector Vd of each layer Ci concerned and choose the one for which the The image composed by the modeling module 322 according to the display information of the application is then transmitted to the modulum. 323 encoding with encoding information.

In step E36 of the method encoding is performed by the encoder 323 from the encoding information transmitted by the modeling module 322.

Referring to FIG. 7b, the traditional internal structure of the video encoder 323 is designed to take advantage of the encoding information transmitted by the modeling module 322.

Part said "asynchronous" is integrated with the application. It is synchronized with the actions of the user. This part of the encoder 323 organizes the data coming from the application, in order to simplify the calculations of the so-called "synchronous" part of the encoding

In a DCT (discrete cosine) transformation step, for each macroblock, each block constituting this macroblock is transformed using a discrete 2-dimensional cosine transformation. The DCT transformation step is followed by a quantization step. This step consists of reducing the number of bits used to code each result coefficient of the DCT transformation. The quantization step is followed by an entropy coding step based on variable length codes. An inverse quantization step is the inverse operation of PE4 quantization. An inverse DCT transformation step is the reverse operation of the DCT transformation. These steps are performed for a differential estimation phase. In the differential estimation phase when a macroblock is encoded in differential mode, a difference between the current image and the previously sent image is made as it will be decoded. To reduce the amount of information to be transmitted, a translation vector is applied to the macroblock of the previous image so that it approaches the best of the current image. The search for this vector is called motion vector search. This difference is then transformed, quantified and encoded.

The video stream composed of the encoded images in accordance with the corresponding encoding information exits the encoder 323 and is sent to the terminal 310 by a link H324m in a step E37.

The transmission of the video stream is via radio waves, it is likely that during the session packets are lost. It must therefore be ensured that each macroblock MBm of the image is regularly encoded in intra mode. This is done in parallel, with a refresh pattern that will be adapted to the available bandwidth, desired responsiveness, and desired packet loss robustness level. This pattern can consist of a macroblock by image, a macroblock by "group of blocks", all the macroblocks of a "group of blocks" out of two etc ...

Synchronization between the application and the encoder should be done carefully. In particular, once the image is encoded, the blocks marked as modified by the application must be marked as unmodified before the application can resume.

The terminal 310 receives the video stream and decodes it using the decoder 312 to display the user interface defined by the video stream on the screen 311.

A fourth embodiment of the invention proposes a method making it possible, in a single step, to record and broadcast a video sequence and / or audio data with a mobile terminal.

In this embodiment of the invention illustrated in FIGS. 8A to 8C, a terminal 410 and a server 420 communicate by means of a 3G GSM type network 401. The terminal 410 includes a video decoder 413, a keypad 412 and a screen 411. The terminal 410 is equipped with a camera module 416 and a microphone 414 for capturing sequences of images and sound accompanying the sequences of images, as well as a video encoder 415 and an audio encoder 416 for encoding respectively a video stream and an audio stream for transmission to the server 420 via an audiovisual link of the H324m protocol 401.

The server 420 comprises an application module 421, a modeling module 422, an encoder for encoding a video stream, and a recording module 425 for recording an audio stream and / or a video stream of the upstream channel of the terminal 410. The registration application module 425 has access to two databases: a first database 431 storing the user profiles and a second database 432 storing the audio-visual content received from a mobile terminal and registered. Of course, the data of the user profiles and the audio-visual content can be stored in the same database 430. The modeling module 422 and the encoder 423 operate in the same way as the modeling module 322 and the encoder 323. of the third embodiment described above.

The recording application 425 can take advantage of functionalities making it possible to display on the screen 411 of the terminal 410 an intuitive interface allowing it to control the beginning and the end of the recording, to give a name to the sequence and to to put it online. The registration application may also take advantage of the incoming caller's number within the application server 420 to create a link between the recorded sequence and the user who created it. The user profile can differentiate, for example, public videos from private videos. The registration application on the server 420 may use real-time transcoding software to a PC-type reproducer format of the audiovisual data, for example, the Flash format etc. It is thus possible to broadcast online the sequence picked up by the terminal in real time with very slight delay.

The user of the terminal 410 records an audiovisual sequence of a scene 430. He consults a user interface generated by the server 420 and displayed on the screen 411 of the terminal 410 to control the beginning and the end of the recording and to give 31 a name to the recorded sequence. Of course the sequence can be sent without being recorded on the terminal. The audiovisual data of the scene recorded by the terminal are then sent via the network 401 to the server 420. On the server 420, the recording module 425 records the audiovisual data and the identification number of the terminal 410 and thus creates a link between the data. audiovisual and the user. The sequence is transcoded in real time to a Flash format, for example, and then sent by an internet network to an online broadcast service 440. The recorded sequence of the scene 430 can be broadcast and thus viewed from the terminal 410 and other terminals such as a personal computer 450.

Thus the system allows a user of a mobile terminal to film and put online very quickly unexpected sequences. It also allows viewing these sequences from a Visio 3G mobile phone with the benefits of interoperability and access time.

In a fifth embodiment of the invention illustrated in FIGS. 9A to 9C, a terminal 510 and a server 520 communicate by means of a GSM GSM network 501 with a link H324m. The terminal 510 is equipped with a video encoding module 514, a video decoder 513, a keypad 512, a camera 517 and a screen 511.

The server 520 comprises an application module 521, a modeling module 522 and an encoder 523. The application module 521, the modeling module 522 and the encoder 523 operate in the same manner as the application module. the modeling module and the encoder of the third embodiment described above. The server 520 further comprises a processing module 525 for decoding and processing a video stream received from the terminal 510.

In this embodiment of the invention illustrates in FIG. 9B the server 520 receives a video stream produced by a Visio 510 mobile phone 3G. This stream, if it is encoded in the rules of the art, contains the vectors of Vm motion calculated by the encoding module 514 of the mobile phone 510 for the purpose of compressing the video sequence. In addition to being used to decompress the video sequence, these motion vectors Vm can be used to estimate the movements of the handset 510.

Referring to FIG. 9b, the motion vectors Vm may be processed by the processing module 515 of the server 520 to determine that the mobile phone 510 has moved from a PSI position to a new position PS2. According to this movement, the application server 520 sends a display S1 corresponding to a first part of a virtual image IV1 when the terminal is at a position PSI, and then when the server 520 detects that the terminal 510 has moved at a position PS2 the server 520 sends a display S2 corresponding to a second part of the virtual image IV1 Thus, without additional calculations in the handset 510 or on the server 520, and without installation of an additional application on the phone 510 , the images presented to the user on the screen 511 of the phone can account for its movements and allow it to navigate a virtual image larger than that of its physical screen 511 such as a map, a list of choices etc.

In addition, a movement to the right of the handset may, for example, mean switching to a next image in the case of viewing a photo album. The movement of the handset can be used to move a pointer on the screen (like a mouse on the computer).

Of course, the invention is not limited to the embodiments described above and shown, from which we can provide other modes and other embodiments, without departing from the scope of the invention. . In variants for example it could of course be envisaged that the first embodiment has features of the other embodiments. For example, in variants the server according to the first, second, fourth or fifth embodiment can be equipped with a modeling module and an encoder according to the third embodiment.

In addition, in other variants, the terminal according to the third, fourth or fifth embodiment of the invention may be equipped with a pointer according to the second embodiment of lineation.

In other variants, the server according to the first, second, third, fifth embodiment of the invention may be equipped with a recording module according to the fourth embodiment to enable direct upload of audio-visual data. by a terminal.

In other variants, the server according to the first, second, third or fourth embodiment of the invention may be equipped with a processing module according to the fifth embodiment to make it possible to determine the movements of a terminal as a function of the motion vectors of video data.

Note that the invention is not limited to a purely software implementation (sequence of instructions of a computer program), but it can also be implemented in a hardware form or any form mixing a hardware part and a software part. In the case where the invention is implemented partially or totally in software form, the corresponding instruction sequence can be stored in a removable storage means (such as for example a floppy disk, a CD-ROM, a DVD-ROM, etc.) or non-removable, the storage means being partially or completely readable by a computer or a microprocessor. 34

Claims (5)

REVENDICATIONS1. A user terminal navigation method (210) on an application running on a remote server (220), the method comprising the steps of displaying a user interface on a display (211) of the terminal based on display data received from the server (210); sensing the movement of a user-mounted positioning device (250) in response to the display content to determine user navigation data on the application; and modulating the motion data in audio format for transmission to the server (220) via an audio channel. 2. Method according to claim 1 wherein the positioning device (250) is arranged to communicate with the terminal (210) via a Bluetooth type link. The method of claim 1 or 2 wherein the display information is received from the server (220) via an audiovisual link (201). 4. Method according to any one of the preceding claims further comprising a step of transcoding the data to Bluetooth transcoding to H324m audio. 25 A method of processing navigation data of a user of a mobile terminal (210) on an application on a remote server (220), the method comprising the steps of: receiving at the server (220) via a telecommunication link (201) of the movement data of a positioning device (250) of the terminal (210) movable according to the movements of the user; and processing the motion data to determine navigation data of the application based on the motion data. The method of claim 5, wherein the motion data is received via an audio channel. The method of claim 5 or 6 wherein the server (220) detects the presence of the positioning device (250) by predefined carrier or sequence detection via the audio channel. A mobile device (210) capable of communicating over a wireless link with an application running on a remote server (220), the device comprising: a terminal handset (210) including a display (211) ) for displaying a user interface from the application and a keypad (212); a positioning device (250) moveable according to the movement of the user, and being able to communicate with the handset via a wireless intermediate; means (254) for generating motion data as a function of the displacement of the positioning device (250); and a modulator (255) for modulating the motion data in audio format for transmission to the server via an audio channel. A device according to claim 8 wherein the terminal (210) comprises a Bluetooth interface (215), and the positioning device (250) comprises a Bluetooth interface (252) for communicating with the Bluetooth interface (215) of the terminal. A device according to claim 8 or 9 wherein the positioning device (250) comprises an accelerometer (254) or a mouse. An application server (220) including an application module (221) for executing at least one application; a demodulator (226) for demodulating the received motion data through an audio channel of a mobile terminal (210); and a motion data processing module (226) for determining navigation data of the application based on the motion data. An application server according to claim 11 further comprising means for detecting the presence of a positioning device. 13. Computer program, for implementing the method according to one of claims 1 to 7, downloadable via a telecommunications network and / or stored in a memory of a processing device and / or stored on a memory medium for to cooperate with a treatment device. 37