EP4341789A1 - Verfahren zur übertragung eines digitalen signals mit einem steuermittel - Google Patents
Verfahren zur übertragung eines digitalen signals mit einem steuermittelInfo
- Publication number
- EP4341789A1 EP4341789A1 EP22729515.1A EP22729515A EP4341789A1 EP 4341789 A1 EP4341789 A1 EP 4341789A1 EP 22729515 A EP22729515 A EP 22729515A EP 4341789 A1 EP4341789 A1 EP 4341789A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- displacement
- phase
- hypothetical
- digital signal
- interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 64
- 230000003993 interaction Effects 0.000 claims abstract description 63
- 230000010365 information processing Effects 0.000 claims abstract description 61
- 230000005540 biological transmission Effects 0.000 claims abstract description 13
- 238000001514 detection method Methods 0.000 claims abstract description 6
- 238000006073 displacement reaction Methods 0.000 claims description 102
- 230000006870 function Effects 0.000 claims description 55
- 238000013213 extrapolation Methods 0.000 claims description 18
- 230000015654 memory Effects 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 12
- 230000001133 acceleration Effects 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 7
- 238000012360 testing method Methods 0.000 claims description 5
- 238000004891 communication Methods 0.000 description 5
- 230000007423 decrease Effects 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 4
- 230000000875 corresponding effect Effects 0.000 description 3
- 230000003466 anti-cipated effect Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 2
- 239000000835 fiber Substances 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000012546 transfer Methods 0.000 description 2
- 101150012579 ADSL gene Proteins 0.000 description 1
- 102100020775 Adenylosuccinate lyase Human genes 0.000 description 1
- 108700040193 Adenylosuccinate lyases Proteins 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 230000002596 correlated effect Effects 0.000 description 1
- 238000005314 correlation function Methods 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000012067 mathematical method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000000513 principal component analysis Methods 0.000 description 1
- 238000012847 principal component analysis method Methods 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000035939 shock Effects 0.000 description 1
- 230000006403 short-term memory Effects 0.000 description 1
- 230000001052 transient effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
- G06F3/038—Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
Definitions
- TITLE Method for transmitting a digital signal by a control means
- the invention relates to a method for transmitting a digital signal by a control means such as a computer mouse.
- the invention also relates to a method of interaction of a control means with a remote server comprising the implementation of such a method of transmitting a digital signal.
- the invention further relates to a computer program product comprising program code instructions for implementing the steps of such methods.
- the invention finally relates to a data recording medium on which is recorded a computer program comprising program code instructions for implementing such methods.
- Information processing machines such as computers or video game consoles are generally controlled by manual or peripheral control means, including a computer mouse or a joystick. These control means are provided with means interaction actionable by a user.
- a digital signal is transmitted by the control means to the information processing machine and interpreted by the latter to display an image on a screen connected to the data processing machine.
- information There is a certain time lag, commonly referred to as latency, between the moment when the user interacts with the means of interaction and the moment when an image taking account of this interaction is displayed on the screen.
- latency time lag
- a sensor detects the movement of the mouse. Then, the mouse transmits a digital signal relating to this movement to a computer.
- This digital signal is received by the computer and interpreted in order to calculate an instruction for moving a pointer and/or a new image to be displayed on the screen. These instructions are then sent from the computer to the screen and then projected on the screen during a next refresh of the image. All of these operations can take a duration of the order of a hundred milliseconds. Users are generally used to such latency.
- the latency between the movement of an interaction means and the display on the screen of an image calculated consecutively to this movement is penalized by the data transfer time between the remote server and the information processing machine.
- Excessive latency can lead to increased user fatigue, loss of performance, and even an inability to use the computer system.
- users resort to means of connecting to the Internet more efficient, such as fiber optics.
- the latency is still too high and the user experience is not completely satisfactory.
- the object of the invention is to provide a method for transmitting a digital signal by a control means overcoming the above drawbacks and improving the methods for transmitting digital signals known from the prior art.
- a first object of the invention is a method for transmitting a digital signal by a manual control means making it possible to reduce the latency between the moment when a user interacts with the control means and the moment when an image taking account of this interaction is displayed on the screen, in particular for video game services on a remote server, called "cloud gaming”.
- the invention relates to a method for transmitting a digital signal by manual control means capable of cooperating with an information processing machine, the method comprising:
- Said hypothetical displacement can be defined by a function of time of a set of hypothetical positions of the means of interaction during the second phase of displacement, said digital signal can comprise at least a part of said set of hypothetical positions of the means of interaction .
- the digital signal associated with a given hypothetical position can be transmitted by said control means at an instant prior to the instant at which the given hypothetical position is reached according to said time function of the set of hypothetical positions.
- the method may include a learning phase comprising:
- Calculating said hypothetical displacement may include determining an extrapolation function as a function of said set of actual positions of the first movement phase.
- Said digital signal may comprise data for defining the extrapolation function.
- the extrapolation function may be a polynomial function, and said digital signal may include coefficients of said polynomial function.
- the method may comprise a step of calculating a space of physically plausible displacements comprising:
- the method may comprise transmission by said control means of a first digital signal describing the first phase of movement of the interaction means, then transmission by said control means of a second digital signal defined as a function of said hypothetical relative movement in the second phase of movement of the means of interaction.
- the invention also relates to a method of interaction of a control means with a remote server comprising the implementation of a method for transmitting a digital signal by a control means to a data processing machine. information as defined above, then the transmission of a third digital signal by the information processing machine to the remote server of the information processing machine, the third digital signal being defined according to said hypothetical displacement.
- the interaction method may comprise the transmission of a fourth digital signal by the remote server to the information processing machine, the fourth digital signal comprising display data calculated according to the third digital signal, and the data display can be transmitted to a screen connected to the information processing machine.
- the display data can be transmitted to the screen only if said hypothetical displacement satisfies a comparison test with a second phase of displacement of the interaction means actually carried out.
- the control means can be a computer mouse or a gamepad.
- the invention also relates to a computer program product comprising program code instructions recorded on a computer-readable medium to implement the steps of the method as defined above when said program runs on a computer.
- the invention also relates to a data recording medium, readable by a computer, on which is recorded a computer program comprising program code instructions for implementing the method as defined above.
- the invention finally relates to a signal from a data medium, carrying the program product as defined above.
- FIG. 1 is a schematic view of a computer system according to one embodiment of the invention.
- FIG. 2 is a first graph illustrating on the one hand the position along an X axis of a computer mouse as a function of time, and on the other hand the position along an X' axis of a pointer displayed on a screen in function of time.
- FIG. 3 is a block diagram of a method for transmitting a digital signal according to a first embodiment of the invention.
- FIG. 4 is a second graph illustrating on the one hand the position along an X axis of a computer mouse as a function of time, and on the other hand the position along an X' axis of a pointer displayed on a screen in function of time.
- FIG. 5 is a third graph illustrating on the one hand the position along an X axis of a computer mouse as a function of time, and on the other hand the position along an X' axis of a pointer displayed on a screen in function of time.
- FIG. 1 schematically illustrates a computer system 1 according to one embodiment of the invention.
- the computer system 1 comprises an information processing machine 20 connected on the one hand to a manual control means 10 and on the other hand to a remote server 30.
- the information processing machine 20 can be, for example, a desktop computer, a laptop computer, a game console, a television, a tablet, a smartphone and more generally any computer equipment with which a user is likely to interact.
- the information processing machine 20 is connected to a screen 21, or monitor, capable of displaying images.
- the screen 21 can refresh the images according to a given frequency, for example 50 Hz or 60 Hz.
- the information processing machine 20 further comprises means 22 for receiving a digital signal from the control means 10, and communication means 23 capable of transmitting and receiving digital signals to the remote server 30.
- the communication means 23 can be Internet connection means.
- the manual control means 10 is a peripheral of the information processing machine 20, such as a computer mouse or a joystick. It allows a user to transmit commands to the information processing machine. For this purpose, it comprises at least one means of interaction 11 intended to be manipulated with one hand or with two hands.
- the interaction means 11 is an element of the control means that can be moved between at least two distinct positions.
- the control means 10 comprises a sensor 12 configured to detect the displacement of the interaction means 11.
- the sensor 12 can detect a plurality of intermediate positions defined between two extreme positions between which the control means is movable.
- the sensor 12 therefore differs from a simple binary sensor, such as a sensor assigned to a button or to a keyboard key, and which is only capable of detecting two distinct positions of the interaction means.
- the sensor 12 can for example be a position sensor, a potentiometer, or an accelerometer.
- the sensor 12 can detect at least one characteristic linked to the movement of the interaction means 11 .
- This feature can be an absolute position of the interaction means and/or a relative position of the interaction means, that is to say a position defined with respect to a previous position of the interaction means.
- the control means 10 also comprises a means 13 for transmitting a digital signal intended for the information processing machine 20.
- the digital signal transmitted by the control means can convey information on a relative position and/or on an absolute position of the means of interaction.
- the transmission means 13 can be connected to the information processing machine 20 via its reception means 22.
- the connection between the control means 10 and the information processing machine can be either a wired connection or a connection wireless, such as a Bluetooth link.
- the control means is capable of transmitting digital signals to the information processing machine at a frequency significantly higher than the refresh rate of the screen 21, for example at a frequency of 500 Hz or 1000 Hz.
- the control means 10 further comprises a microprocessor 14, a memory 15 for storing data and an internal clock 16.
- the microprocessor 14 is connected to the memory 15, to the sensor 12, to the transmitting means 13 and to the internal clock 16.
- the internal clock 16 makes it possible to associate a time base with a position detected by the sensor 12.
- the memory 15 is a data recording medium on which is recorded a computer program comprising instructions for program code for implementing a method for transmitting a digital signal according to one embodiment of the invention.
- the microprocessor 14 is capable of performing this method at least in part.
- the control means 10 can for example be a computer mouse or a game joystick. In general, the control means is intended to be controlled with the hands, in particular the fingers of a user.
- the control means 10 is separate from the information processing machine 20 to which it can be connected by a wired or wireless connection means.
- the control means 10 is also separate from the screen 21 to which it is connected via the information processing machine 20.
- control means is a computer mouse
- it preferably comprises a casing, movable with the hand of a user on a support plane.
- the box then constitutes a means of interaction of the mouse.
- the sensor 12 can for example be an optical sensor integrated into a sole of the mouse to detect its movements on the support plane.
- a pointer in the form of an arrow is displayed on the screen 21. The movements of the pointer are correlated with the movements of the mouse.
- the computer mouse can include other means of interaction such as for example a joystick linked to the casing and orientable in at least one direction.
- the interaction means can for example be a more or less tiltable joystick in at least one given direction, a trigger whose position can be controlled with a user's finger.
- the position of the joystick and/or of the trigger can be detected by means of a sensor integrated into the joystick, such as a potentiometer for example.
- the remote server 30 is connected to the information processing machine 20 via communication means 23.
- the connection between the remote server 30 and the information processing machine 20 can be established via an Internet connection box (not shown).
- This Internet connection box may be on the one hand in wireless connection, for example in Wi-Fi connection, with the information processing machine 20, and on the other hand it can be in wired connection, for example in an ADSL or fiber optic link to the remote server 30.
- the remote server 30 comprises a microprocessor 31, a memory 32, and communication means 33 allowing it to be connected to the information processing machine 20.
- the remote server 30 can be configured to run a video game.
- the control instructions are received from the information processing machine 20 by the remote server 30. Sequences of the video game are calculated by the remote server and images to be displayed on the screen 21 are sent to the machine. information processing 20.
- the remote server 30 is configured to run a video game service, called "cloud gaming”. It is further assumed that the images to be displayed on the screen 21 depend on the position and/or movement of the interaction means 11 of the control means 10. For example, if the pointer reaches a specific zone such as the edge of the screen, a background displayed on the screen can scroll automatically as long as the pointer remains positioned in that specific area.
- the example of a computer mouse will be used as the control means.
- the invention may be transposed to the use of any other manual control means.
- the mouse is freely movable in a plane.
- the movements of the mouse can therefore be represented in a two-dimensional frame.
- only the displacements of the mouse along a first axis X will be considered below.
- the invention may be transposed to the displacement of the mouse along a second axis Y perpendicular to the first axis. More generally, the invention can be transposed to any manual control means as defined previously.
- FIG. 2 is a graph representing, by a first curve C1 in solid line, the actual position of the mouse relative to the axis X as a function of time t during a movement.
- the movement is limited temporally by two instants t1 and t5 when the speed of the mouse is zero.
- Moving the mouse is characterized by an acceleration phase followed by a holding phase where the mouse is moved at a relatively stable speed and then a slowing down phase down to zero speed.
- the duration of a mouse movement can typically be between a hundred milliseconds for a very fast movement, and several seconds for a long movement.
- This same graph shows a second curve C2 in solid lines corresponding to the position along an axis X′ of a pointer displayed on the screen 21 as a function of time t, without the implementation of the transmission method according to the 'invention. It is observed that there is a time lag DT between the physical displacement of the mouse and the displacement of the pointer displayed on the screen. This time shift DT is called latency. To simplify the presentation, it is considered that this latency is identical throughout the movement of the mouse. Thus the curve C2 is identical to the curve C1 but translated on the time axis by a value equal to the latency DT.
- the latency DT may for example be of the order of 100 ms, or even 200 ms. It is mainly related to the time required to send digital signals from the information processing machine to the remote server, for the execution of calculations by the remote server, and finally to send digital signals from the remote server to the machine. information processing.
- the invention proposes to reduce the latency delay DT by means of hypotheses, or in other words predictions, on a future movement of the mouse.
- the future movement of the mouse can be anticipated by considering kinematic and biomechanical laws to which the user holding the mouse in his hand is subject. More generally, any means of interaction is constrained by kinematic and biomechanical laws of the human body.
- the mouse emits a digital signal determined according to this hypothetical displacement.
- This digital signal decoys the information processing machine to which it is connected.
- the curve C3, in dotted lines represents the mouse movement information sent by the mouse to the information processing machine as a function of time t.
- the pointer displayed on the screen which is represented by the dotted curve C4, is moved with the latency DT relative to the movement signal emitted by the mouse.
- Curve C4 is therefore shifted by the latency delay DT with respect to curve C3.
- a latency DT' is defined as equal to the time difference between the actual movement of the mouse (curve C1) and the movement of the pointer on the screen (curve C4).
- a latency DT′ lower than the latency DT is thus obtained. This latency may even be zero.
- the transmission method according to the invention can be repeated according to a predefined frequency, in particular a frequency corresponding to the frequency at which the mouse is able to transmit new signals to the information processing machine, that is to say say 500 Hz, or even 1000 Hz.
- a predefined frequency in particular a frequency corresponding to the frequency at which the mouse is able to transmit new signals to the information processing machine, that is to say say 500 Hz, or even 1000 Hz.
- the method begins with a first phase P1, called the learning phase.
- the learning phase may comprise a step E01 of moving the mouse, a step E02 of detecting the positions of the mouse by means of the sensor 12, and a step E03 of storing in the memory 15 the positions and the mouse as a function of time during each of these movements.
- the learning moves can be stored as a pair set including time and position.
- a database is thus constituted, recorded in the memory 15, which characterizes the movements of the mouse, in particular the trajectory followed by the mouse, its amplitude of movement, its speed of movement, and its acceleration.
- a so-called principal component analysis method also known by the acronym PCA for “principal component analysis”.
- PCA principal component analysis
- a given number of reference displacements can be defined, the characteristics of which are refined as the different learning displacements are executed.
- the number of reference movements may for example be between ten and twenty.
- a reference move can be defined by a set of reference positions as a function of time.
- a reference displacement can be calculated, by example, by carrying out an average of all the learning displacements satisfying a test of resemblance with the reference displacement. If a learning movement does not satisfy any test of resemblance to a reference movement, a new reference movement can be created. If the maximum number of predefined reference movements is reached, a reference movement for which the lowest number of learning movements has been carried out can be deleted. This frees up storage space in the memory.
- the learning phase may be based on artificial intelligence methods, in particular deep learning methods (commonly referred to as deep learning) and/or on the use of a neural network based on an architecture known as LSTM (English acronym for Long Short Term Memory).
- LSTM Long Short Term Memory
- This learning phase is then used to calculate a hypothesis on a future movement of the mouse.
- the movement of the mouse as represented by the curve C1 in FIG. 2 can be broken down into two successive phases: a first phase of movement D1 is directly followed by a second phase of movement D2.
- the first phase D1 is defined between times t1 and t2 and the second phase is defined between times t2 and t5.
- a set of positions of the mouse during the first movement phase D1 is detected by means of the sensor 12.
- this set of positions can be a set of relative positions or a set of absolute positions.
- This set of positions is then recorded at least temporarily in the memory 15 of the mouse in the form of a function of time.
- the mouse can transmit to the information processing machine 20 first digital signals relating to the positions of the mouse during the first movement phase D1 previously detected.
- a hypothetical movement relating to the second phase of movement of the mouse is calculated during a step E3, and by means of the microprocessor 14 integrated into the mouse.
- This hypothetical displacement is calculated according to the positions of the mouse during the first phase of displacement D1, in particular by comparing the set of effective positions of the means of interaction during the first phase of displacement D1 with the learning displacements or the reference displacements.
- the sensor 12 integrated into the mouse is capable of supplying the microprocessor 14 with a large quantity of information on the first phase of movement. Sensor 12 can typically acquire data at a frequency of 2000Hz. All of this information is not transmitted to the information processing machine by the first digital signals because it is useless for the information processing machine or for the remote server to know the position of the mouse with a very high precision.
- the frequency of sending information from the mouse to the information processing machine can be lower than the frequency of data acquisition by the sensor 12, typically of the order of 125 Hz.
- the large amount of information detected by the sensor 12 can be used by the microprocessor 14 to determine with great precision the movement of the mouse during its first phase of displacement. This allows the hypothetical displacement to be calculated more accurately and reliably.
- Said hypothetical displacement can be defined by a function of time of a set of hypothetical positions of the means of interaction during the second phase of displacement. It can be considered that this hypothetical displacement is calculated instantaneously or almost instantaneously at time t2.
- this hypothetical displacement it is possible to search in the database storing the learning displacements or the reference displacements for a particular displacement whose first phase resembles the first displacement phase D1 as closely as possible. It is for example possible to calculate a score equal to the result of a correlation function between on the one hand the first phase of movement and on the other hand each learning movement or each reference movement stored in the memory 15. Then, selects the learning move or the reference move that gets the highest score. Finally, the hypothetical displacement is defined as equal to the second displacement phase of the selected displacement.
- the method may comprise a step of identification by the mouse of software executed by the remote server, in particular of a video game executed by the remote server.
- the learning phase P1 and/or the calculation step E3 can then be made specific to the use of this software.
- the movements of the mouse necessary to issue commands may be specific to the software being executed. For example, when the software is a so-called "to the first person", that is to say a video game in which the images are displayed according to the point of view of a character, a command to flip the character is carried out by a very specific movement of the mouse.
- the method can also comprise a step of identifying a user of the mouse, the learning movements and/or the reference movements being specific to each user.
- a step E4 the mouse transmits to the information processing machine 20 second digital signals relating to the previously calculated hypothetical displacement. From time t2, the mouse emits hypothetical positions instead of the positions actually reached. The second digital signals therefore convey hypothetical positions of the mouse.
- these hypothetical positions are emitted by the mouse before they are actually reached: a movement of the mouse is thus simulated which is faster than its real movement.
- the digital signal associated with a given hypothetical position is transmitted by said control means at a time prior to the time at which the given hypothetical position is reached according to the previously calculated hypothetical displacement. In FIG. 2, this results in a steeper slope of curve C3 than the slope of curve C1 from time t2.
- the movement of the pointer is faster but is however not instantaneous.
- the curve C4 describes the displacement of the pointer with the implementation of the invention.
- the first hypothetical position is transmitted at time t2 by the mouse.
- the movement of the pointer following the transmission of the first hypothetical position occurs at time t3. From this instant t3, the speed of movement of the pointer on the screen is therefore greater than the speed it would have had in the absence of implementation of the invention.
- the latency DT' defined between the real movement of the mouse and the real movement of the pointer is thus a decreasing function of time. According to the example of FIG.
- the latency DT′ reaches a value of zero or close to zero at time t4.
- the second digital signals can be transmitted so as to obtain a linear decrease in the latency DT′.
- the example of figure 2 is built by considering that the hypothetical movement corresponds exactly to the real movement of the mouse. In practice, the hypothetical displacement does not always correspond exactly to the displacement actually carried out. However, as the process is repeated several times during the movement of the mouse, it is possible to recalculate and correct the hypothetical movement as the actual movement of the mouse progresses. Any inaccuracy in the calculated hypothetical displacement may result in a lower reduction in the latency period. However, the latency period obtained thanks to the invention is never greater than the latency period which would be obtained in the absence of the method according to the invention.
- Figure 4 illustrates by way of example a case where the hypothetical movement differs from the movement actually carried out. It can be seen that the curve C3 describing the hypothetical displacement deviates slightly from the real displacement of the mouse. Consequently, the curve C4 representing the displacement of the pointer on the screen also undergoes these slight variations. However, thanks to a recognition of the displacement phase of the mouse during which its speed decreases, we still manage to anticipate the position at which the mouse stops.
- the remote server 30 When the signal emitted by the mouse is received by the information processing machine, the latter forwards it to the remote server 30 via a third digital signal in a step E5.
- the remote server interprets this signal as a command from the user and calculates an image to be displayed on the screen during a step E6.
- the remote server executes the algorithms of the video game. Once this calculation has been executed, it sends back, in a step E7, a fourth digital signal intended for the information processing machine.
- This fourth digital signal includes data for displaying an image on the screen 21. This is data which can be used like a video signal to display the image on the screen 21.
- the machine information processing must not execute the algorithms of the video game.
- the video game must not be installed in the memory of the information processing machine.
- the information processing machine therefore acts as a simple gateway between the mouse, the remote server and the screen.
- the actual mouse movement continues.
- the information processing machine receives the display instructions, it can be confirmed whether the hypothetical movement conforms to the movement actually performed.
- the display instructions can be transmitted to the screen only if said hypothetical displacement satisfies a comparison test with a second phase of displacement of the interaction means actually carried out.
- the mouse can transmit to the information processing machine and in parallel position signals, a confirmation signal indicating whether the actual movement of the mouse has been consistent with the hypothetical movement.
- the mouse can emit in parallel digital signals conveying the predictive position of the mouse, digital signals conveying the actual position of the mouse. This double information can be transferred to the remote server which then calculates two image instructions corresponding to these two digital signals.
- the information processing machine can then arbitrate, when it receives these two instructions, which it must actually display on the screen, according to a movement actually made by the mouse.
- FIGS. 2 and 3 the method is implemented so that the latency delay is close to zero. It is however possible to implement the method simply to reduce the latency period. The more a displacement is anticipated in the future, the greater the risk that the hypothetical displacement will be far from the real displacement. Thus, it may be advantageous to implement the method solely for the purpose of anticipating the movement of the mouse by about ten milliseconds or a few tens of milliseconds. It is thus possible to make a video game experience using a remote server close to that of a video game directly executed by the information processing machine and for which a reasonable latency period is accepted.
- FIG. 5 illustrates by way of example a method in which the latency delay is reduced from a value DT to a non-zero value DT′ but strictly less than the latency delay DT.
- the invention also provides for the reception by the mouse of information relating to an effective latency period between the mouse and the information processing machine and/or the remote server. Then, a more or less distant prediction can be selected and emitted by the mouse to replace the original data. This helps to smooth the perceived latency and therefore helps to correct jerky operation related to erratic latency.
- a certain number of variant embodiments of the process which has just been described will now be described. These different variants can be freely combined with each other. According to a variant embodiment of the method according to the invention, the method could base the calculation of the hypothetical displacements on the basis of an extrapolation function.
- the extrapolation functions can be, for example, polynomial functions and in particular Bézier functions and/or splines, that is to say piecewise polynomial functions.
- the extrapolation function can be a third degree polynomial defined by the following formula:
- a, b, c and d are parameters of the extrapolation function to be determined.
- the set of effective positions of the mouse during the first movement phase is then used to determine these parameters.
- These parameters can be determined by the least squares method or by an equivalent method.
- a space of physically plausible movements of the mouse Prior to the determination of the parameters of the extrapolation function, it is possible to determine a space of physically plausible movements of the mouse. This displacement space characterizes the plausible displacements of the mouse taking into account the kinematic and biomechanical laws of the human body. For example, a mouse intended to be held in the hand of a user cannot exceed a certain maximum displacement amplitude, a certain maximum speed, or even a certain maximum acceleration.
- the maximum distance traveled by the mouse is 3m.
- the maximum mouse speed is 3m. s -1 .
- the maximum values can be calculated on the basis of different learning movements carried out during a learning phase. This makes it possible in particular to define maximum values adapted to a particular user. They can also be assigned to a particular user, previously identified.
- the space of physically plausible displacements can be used to filter out signals manifestly beyond the control of the user. For example, if the mouse stumbles against an obstacle during its movement, the shock against the mouse can result in a very significant acceleration of the mouse. The method can then be implemented to predict the intentional movement of the user, that is to say the movement that the mouse would have followed if it had not come up against the obstacle.
- the mouse is sometimes raised by the user relative to the support on which it rests in order to generate a large-amplitude displacement of the pointer displayed on the screen. Although the mouse is lifted, the sensor can detect mouse movement. Raising the mouse and moving it out of the support then leads to the detection of an unintentional movement resulting in an undesired movement of the pointer. Such uplift can be detected and filtered. Instead, the mouse can emit a signal to move the pointer in the same direction it was following before the mouse was lifted.
- a communication protocol can be defined between the mouse, the information processing machine and the remote server so that the parameters defining the extrapolation function are sent as a replacement or in addition mouse position information.
- Sending extrapolation function parameters instead of mouse positions reduces the amount of information exchanged with the remote server, and therefore frees up bandwidth that can be usefully used to transfer other data operation of a video game and thus improve the user experience.
- Sending the parameters of the extrapolation function in addition to the positions of the mouse could make it possible to subcontract the calculation of the hypothetical displacement to the information processing machine or even to the remote server.
- a system comprising a computer, a screen and a peripheral, the entire system operating without the intervention of an external Internet network, that is to say without a remote server.
- the display frequency of the screen is 60Hz, ie the image displayed on the screen is refreshed approximately every 16 ms.
- the capture frequency of user commands is 500Hz.
- the duration of a movement of the control means is 180ms.
- the inherent latency of the considered system is 50ms.
- the target latency is 2ms.
- Table 1 illustrates the latency observed during the display of each image during the movement of the control means, with and without implementation of the method according to the invention.
- the initial conditions are identical to the first example, and the latency is also smoothed so that it decreases more gradually.
- the lower bound of the latency is inherent to the system. It was assumed that the latency of the system is 2 ms, however this value depends on many parameters such as the hardware configuration of the information processing machine, the load of the remote server, the congestion of the Internet network, possible interference in connecting cables. The lower bound for latency can also fluctuate over time.
- a system comprising a computer, a screen, a peripheral, the computer being connected to a remote server.
- the screen display frequency is 60Hz, ie the image displayed on the screen is refreshed approximately every 16 ms.
- the capture frequency of user commands is 500Hz.
- the duration of a movement of the control means is 180ms.
- the inherent latency of the considered system is 80ms.
- the target latency is 2ms. [Table 3]
- the initial conditions are identical to the third example, and the latency is also smoothed so that it decreases more gradually.
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Position Input By Displaying (AREA)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
FR2105222A FR3123132A1 (fr) | 2021-05-19 | 2021-05-19 | Procédé d'émission d'un signal numérique par un moyen de contrôle |
PCT/EP2022/063104 WO2022243206A1 (fr) | 2021-05-19 | 2022-05-13 | Procédé d'émission d'un signal numérique par un moyen de contrôle |
Publications (1)
Publication Number | Publication Date |
---|---|
EP4341789A1 true EP4341789A1 (de) | 2024-03-27 |
Family
ID=76730792
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP22729515.1A Pending EP4341789A1 (de) | 2021-05-19 | 2022-05-13 | Verfahren zur übertragung eines digitalen signals mit einem steuermittel |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP4341789A1 (de) |
FR (1) | FR3123132A1 (de) |
WO (1) | WO2022243206A1 (de) |
Family Cites Families (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9897805B2 (en) * | 2013-06-07 | 2018-02-20 | Sony Interactive Entertainment Inc. | Image rendering responsive to user actions in head mounted display |
US10585472B2 (en) * | 2011-08-12 | 2020-03-10 | Sony Interactive Entertainment Inc. | Wireless head mounted display with differential rendering and sound localization |
US9063330B2 (en) * | 2013-05-30 | 2015-06-23 | Oculus Vr, Llc | Perception based predictive tracking for head mounted displays |
-
2021
- 2021-05-19 FR FR2105222A patent/FR3123132A1/fr active Pending
-
2022
- 2022-05-13 WO PCT/EP2022/063104 patent/WO2022243206A1/fr active Application Filing
- 2022-05-13 EP EP22729515.1A patent/EP4341789A1/de active Pending
Also Published As
Publication number | Publication date |
---|---|
FR3123132A1 (fr) | 2022-11-25 |
WO2022243206A1 (fr) | 2022-11-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10586395B2 (en) | Remote object detection and local tracking using visual odometry | |
EP2400460B1 (de) | Bewertungsverfahren der horizontalen Geschwindigkeit einer Drohne, insbesondere einer Drohne mit über den Autopiloten gesteuertem stationärem Flug | |
EP2572319B1 (de) | Verfahren und system zur fusion von daten aus bildsensoren und aus bewegungs- oder positionssensoren | |
EP2724203B1 (de) | Erzeugung von kartendaten | |
EP3112803A1 (de) | Kamerablock zur aufnahme an bord einer drohne zur kartografischen erfassung eines geländes, und verwaltungsverfahren der bilderfassung durch einen kamerablock | |
FR2917931A1 (fr) | Procede et systeme de mise en relation entre des personnes dans un systeme de telecommunications. | |
FR2878641A1 (fr) | Procede de navigation automatique contrainte vers des regions d'interet d'une image | |
US20180250593A1 (en) | Cut-scene gameplay | |
EP3392728A1 (de) | Steuerungsverfahren einer drehflügel-drohne, entsprechende software, entsprechendes elektronisches gerät und entsprechende drohne | |
US20200106819A1 (en) | Inducing higher input latency in multiplayer programs | |
EP3040107B1 (de) | Verfahren zur steuerung eines zielcursors mithilfe eines joysticks, und entsprechender joystick | |
EP3187979B1 (de) | Verfahren zur steuerung eines zielcursors mithilfe eines joysticks, und entsprechender joystick | |
EP4341789A1 (de) | Verfahren zur übertragung eines digitalen signals mit einem steuermittel | |
FR2607298A1 (fr) | Procede de lecture d'une scene mobile enregistree, notamment sur videodisque et application a des simulateurs de conduite | |
EP3367353B1 (de) | Steuerverfahren für eine ptz-kamera, zugehöriges computerprogramm-produkt und steuervorrichtung | |
FR2876470A1 (fr) | Procede de controle d'affichage utilisant un equipement portable a capteur d'images | |
EP3918576A1 (de) | Verfahren zur dynamischen dreidimensionalen bildgebung | |
EP2534610A1 (de) | System und verfahren zur echtzeitbestimmung eines parameters einer repetitiven bewegung | |
EP3942390A1 (de) | Vorrichtung, system und verfahren zur verarbeitung von daten von virtueller realität | |
WO2023046902A1 (fr) | Systeme d'interface homme-machine | |
FR3069664A1 (fr) | Procede d’assistance d’au moins un mouvement d’un utilisateur et dispositif correspondant | |
EP2234066B1 (de) | Entfernungsmessung aus Stereobildern | |
FR2836570A1 (fr) | Procede de transmission de donnees d'images entre un serveur et un terminal | |
EP1739626B1 (de) | Verfahren zur Darstellung virtueller graphischer Objekte in Bildern eines Felds, insbesondere eines Spielfelds für ein Sportereignis | |
FR3144369A1 (fr) | Procede et dispositif de prise de vue d’une zone d’interet d’un sujet en mouvement |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: UNKNOWN |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE |
|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
17P | Request for examination filed |
Effective date: 20231116 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAV | Request for validation of the european patent (deleted) | ||
DAX | Request for extension of the european patent (deleted) |