WO2016071171A1

WO2016071171A1 - Method and device for changing a view point of a three-dimensional map (3d) or of an image of an observed three-dimensional physical object (3d) by recognising a gesture on a multi-point touch screen

Info

Publication number: WO2016071171A1
Application number: PCT/EP2015/074943
Authority: WO
Inventors: David Arneau; Fabien RICHARD
Original assignee: Thales
Priority date: 2014-11-07
Filing date: 2015-10-28
Publication date: 2016-05-12
Also published as: FR3028330B1; FR3028330A1

Abstract

The invention relates to a method for changing a view point of a three-dimensional map (3D) or of an image of an observed three-dimensional physical object by recognising a gesture on a multi-point touch screen, which comprises a step of recognising a gesture, in which is determined (204) a first common touch movement Dy (212) of sliding (208) first and second fingers, arranged side-by-side without separating (210), in a first predetermined direction Diry over the surface of the touch screen between a start of the movement and an end of the movement of the two fingers. The method also includes a control step (214) during which a command Cy to modify a first current azimuth angle of the view point of the adjustable camera is determined (206) in accordance with the first touch movement DY. A device or a system for changing a view point of a three-dimensional map or of an image of an observed three-dimensional physical object is configured such as to implement the method for changing the view point.

Description

METHOD AND DEVICE FOR CHANGING THE POINT OF VIEW OF A THREE DIMENSIONAL CARD (3D) OR IMAGE OF A PHYSICAL OBJECT

THREE-DIMENSIONAL (3D) OBSERVED BY RECOGNITION

GESTURAL ON A MULTI-POINT TOUCH SCREEN

The present invention relates to a method and a device or system for changing the point of view of a three-dimensional (3D) map or an image of a three-dimensional observed physical object (3D) by recognizing a gesture on a touch screen multipoint.

Equivalently, the invention relates to a method and a device or system for changing the point of view of a virtual or real-world camera whose angles of attitude or orientation can be modified by commands. A virtual camera is, for example, a virtual camera produced by a cartographic engine, and a physical or material camera is, for example, a video camera, a camera, or any other device. physical sensor of radiation images whose orientation can be controlled and modified. More particularly, the camera may be an onboard video camera on board an aircraft visualizing the earth's surface.

The invention also relates to a computer program or an architecture configuration of a dedicated circuit such as an ASIC or an FPGA, which when implemented in the point-of-view change system allows the implementation implementation of the method of change of point of view.

Most Geographic Information Systems (GIS) allow the display and manipulation of digital mappings. Some of these GIS systems allow the display of a three-dimensional globe, such as Google Earth, Nasa WorldWind, AGI lnsight3D. Historically, the physical interface to manipulate the map was the mouse and allowed to adjust the point of view of the camera with the right button and the left mouse button.

But with the proliferation of tactile interfaces, there are no more right and left buttons and new gestural interactions have been developed to adapt to this new medium. This is the case for example of the so-called "pinch to zoom" gesture, where two fingers move apart to zoom in on an area on the map. In the technical field of developing three-dimensional 3D cartographic applications of the terrestrial globe on restricted tactile surfaces, for example those of screens of about ten inches, and in a turbulent environment, for example in a cramped cockpit of an airplane or aircraft. helicopter subjected to atmospheric disturbances or a hostile environment, the establishment of a specific gesture to quickly and simply orient the point of view of a camera displaying a three-dimensional map is sought.

More specifically, the support of a fast interaction gesture and simple, on touch surface, to adjust the point of view of a camera along the three axes of a marker by changing the angles from the point of view to through gestures, is sought.

To date, various solutions to solve this problem have been realized or proposed.

In a first solution, the right mouse button is held down while moving the mouse, which makes it possible to manage the different angles of view of the camera. However, because of the absence of a right button on a touch surface, the use of such a user interface surface requires that the right click of the mouse is performed by a long-term support on the touch surface, the order of the second. This second waiting is not compatible with use in a turbulent environment in an aircraft cockpit because it is too long and anti-ergonomic.

According to a second solution, 3-finger or 4-finger gestures on tactile surfaces have been developed for manipulating the point of view of three-dimensional cards by a user. However the use of more than two fingers on a touch pad by a driver or an operator is inconvenient and anti-ergonomic turbulent embedded environment. Thus, the use of a gesture to a finger or two fingers maximum should be preferred in the case of avionics applications on tablets embedded in a cockpit of an aircraft, which is not currently the case, especially for the adjustment of the angles of point of view of the camera.

The patent application published under the reference WO 2014/014806 A1 describes a multi-touch screen, configured to manipulate displayed images of a three-dimensional object through gestures. using two fingers. The document particularly describes a "pinch to zoom" gesture for zooming the map, a vertical sliding gesture of two fingers held side by side to make a change in the angle of inclination of the camera, and a pivot gesture slid two fingers around a point on the screen to make a change in the azimuthal angle from the point of view of the camera. However, the gesture proposed to change the angles of point of view of the camera remains uncomfortable and complicated to use in the case of use in a turbulent environment, such as that of an airplane or helicopter cockpit.

The technical problem is to find a method and a device or system for changing the point of view of a three-dimensional (3D) map by changing the attitude angles of the camera by recognizing a gesture on a multi-touch screen that is more ergonomic, easier to use and more efficient in terms of speed in order to control the change of the point of view in the case of use in a turbulent environment, such as that of a cockpit. plane or helicopter.

For this purpose, the subject of the invention is a method of changing from a point of view of a three-dimensional (3D) map or an image of a three-dimensional object (3D) by a recognition of a gesture on a multi-touch screen, the three-dimensional map or the image of the three-dimensional object (3D) being provided by a map engine through a virtual camera or by a real-life camera, including steps consisting of in that,

through tracking a first trajectory and a second trajectory of contact points on the touch screen respectively corresponding to the contacts of a first finger and the contacts of a second finger,

in a first step of recognizing a gesture, it is determined a first tactile displacement Dy common slip of the first and second fingers, arranged side by side without departing, in a first predetermined direction Diry on the surface of the screen tactile between a beginning of the movement and an end of the movement of the two fingers; and

in a second control step, a first Cy command modifying a first azimuth angle current from the point of view of the camera is determined according to the first tactile movement Dy.

According to particular embodiments, the method of changing the point of view comprises one or more of the following features:

the first step of recognizing a gesture comprises a third step during which the beginning of the sliding movement of two fingers simultaneously in contact on the surface of the touch screen is detected, followed by a fourth step during from which, from the beginning of the first and second trajectories of the contact points, it is determined whether the sliding of the two fingers is carried out the two fingers side by side without departing, followed in the affirmative by a fifth step at during which the first tactile displacement Dy following the first predetermined direction Diry of the surface of the touch screen is determined between the beginning of the movement and an end of the movement of the two fingers; and the second control step is followed by a sixth step in which, from the first command Cy, received by the map engine or the real-world camera, and a current azimuth angle Yc d attitude of real or virtual camera, the virtual camera of the cartographic engine or the real camera rotates a current point of view of a first azimuth angle Ay controlled, according to the first command Cy;

in the first step of recognizing a gesture, it is also determined a second tactile displacement Dx common slip of the first and second fingers, arranged side by side without departing, in a second predetermined direction Dirx on the surface of the touch screen between the beginning of the movement and the end of the movement of the two fingers, and in the second control step, a second command Cx for modifying a second angle of inclination current from the point of view of the gripping device. views is further determined according to the second touch shift Dx;

the second tactile displacement Dx along the second predetermined direction Dirx of the surface of the touch screen between the beginning of the movement and the end of the movement of the two fingers is determined in the fifth step, and in the sixth step, from the second command Cx, received by the map engine or the real-world camera, and a second current angle of inclination x of the attitude of the camera, the virtual camera of the cartographic engine or the actual camera rotates a current viewpoint of a second inclination angle Δχ controlled, a function of the second command Cx;

the virtual or real-life camera comprises a retina with an optical center C and an optical axis, and is geometrically marked by a current position and a current attitude, the current position being defined in a spherical coordinate system of center O by a longitude, a latitude and an altitude of the optical center C, and the current attitude being defined in a local coordinate system by a first current azimuthal angle y and a second current angle of inclination x, the first current azimuth angle x being an angle an orientation separating the projection of the optical axis in a current orientation on an azimuthal projection plane containing the optical center C, and tangent to the center sphere O and containing the optical center C, and an azimuthal reference axis, contained in the same azimuthal projection plane and whose direction depends on the current position of the camera, starting from the azimuthal reference axis, the second angle of inc current lineage x being an oriented angle separating the optical axis of the retina in the current orientation and the axis joining the optical axis C and the center O of the spherical coordinate system starting from the current optical axis, the touch screen has a flat tactile surface with a first Diry direction and and gesture recognition being implemented by one or more electronic computers;

the azimuthal reference axis is the projection on the azimuthal projection plane of an axis joining the optical center and a fixed point situated on a sphere of center O and of radius less than the altitude of the setting apparatus views;

.- the three-dimensional maps are maps of the terrestrial globe, the spherical geometric landmark is a geocentric landmark rotating with the Earth, and the fixed point is a point equal to or near the geographical north;

the fixed point is the magnetic north and the azimuthal reference axis is the magnetic north axis provided in particular by a gyrocompass; .- the three-dimensional maps provided are views calculated by the cartographic engine from a database when the camera is virtual, and the images of the three-dimensional object are actual images captured by the camera shooting when the camera is real;

the determinations of the first and second tactile displacements Dy, Dx are implemented in series or in parallel, and / or the execution steps of the first and second commands Cy, Cx are executed in series or in parallel;

the first step is preceded by an initialization step during which a reliable starting state of the two fingers is detected, and a starting position of each finger as well as the distance separating the two fingers and the central position between the two fingers are determined, the starting position of a finger being an average position of the positions received from said finger during an initial observation time window;

in the third step, the beginning of a sliding movement of the two fingers is detected if the distance traveled by the central position of the two fingers is greater than a first threshold value ε during a first time window of start of observation of the trajectories, and during the fourth step and during a first time window of start of observation of the trajectories a scalar product between two normalized vectors of the displacements of the two fingers is calculated, then compared to a positive threshold value, preferably between 0.75 and 1;

the second step comprises a step, implemented when the scalar product between the two normalized vectors of the displacements of the two fingers is greater than the second threshold value, in which it is verified that the standards of the movements of the two fingers are of the same order of magnitude and decide to change at least the first tilt angle of the camera.

The invention also relates to a device or a system for changing from a point of view of a three-dimensional (3D) map by recognizing a gesture on a multi-touch screen, comprising: real views or a map engine with a virtual camera to provide an image or a map three-dimensional view of a scene of a three-dimensional object according to a current viewpoint of the real or virtual camera;

a multi-touch screen for following a first trajectory and a second trajectory of contact points on the touch screen respectively corresponding to the contacts of a first finger and the contacts of a second finger, in the form of tactile events;

one or more electronic computers, configured for starting from the tactile events of the two fingers, determining in a first step of recognizing a gesture a first tactile displacement Dy common sliding of the first and second fingers, arranged side by side without s spacing, in a first predetermined direction Diry on the surface of the touch screen between a beginning of the movement and an end of the movement of the two fingers; and. determining in a second control step, a first command Cy modifying a first azimuth angle current from the point of view of the camera according to the first tactile displacement Dy.

The invention also relates to a product or computer program comprising a set of instructions configured to implement the method defined above when they are loaded into and executed by one or more computers of the device or system defined herein. -above.

The invention will be better understood on reading the description of several embodiments which will follow, given solely by way of example and with reference to the drawings in which:

FIG. 1 is a view of an architecture of a first embodiment of a point-of-view angle-change device of a virtual camera according to the invention;

FIG. 2 is a view of an architecture of a second embodiment of a device for changing angles from the point of view of a real-life camera according to the invention;

FIG. 3 is a flow chart of a first embodiment of a point-of-view angle change method according to the invention;

FIG. 4 is a flow chart of a second embodiment of a point-of-view angle change method according to the invention; FIG. 5 is a view of a control gesture making it possible to carry out the method of changing the azimuth angle from the point of view according to the methods described in FIGS. 3 and 4;

FIGS. 6A, 6B, 6C are respectively a view of the starting state from a point of view of a camera and a terrestrial map corresponding to a current starting azimuth angle of the camera, a view of a horizontal movement of two fingers on a touch screen typical of the type of gesture described in Figure 5, and a view of the arrival state of the point of view corresponding to an azimuth angle of arrival of the camera as a result of the application of horizontal displacement;

7 is a view of a control gesture additional to that described in FIG. 5 making it possible to change the angle of inclination from the point of view of the camera according to the method described in FIG. Figure 4;

FIGS. 8A, 8B, 8C are respectively a view of the starting state from a point of view of the camera and of a terrestrial map corresponding to a current starting angle of inclination of the camera. camera, a view of a vertical movement of two fingers on a touch screen typical of the type of gesture described in Figure 7, and a view of the arrival state of the point of view corresponding to an angle of arrival angle of the camera resulting from the application of vertical downward movement;

FIG. 9 is a view of a general gesture of control of variation of the azimuth angle and / or angle of inclination from the point of view of the camera suitable for the method described in FIG. 4;

FIG. 10 is a flowchart of a complete method of changing the point of view of a camera incorporating the change of viewing angles of the camera described in FIGS. 3 and 4. ;

Figure 11 is a detailed flowchart of the complete point-of-view switching method described in Figure 10.

According to FIG. 1 and a first embodiment, a device 2 for changing from a point of view of a three-dimensional (3D) map by recognition of a gesture on a multi-touch screen, includes a multi-touch screen 4, an electronic calculator 6 and a memory 8 for data backup, interconnected by bidirectional links 10, 12.

The multi-touch screen 4 is configured to detect and follow a first trajectory 14 and a second trajectory 1 6 of contact points on a flat surface 18 of the touch screen 4 respectively corresponding to contacts 20, 22 of a first finger and contacts 24, 26 of a second finger, in the form of tactile events, only four contact points 20, 22, 24, 26 of fingers being shown in FIG.

The electronic computer 6 comprises a first computer module 32, a second computer module 34 and a third computer module 36 which implement one or more computer programs.

The first computer module 32 is a touch screen operating system, configured to manage a multi-touch interface and provide a set of contact events that serve to define the contact points of each path. Indeed, when a user touches the touch surface 18, contact events are produced by the operating system 32. These events are of various nature depending on the action made by the finger, and are for example:

.- an event called START KEY, created when a finger is placed on the screen,

.- an event called TOUCH END, created when a finger is lifted from the screen, and

.- an event called TOUCH MOVE, created when a finger moves on the screen.

For each event, a certain number of parameters are also received, such as the x / y position of the finger on the screen which is calculated by the first module 32 as the averaged position on all the affected pixels.

These events are then made available to the third computer module 36 which serves as a controller of a virtual camera, here a virtual camera, and will be used to prescribe motion instructions to the camera. Whatever the operating system used, including Windows®, iOs®, Android®, Linux®, touch events as objects will be produced and can be exploited.

The first computer module 32 is also configured to manage a graphical display interface and to display on a visible surface of the card touch screen from processed map data provided by the second computer module 34.

The second computer module 34 is a cartographic engine, equipped with a virtual camera and configured to calculate in the form of maps the views received by the virtual camera from different points of view of the camera and actual cartographic raw data on a surface globally spherical previously observed and recorded in the electronic memory. A point of view of the virtual camera is characterized by a geometric position of the camera in a spherical coordinate system common to the camera and the observed three-dimensional surface and an attitude of the camera in a suitable local coordinate system, centered on the virtual camera.

The virtual camera comprises in a conventional manner, like a physical camera, a retina with an optical center C and an optical axis. The camera is geometrically marked by a current position and a current attitude. The current position of the virtual camera is defined in a spherical coordinate system of center O by a longitude, a latitude and an altitude of the optical center C. The current attitude of the virtual camera is defined in a local coordinate system by a first current azimuth angle y and a second current inclination angle x.

The first current azimuthal angle is an angled angle separating an azimuthal reference axis and the projection of the optical axis in a current orientation on an azimuthal projection plane Paz, starting from the azimuthal reference axis. The azimuthal projection plane contains the optical center C of the camera and it is tangent in C to the sphere of center O whose radius is the distance between A and C. The azimuthal reference axis is contained in the same azimuth projection plane Paz and its orientation direction depends on the current position of the camera.

The second current angle of inclination x is an oriented angle separating the optical axis of the retina in the current orientation and the axis joining the optical axis C and the center O of the spherical coordinate system starting from the current optical axis . For example, the azimuthal reference axis is the projection on the azimuthal projection plane of an axis joining the optical center and a fixed point located on a sphere of center O and of radius less than the altitude of the camera.

Such camera tracking systems are used when three-dimensional mapping concerns a planet or one of its satellites, in particular the terrestrial globe. When the three-dimensional maps are maps of the Earth, the spherical geometric reference is a geocentric landmark rotating with the Earth, and the fixed point for defining the azimuthal reference axis is preferably a point equal to or adjacent to the geographic north. In particular, this reference point may be the magnetic north of the Earth and the azimuthal reference axis will then be the magnetic north axis provided in particular by a gyrocompass.

The third computer module 36 is a camera controller configured for, from a series of two-finger tactile events provided by the first computer module 32, to determine in a first step of recognizing a gesture a first tactile displacement Dy common sliding of the first and second fingers, arranged side by side without departing, in a first predetermined direction Diry on the surface of the touch screen between an early movement and an end of the movement of the two fingers.

The third computer module 36 is also configured to determine in a second control step, a first command Cy modification of a first azimuth angle current from the point of view of the camera according to the first tactile displacement Dy.

In particular, the third computer module 36 is configured, when it implements the first step of recognizing a gesture, to detect in a third step the beginning of the sliding movement of two fingers simultaneously in contact on the surface of the touch screen, then in a fourth step and from the beginning of the first and second trajectories of the contact points, determine whether the sliding of the two fingers is carried out the two fingers side by side without departing, and if so , in a fifth step, determining the first tactile displacement Dy according to the first predetermined direction Diry of the surface of the touch screen is determined between the beginning of the movement and an end of the movement of the two fingers.

The map engine 34 is configured to, from the first command Cy it receives from the camera controller and from a current azimuth angle yc of attitude of the virtual camera, rotate the current viewpoint of the virtual camera. a first azimuthal angle Ay controlled, function of the first command Cy.

Alternatively and according to Figure 2, the device 2 forming a tablet of Figure 1, wherein the touch screen, the camera controller, and the virtual camera are physically integrated in the tablet 2, is replaced by a system 52 comprising on the one hand a tablet 54 integrating the touch screen 4 with the operating system 32 and a real camera controller identical or substantially identical to the controller 36 of Figure 1, and on the other hand a device actual shooting, here a real physical camera 56 with an actuator of his attitude. The actual camera 56 and the tablet 54 are interconnected by a communication cable 58 through which the tablet 54 provides attitude control data to the actual camera 56 made by the camera controller 36, and the actual camera 56. provides the tablet 54 image data corresponding to the prescribed azimuthal attitude angle, or the prescribed angle of inclination and azimuthal angle.

Alternatively, a plurality of electronic computers are configured to perform the functional steps of the camera controller and / or the map engine.

Alternatively, the touch interface manager, the camera controller and the map engine are each different dedicated electronic circuits.

Thus, the point of view of the virtual or real camera is changed quickly using an ergonomic gesture, simple and effective, limited to a single type of movement and two fingers. The operator places his two fingers side by side on the touch surface, and moves them in a desired direction in the plane of the tactile surface decomposable into a first directional component with respect to a first fixed direction and a second directional component with respect to a second fixed direction, different from the first direction. The first component, by for example, a horizontal displacement on the touch surface corresponds to a setting of the orientation of the azimuthal angle of the camera while the second component, for example a vertical displacement on the touch surface corresponds to a tilt adjustment of the camera.

According to FIG. 3 and a first general mode of the invention, a method 202 of changing from a point of view of a three-dimensional (3D) map or of an image of a three-dimensional (3D) object by recognition of a gesture on a multi-touch screen, the three-dimensional map being provided by a map engine through a virtual camera, or the image of the object being provided by a real camera, is implemented for example by the device or the system of Figures 1 and 2, and comprises a first step 204 of recognition of a gesture and a second step 206 control.

Through the tracking of a first trajectory and a second trajectory of contact points on the touch screen respectively corresponding to the contacts of a first finger and the contacts of a second finger, in the first step 204 of recognition of a gesture, it is determined a first tactile displacement Dy common sliding of the first and second fingers, arranged side by side without departing, in a first predetermined direction Diry on the surface of the touch screen between an early movement and an end of the movement of the two fingers.

In the second control step 206, a first control Cy of modifying a first current azimuthal angle from the point of view of the camera is determined according to the first tactile movement Dy.

The first gesture recognition step 204 includes a third step 208, a fourth step 210, and a fifth step 212.

In the third step 208, the beginning of the sliding movement of two fingers simultaneously in contact on the surface of the touch screen is detected. Then, in the fourth step 210, from the beginning of the first and second trajectories of the contact points, it is determined whether the sliding of the two fingers is carried out the two fingers side by side without departing. If yes, the fifth step 212 is executed in which the first Dy touch displacement according to the first predetermined Diry direction of the surface of the touch screen is determined between the beginning of the movement and an end of the movement of the two fingers.

The second control step 206 is followed by a sixth step 214 during which, from the first command Cy, received by the cartographic engine or the real camera, and from a current azimuth angle or current of attitude of the camera, the virtual camera of the cartographic engine or the real camera rotates the current point of view of a first azimuth angle Ay controlled, function from there first order Cy.

According to Figure 4, a second embodiment of the method 252, derived from the first embodiment, comprises the same steps as those described in Figure 3 and new steps 254, 256, 262 and 264.

The new first step 254 is the first step 204 of recognition of a gesture, in which it is also determined a second tactile displacement Dx common slip of the first and second fingers, arranged side by side without departing, in a second direction Dirx predetermined on the surface of the touch screen between the beginning of the movement and the end of the movement of the two fingers.

The new fifth step 262 is the fifth step 212 in which the second tactile displacement Dx following the second predetermined direction Dirx of the surface of the touch screen is determined between the beginning of the movement and an end of the movement of the two fingers in addition to the first moving Dy.

The new second control step 256 is the second control step 206 in which a second command Cx for modifying a second tilt angle which is current from the point of view of the camera is further determined according to of the second Dx touch shift.

The new sixth step 264 is the sixth step 214 during which additionally, from the second command CX, received by the map engine or the real camera, and a second angle of inclination current x attitude of the camera, the virtual camera of the cartographic engine or the real-world camera pivots a current point of view of a second inclination angle Δχ controlled, a function of the second command Cx.

It should be noted that the determinations of the first and second tactile displacements Dy, Dx can be implemented in series or in parallel, and / or the execution steps of the first and second commands Cy, Cx can be executed in series or in series. parallel.

In all the embodiments of the method described above the first step is preceded by an initialization step during which a reliable starting state of the two fingers is detected, and a starting position of each finger as well as the distance separating the two fingers and the central position between the two fingers are determined, the starting position of a finger being an average position of the positions received from said finger during an initial observation time window.

Optionally, during the third step the beginning of a sliding movement of the two fingers is detected if the distance traveled by the central position of the two fingers is greater than a first threshold value ε during a first time window of the beginning of observation of trajectories.

Optionally, during the fourth step and during a first time window of start of observation of the trajectories a scalar product between two normalized vectors of the displacements of the two fingers is calculated, then compared to a second positive threshold value, preferably between 0.75 and 1.

Optionally, the second step comprises a seventh step, implemented when the scalar product between the two normalized vectors of the displacements of the two fingers is greater than the second threshold value, in which it is verified that the standards of the movements of the two fingers are of the same order of magnitude and decide to change at least the first angle of inclination of the camera.

These options are useful when the two-fingers gesture of the invention intended to modify the angles of point of view of the camera must be distinguished from other gestures with two unwanted fingers or gesture desired two fingers of the "pinch to zoom As part of the point of view change process. According to Fig. 5, a two-finger sliding movement gesture arranged side by side is shown in which the displacement takes place along a first direction Diry on the surface of the touch screen, fixed here horizontally. The system and method of the invention according to Figs. 1 to 4 are configured to change the azimuthal angle of the camera according to a Dy motion oriented along the Diry direction in Fig. 5.

According to FIGS. 6A, 6B and 6C, the effect of the gesture of FIG. 5 following a first collective shift to the right Dy of the two slipped fingers and through the method of the invention is represented.

According to the upper part of FIG. 6A, a current or current starting azimuth attitude of the camera is represented in its azimuthal plane by a current azimuth departure angle of zero value between a current azimuth component of the axis Optical 270 and the north direction of the Earth's magnetic north axis, here forming azimuthal angular reference axis 272 in the azimuthal plane Paz. The azimuthal plane Paz of the camera is the plane tangent to the camera and the sphere having as center, the center O of the Earth and as ray the distance separating the optical center C of the camera and the center O of the earth. The camera and more precisely its optical center C are here supposed to be located in a current or current position nadir of a point of the terrestrial equator at a current or current altitude. The optical axis of the camera in the starting or current position of the camera is represented here in its azimuthal component 270 in the azimuthal plane of the camera which is confused with the plane of the upper part of Figure 6A. The azimuthal component 270 of the optical axis, that is to say its projection in the azimuthal plane Paz, is here merged with the azimuthal reference axis 272. According to the lower part of FIG. 6A, a partial view taken by the camera is represented, here a map of France following the South-North direction.

According to FIG. 6B, the first sliding movement Dy of the ends of the two fingers of the right hand, index and middle finger, is represented by an arrow 274. The starting position of the two fingers and the arrival position of the two fingers is complete. are respectively represented using a hand in dotted lines on the left in the figure and a hand in unbroken lines on the right in the figure.

According to the upper part of FIG. 6C, a controlled azimuthal angular displacement y of the starting azimuthal attitude of the camera, that is to say a rotation of the current azimuthal projection of the optical axis 270 corresponding to the first displacement Dy is represented in the same azimuthal plane Paz and with respect to the same azimuthal reference axis 272 as those described in FIG. 6A. According to the lower part of FIG. 6B, a partial view of the camera in the modified azimuthal viewpoint of the camera is represented, here the map of France whose South axis is North turned, the tilt angle of the camera was not changed.

According to Figure 7, a slid motion gesture of two fingers arranged side by side is shown in which the displacement takes place along a second direction Dirx surface of the touch screen, different from the first direction Diry, and fixed here vertical. The system and method of the invention according to FIGS. 1, 2 and 4 are configured to change the angle of inclination of the camera according to a displacement Dx oriented along the direction Dirx on Figure 7, in addition to a possible modification of the azimuthal angle as described above.

According to FIGS. 8A, 8B and 8C, the effect of the gesture of FIG. 7 following a second collective vertical displacement Dx of the two slipped fingers and through the method of the invention is represented.

According to the upper part of FIG. 8A, a current or current starting attitude of the camera is represented by a current starting angle of inclination of zero value between the optical axis 280 of the camera in its current starting orientation and Nadir axis 282 passing through the optical center C of the camera towards the center O of the Earth and forming the reference axis of the angle of inclination. Following the lower part of Figure 8A, a partial view of the camera is shown, here a map of France in the South-North direction which suggests that the current or current azimuth angle of the camera is Nil by taking the same definition of the azimuthal angle as in that in Figure 6A. According to FIG. 8B, the second sliding displacement Dx of the ends of the two fingers of the right hand, index and major, is represented by a vertical arrow 284 directed downwards in the figure. The starting position of the two fingers and the position of arrival of the two fingers the finished slip are respectively represented with the help of a hand in dotted lines at the top in the Figure and a hand in unbroken lines down on the Fig.

According to the upper part of FIG. 8C, there is shown a second angular displacement of controlled tilt x of the camera, that is to say a rotation of the current optical axis 280 corresponding to the second displacement Dx around the normal 286 in C of a tilting plane Pincl containing the optical center C of the camera, the current azimuth projection 288 of the optical axis in the current azimuthal plane and the axis d reference inclination 282.

According to the lower part of FIG. 8C, a partial view of the camera in the modified inclination point of view of the camera is represented, here the map of France, the current azimuthal angle of the camera remaining unchanged.

According to Fig. 9, a two-finger sliding movement gesture arranged side by side is shown in which the displacement on the tactile surface takes place along any direction 290 decomposable into a first displacement component Dy along the first direction Diry , fixed here horizontal and a second displacement component Dx along the second direction Dirx, here fixed vertical. The system and method of the invention according to Figures 1, 2 and 4 are configured to change the azimuthal angle y and the tilt angle x of the camera according to the first component of displacement Dy and the second displacement component Dx.

According to FIG. 10, a complete method 302 for modifying the three-dimensional point of view of an apparatus for shooting with a complete and simple gesture of manipulation of geographical maps or images of a three-dimensional object is configured to use a gesture with two fingers to modify the angles of the three-dimensional point of view of the camera as described above and to exploit a set of other distinct gestures responding to different uses of card manipulation, such as zooming, or moving the card. The gesture of modification of the angles of three-dimensional point of view of the camera must thus be complementary to other gestures of manipulation of the card or the image, so that the whole of the complete gesture forms a coherent whole. and easy to use.

According to FIG. 10 and the complete method 302 of handling from the point of view of three-dimensional maps or images of a three-dimensional object is realized using and from three different basic gestures, corresponding to three settings of the shooting device displaying the map or image. When the user interacts with the touch surface, one of the following three modes must be enabled:

a first mode 304 called "MODE MOVE", sliding the position of the camera which is made with a single finger sliding on the screen;

a second mode 306 called "ZOOM MODE" to zoom in or out on the map that is made with two slid fingers that deviate on the touch surface, this action being referred to in English as "pinch and zoom"; and

a third mode 308, called "3D MODE" of changing the angles of view of the camera (azimuthal angle and angle of inclination), which is made with two slid fingers that move in the same direction horizontally and / or vertically and which constitutes the main object of the invention described above.

The choice of the mode in which we are, and therefore of the action that is produced, is done according to the number of fingers placed, and the displacement actions carried out through the tests 312, 314. Once one of these three modes 304, 306, 308 has been identified, the actions of displacement of one or two fingers cause a change in the point of view of the camera in accordance with the mode in which one is currently. The current mode output is only output when the fingers leave the touch surface.

A tactile movement is a user interaction performed with one or more fingers that can be broken down into three distinct phases: - a first phase during which the user presses the screen (PRESS);

a second phase during which the user performs a movement (MOVE);

- a third phase during which the user stops pressing (RELEASE).

The first PRESS phase and the second RELEASE phase are composed of a single event. Conversely, the second MOVE phase can occur as many times as the user moves his finger. In practice, the second MOVE phase can occur if the user involuntarily moves the finger over a very short distance, which can be problematic in the detection of certain events.

If the user uses two fingers, two modes of action are possible: the second mode 306 (ZOOM MODE) and the third mode 308 (3D MODE).

In the second mode 306, the zoom is made by an opposite movement of the two fingers, or by the movement of a finger while the second remains fixed.

In the third mode 308, the azimuthal rotation and the inclination of the camera are triggered by the parallel movement of the two fingers in a direction of movement, the displacement being decomposable into a first horizontal displacement which rotates in azimuth rotation of the camera and a second vertical movement that changes the tilt of the camera.

The modes must be able to be identified from each other quickly, so that it is imperceptible to the user.

In accordance with FIG. 11, the complete method of changing the point of view of the camera which recognizes the gesture to the three modes described in FIG. 10 is described in more detail according to a particular embodiment.

To detect one of the two modes among the second and third modes, ZOOM MODE and 3D MODE, the detection of these two-finger modes and their distinction 314 is done in an initialization step 318 followed by a decision step 320 . In the initialization step 318 to obtain a reliable starting point, the events received during X milliseconds (X being for example equal to 100 ms) are analyzed and averaged all the positions received from each of the two fingers. One of the main difficulties of tactile controls is the relative inaccuracy of the information received, a finger touching several tens or even hundreds of pixels. This is to know the real center of contact. The first events received are therefore not directly processed but an average on the first events is calculated to obtain a starting position of each finger, as well as the distance separating the two fingers and the central position between the two fingers.

In the decision step 320, an attempt to detect an azimuthal rotation and / or inclination gesture (corresponding to the 3D mode) is implemented for Y milliseconds, Y being for example equal to 200 ms. If the detection is successful, the transition to the third mode 308, 3D MODE activated, is performed. If the detection is unsuccessful, the passage in the second mode 306, MODE ZOOM activated, is performed.

The values of the duration X and Y of the windows of initialization and decision can be between 0 and 500 ms according to the uses of the users and the reactivity of desired action which is characterized by a latency time before starting. The longer the latency, the more accurate the processing is, but in return the more the initialization effect is noticeable.

In the decision step 320, to know which mode from the second mode and the third mode is desired, the two vectors corresponding to the displacement of the two fingers are calculated as well as the scalar product of these two vectors. A positive scalar product indicates that the fingers move in the same direction and that the third mode, 3D MODE, must be active while a negative dot product indicates that the two fingers are moving in opposite directions and that the second mode, MODE ZOOM, must be enabled.

However, it is not possible to make this calculation on the first two tactile events received because it is possible that the desired action is the activation of the 3D MODE, but that the first events give a spread information of the fingers. This happens because the first supports are generally imprecise and do not necessarily correspond to the action desired by the user (it is called "false positive"). A delay of the first actions and a smoothing of the calculations in the time are thus required.

The decision to activate the mode of change of the angles of point of view of the camera (3D MODE) is realized in the following way.

Each received event is "smoothed", that is to say, weighted by the initial event, in order to avoid false-positives then the spacing of the fingers is deduced, the center of the movement, and the displacement of each finger.

To verify that we are in the presence of a 3D MODE to activate:

first, it is verified that the distance between the two fingers has remained proportional between the beginning of the movement and the event received; then

the scalar product between the normalized vectors of the two fingers is calculated, the detection 324 of a parallel movement in the same direction requiring that the dot product be positive, and even greater than 0.75 (in practice when the dot product is between 1 and 0.75, the movement of the two fingers can be considered as parallel and in the same direction); then

It is also verified that the vectors of each finger are of the same order of magnitude.

Similarly, not to trigger a decision too fast passage in the second mode (ZOOM MODE) or the third mode (3D MODE), is initially accepted 328 the activation of a mode that if the distance traveled by each of the two fingers is sufficient, greater than a minimum value of a threshold ε. This condition is not applied on the latest events because if the user performs a slow motion, this condition may never be validated.

If all the conditions required in the decision step 320 are met, then the transition to 3D MODE is performed, otherwise, at the end of the decision step, the ZOOM MODE is retained.

When the third mode 308, 3D MODE has been activated, in this mode a first displacement Dy of the two fingers slipped in a first direction fixed here horizontal and / or a second displacement Dx of the two fingers slid in a second direction, fixed here vertical are determined.

An azimuthal rotation from the camera's point of view is performed 332 as a function of the first horizontal displacement, for example a linear variation in which a movement dragged one pixel to the right creates a 0.25 degree rotation of the camera.

An inclination from the camera point of view is made 332 as a function of the second vertical displacement, for example a linear variation in which a dragged movement of a pixel downwards changes the 0.25 degree inclination of the camera. views.

The method 302 described above makes it possible to change the point of view of a camera in all its forms by using simple and rapid gestures involving at most two fingers, and to change in particular the azimuthal rotation angles. and / or tilt point of view in single fast and simple mode to run. The control of the orientation of the camera is simple and intuitive using the gestures offered on the touch screen.

A product or computer program comprises a set of instructions, configured to implement the steps of a three-dimensional view change method according to the invention described above when loaded and executed by one or more several electronic calculators of a device or a system of three-dimensional point of view change.

It is thus possible to imagine a control of a "physical" camera using a tactile surface, in place of the joystick that is commonly used today, for example, for applications video surveillance with stationary cameras in the context of police, security, or with a camera embedded in a helicopter or maritime patrol aircraft.

Touch media is not limited to a screen or tablet. The same gesture can be applied to any type of tactile surface such as a "multitouch touchpad", for example that of a laptop, or a touchpad that would replace a mouse on a computer, or an interactive movie "multi-touch"". This invention can be extended to the handling of any type of camera by touch interface (screen, but also touchpad)

The main application of the invention is to adjust the point of view of a virtual camera in a 3D map application hands it can be extended to the manipulation of real-world camera such as physical cameras in the context of surveillance activities, for example.

A physical or material shooting apparatus is for example a video camera, a camera, or any radiation image sensor whose orientation can be controlled. More particularly, the camera may be an onboard video camera on board an aircraft visualizing the earth's surface.

Claims

1. A method of changing a point of view of a three-dimensional (3D) map or an image of a three-dimensional object (3D) by recognizing a gesture on a multi-touch screen (4), the three-dimensional map or the image of the three-dimensional object (3D) being provided by a cartographic engine through a virtual camera or by a real camera,

The method of changing a point of view comprising steps of tracking a first trajectory (14) and a second trajectory (1 6) of contact points (20, 22; 24, 26). ) on the touch screen (4) respectively corresponding to the contacts of a first finger and the contacts of a second finger,

. in a first step (204; 254) of recognition of a gesture, it is determined a first tactile displacement Dy common slip of the first and second fingers, arranged side by side without departing, in a first predetermined direction Diry on the touch screen surface between a beginning of the movement and an end of the movement of the two fingers, and

. in a second control step (206; 256), a first change command Cy of a first current angle from the point of view of the camera is determined according to the first tactile movement Dy;

the method being characterized in that

the virtual or real-life camera comprises a retina with an optical center C and an optical axis, and is geometrically marked by a current position and a current attitude, the current position being defined in a spherical coordinate system of center O by a longitude, a latitude and an altitude of the optical center C, and the current attitude being defined in a local coordinate system by a first current azimuthal angle y and a second current angle of inclination x,

the current first azimuth angle being an angled angle separating the projection from the optical axis in a current orientation on an azimuth projection plane containing the optical center C, and tangent to the sphere of center O and containing the optical center C, and an azimuthal reference axis contained in the same azimuthal projection plane and whose direction depends on the current position of the camera, starting from the azimuthal reference axis, and

the second current angle of inclination x being an oriented angle separating the optical axis of the retina in the current orientation and the axis joining the optical axis C and the center O of the spherical coordinate system starting from the current optical axis ; and .- the first current angle y from the point of view of the camera modified by the command determined in the second step (206; 256) is said first current azimuthal angle y.

The method of changing the point of view of claim 1 wherein

the first step (204; 254) of gesture recognition comprises. a third step (208) during which the beginning of the sliding movement of two fingers simultaneously in contact on the surface of the touch screen is detected, followed

. a fourth step (210) during which, from the beginning of the first and second trajectories of the contact points, it is determined whether the sliding of the two fingers is carried out the two fingers side by side without departing, followed in the affirmative

. a fifth step (212; 262) during which the first tactile displacement Dy in the first predetermined direction Diry of the surface of the touch screen is determined between the beginning of the movement and an end of the movement of the two fingers, and

the second control step (206; 256) is followed by a sixth step (214; 264) in which, from the first command Cy, received by the map engine or the real-world camera, and a current azimuthal angle Yc of attitude of the real or virtual camera, the virtual camera of the cartographic engine or the real camera rotates a current point of view of a first azimuthal angle Ay controlled, function of the first command Cy.

A method of changing a point of view according to any one of claims 1 to 2 wherein

in the first step (254) of recognizing a gesture, it is also determined a second tactile displacement Dx common of sliding the first and second fingers, arranged side by side without departing, in a second predetermined direction Dirx on the surface of the touch screen between the beginning of the movement and the end of the movement of the two fingers, and

in the second control step (256), a second command Cx for modifying a second current angle of inclination from the point of view of the camera is further determined as a function of the second tactile displacement Dx.

4. A method of changing point of view according to claims 2 and 3 wherein

the second tactile displacement Dx in the second predetermined direction Dirx of the surface of the touch screen between the beginning of the movement and the end of the movement of the two fingers is determined in the fifth step (262), and

in the sixth step (264), from the second command Cx, received by the map engine or the actual camera, and a second current tilt angle x of the attitude of the take-up device the virtual camera of the cartographic engine or the real-world camera pivots a current point of view of a second angle of inclination Δχ controlled, a function of the second command Cx.

5. A method of changing the point of view according to any one of claims 1 to 4 wherein the azimuthal reference axis is the projection on the azimuthal projection plane of an axis joining the optical center and a fixed point located on a sphere of center O and radius less than the altitude of the camera.

6. A method of changing point of view according to claim 5, wherein the three-dimensional maps are maps of the terrestrial globe, the spherical geometric reference is a geocentric landmark rotating with the Earth, and the fixed point is a point equal to or adjacent to from the geographical north.

7. A method of changing point of view according to claim 6 wherein the fixed point is the magnetic north and the azimuthal reference axis is the magnetic north axis provided in particular by a gyrocompass.

A viewpoint change method according to any one of claims 1 to 7 wherein the provided three-dimensional maps are views computed by the map engine from a database when the camera is virtual, and the images of the three-dimensional object are real images captured by the camera when the camera is real.

9. A method of changing point of view according to claim 3 and any one of claims 4 to 8 wherein

the determinations of the first and second tactile displacements Dy, Dx are implemented in series or in parallel, and / or the execution steps of the first and second commands Cy, Cx are executed in series or in parallel.

A method of changing a viewpoint according to any of claims 1 to 9, wherein the first step is preceded by an initialization step (318) in which a reliable starting state of the two fingers is detected, and a starting position of each finger as well as the distance separating the two fingers and the central position between the two fingers are determined, the starting position of a finger being an average position of the positions received from said finger during a time window initial observation.

1 1. A viewpoint change method according to any one of claims 2 to 10, wherein

during the third step (208; 328) the beginning of a sliding movement of the two fingers is detected if the distance traveled by the central position of the two fingers is greater than a first threshold value ε during a first start time window trajectory observation, and

during the fourth step (210) and during a first time window of start of observation of the trajectories a dot product between two normalized vectors of the displacements of the two fingers is calculated, then compared (324) with a positive threshold value, preferably between 0.75 and 1.

12. A method of changing the point of view of claim 1 1 wherein the second step (206; 256) comprises a step, implemented when the dot product between the two normalized vectors of the movements of the two fingers is greater than the second. threshold value, in which it is verified (326) that the standards of the movements of the two fingers are of the same order of magnitude and decide to modify at least the first angle of inclination of the camera.

13. Device or system for changing from a point of view of a three-dimensional (3D) map by recognizing a gesture on a multi-touch screen, comprising

a real-life camera (56) or a map engine (34) with a virtual camera for providing a three-dimensional image or map of a scene of a three-dimensional object from a current viewpoint of the real or virtual camera;

a multi-touch screen (4) for following a first trajectory (14) and a second trajectory (1 6) of contact points (20, 22; 24, 26) on the touch screen (4) respectively corresponding to the contacts of a first finger and the contacts of a second finger, in the form of tactile events;

one or more electronic computers (6), configured for the tactile events of the two fingers,

. determining in a first step of recognizing a gesture a first tactile displacement Dy common sliding of the first and second fingers, arranged side by side without departing, in a first predetermined direction Diry on the surface of the touch screen between a beginning of the movement and an end of the movement of the two fingers, and

. determining in a second control step, a first command Cy modifying a first current azimuthal angle from the point of view of the camera according to the first tactile displacement Dy,

The device or system for changing point of view being characterized in that

the current first azimuth angle being an angled angle separating the projection from the optical axis in a current orientation on an azimuth projection plane containing the optical center C, and tangent to the sphere of center O and containing the optical center C, and an azimuthal reference axis contained in the same azimuthal projection plane and whose direction depends on the current position of the camera, starting from the azimuthal reference axis,

the second current angle of inclination x being an oriented angle separating the optical axis of the retina in the current orientation and the axis joining the optical axis C and the center O of the spherical coordinate system starting from the current optical axis ; and in that

the first current angle y from the point of view of the camera modified by the command determined in the second step (206; 256) is said first current azimuthal angle y.

A computer product or program comprising a set of instructions configured to implement the method defined in any one of claims 1 to 12 when loaded into and executed by one or more computers (6) of the device. or system (2; 52) defined in accordance with claim 13.