ES2543038B2 - Spatial location method and system using light markers for any environment - Google Patents

Abstract

Method and spatial location system of a lens (10) in a three-dimensional environment (11) comprising at least one luminous marker comprising: # - a stereo camera (12) to capture a first image frame at a current time and a second image frame at an earlier time; # - an angle measuring device (13) to obtain a rotation angle of the lens (10); # - a signal processor (14) with access to a memory (15) that stores among others a radius of the at least one marker detected in an instant of current time n and in an instant of previous time n-1 configured to calculate coordinates (x {sub, i}, and {sub, i}) of the target (10) in an instant of time i as follows: # - if the angle of rotation at the current time instant and at the previous time instant are different, (x {sub, n}, and {sub, n}) = (x {sub, n-1}, and {sub, n-1); # - if the two image frames are equal, (x {sub, n}, and {sub, n}) = (x { sub, n-1}, and {sub, n-1}); # - otherwise: # - if the radios are equal and there are several markers, (x {sub, n}, and {sub, n}) are calculated by triangulation using both image frames; # - if the radios are different and there are several markers, (x {sub, n}, and {sub, n}) are calculated by triangulation using a single image frame; # - if the radii are different and there is only one marker, (x {sub, n}, and {sub, n}) are calculated using stereo geometry; # - if the radii are equal and there is only one marker, (x {sub, n}, and {sub, n}) are calculated using image coordinates of the marker at the current and previous times.

Description

Spatial Location Method and System using Luminous Markers for any environment

OBJECT OF THE INVENTION

The present invention belongs to the fields of electronics and telecommunications. Specifically, the present invention applies to the industrial area that includes the techniques of detection of reference points for the location and positioning of an objective (a

person, animal or object) in controlled environments.

More particularly, the present invention relates to a method and system for obtaining, from the use of light markers, the position and orientation of an object or subject, applicable to any type of environment, interior or exterior.

BACKGROUND OF THE INVENTION

In recent years, there is a growing interest in those systems or products related to the location of objects in three dimensions (3D). The sectors that cover this technology are very wide, such as robotics, medicine or video games, among others.

Specifically, object tracking is closely related to augmented reality, where knowledge of the individual's position is essential. This technology mixes virtual elements with real images, allowing the user to expand their information from the real world or interact with it. Virtual reality, however, replaces physical reality with computer data. Augmented reality systems can use as transparent display devices Cdisplays' in English) a transparent optical screen (for example, Google Glass) or an image mixing screen (for example, that of a smart mobile phone, 'smartphone' in English ). In addition to knowing the position of the individual accurately, they can be based on the use of cameras, optical sensors, accelerometers, gyroscopes, GPS, etc. In the case that vision systems are used, it is necessary to perform a preprocessing of the region of interest where the individual is located, using image algorithms that allow to detect corners, edges or reference markers; for later, with this data obtain the real 3D coordinates of the environment. These systems require the use of a CPU and RAM with sufficient capacity to

computation, to be able to process the images of the cameras in real time and with the lowest possible latency.

When only one reference marker can be displayed, the use of stereo cameras and epipolar geometry must be used. The characterization of a point in three-dimensional space requires knowledge of its coordinates (x, y, z) within the environment where it is located, with respect to a reference position. The most common technique is based on the use of two or more calibrated cameras, which provide a left and right image of the same scene. To obtain the 3D coordinates of the point or object to be characterized, stereo correspondences are applied (look for the same point in both images) and the projective or epipolar geometry is calculated (it describes the relationship between the image planes of the cameras and the point ).

In the case of having more than one marker or pattern available, other techniques can be applied to locate the object on the stage. Through triangulation, knowing the real distance between markers, it is possible with only two markers and a single camera to obtain the parameters to achieve depth to the target and position it in the environment. This practice simplifies the computational cost, by not having to analyze two images and their correspondences; but it requires greater precision when it comes to detecting markers. Despite this, some authors ("Qptical tracking using projective invariant marker pattern properties", R. van Uere et al., Proceedings of the IEEE Virtual Reality Conference 2003, 2003) consider that, in order to better track the object, it is necessary that the object has four or more markers and that stereo vision is also used; to get a more precise but slower system.

One of the problems that have been found in other studies (US 7,231,063 82, "Fudicial Detection System", L Naimark et al.) When using markers, is the brightness of the environment where the images are to be taken. The points to be detected may be lost in the scene due to lack of light. This limits the applications that use this system indoors or with controlled brightness. In addition, the use of contrast enhancement algorithms and / or the use of specific markers stored in a database is necessary, which substantially increases the computation time of these systems. In addition to considerably limiting the distance between the printed markers and the system user, unless their size is large enough for the image sensor to capture. In some cases (WO 2013/120041 A 1, "Method and apparatus for 3D spatial localization and tracking of objects using active optical illumination and sensing") these types of light sources with variable luminance or pulsed light have been proposed, which may cause synchronization failures. Even so, the use of light markers can pose problems, specifically in environments where there are light sources with a luminance much greater than the marker itself (in the worst case, sunlight) or sources that emit radiation in the same direction; In these situations, the image sensor is not able to differentiate one light source from another, so it will force, as previously, to use this technology in bright environments without large sources of light in it.

The idea of detecting and positioning the markers serves to characterize the objects or individuals that are in it, that way they are located in space. In the article "Wide area optical tracking in unconstrained indoor environments" (by A. Mossel et al., 23rd International Conference on Artificial Reality and Telexistence (ICAT), 2013) it is proposed to incorporate infrared light markers in a tape located on the head of the Username. To do this, they place on the stage two independent cameras, which require a synchronization process to perform the shot simultaneously, located at a distance equal to the length of the wall of the room where it is to be tested. The algorithm used to estimate the position is based on the search for stereo correspondences. One of the disadvantages that it presents is that it cannot be implemented for augmented or simulated reality systems, because the cameras do not show what the user sees, in addition to being restricted to indoor environments with limited dimensions.

Other studies such as "Tracking of user position and orientation by stereo measurement of infrared markers and orientation sensing" (by M. Maeda, et aL, Proceeding of the 8th. International Symposium on Wearable Computers (ISWC'04), 2004) raise the use of infrared markers located on the wall of a room, to locate the user. Specifically, they propose the use of two types of markers: active and passive. The active markers are formed by a set of three infrared LEOs and a signal emitter, which sends data of their real position to a signal decoder that the user carries, so once they are detected they know their absolute position. Passive markers are only an infrared light source, from which they obtain the relative position of the user. In addition to relying on the reception of signals from active markers, they calculate the relative distance to the marker from stereo vision. The use of this technique, as in 105 cases explained above, is limited to interior spaces.

There are other methods, which do not require direct vision of one or more cameras with reference markers, for user tracking and tracking systems. Radio frequency techniques consist of measuring distances, of static or mobile objects, from the emission of electromagnetic pulses that are reflected in a receiver. These electromagnetic waves will be reflected when there are significant changes in the atomic density between the environment and the object, so they work particularly well in cases of conductive materials (metals). They are able to detect objects at a greater distance than other systems based on light or sound, however they are quite sensitive to interference or noise. It is also difficult to measure objects that are between each other at different distances to the emitter, because the pulse frequency will vary (slower the farther and vice versa). Even so, there are experimental studies such as uRADAR: an inbuilding RF-based user location and tracking system "(by P. Bahl et al., Proceedings of IEEE INFOCOM 2000, Tel-Aviv, 2000) that demonstrate its use to estimate the location of the user with a high level of precision This technique is not appropriate in augmented reality applications.

Another example of existing solutions is the LlDAR systems, which calculate the distance over the time it takes for a light pulse to be reflected on an object or surface, using a device with a pulsed laser as a light emitter and a photodetector as a signal receiver reflected. The advantage of these systems is the precision they achieve over long distances (using lasers with a wavelength> 1000 nm) and the possibility of mapping large areas, by sweeping light pulses. Its drawbacks are the need to perform the analysis and processing of each point, as well as the difficulty of automatically reconstructing three-dimensional images.

The objective technical problem that arises is thus to provide a system for the detection of the position and orientation of an individual or object in any type of environment, interior or exterior, with whatever their lighting conditions.

DESCRIPTION OF THE INVENTION

The present invention serves to solve the aforementioned problems, solving the drawbacks presented by the solutions mentioned in the prior art, providing a system that, from the use of one or more reference light markers and a single stereo camera, allows spatially locate objects or individuals in a scenario under any environmental condition and with greater distances between the user and the marker. The system is mainly based on the use of luminous markers to calculate relative positions of the object / individual, a stereo camera to visualize those markers on the stage image and an electronic device for measuring angles, such as a gyroscope or electronic compass , to provide angles of rotation of the target user (object, person or animal).

The present invention makes it possible to detect reference luminous markers in any type of environment, regardless of the light sources that determine the environmental conditions.

One aspect of the invention relates to a method for positioning or locating an objective by using reference markers in any 3D environment that, from a first image frame at an instant of current time and a second image frame in a previous moment of time captured by a stereo camera, images in which at least one marker is detected, obtains the coordinates (xn, Yn) of the objective at the current time instant n, for which it performs the following steps:

-

obtain an angle of rotation of the target at the current time instant and at the previous time instant; -if the angle of rotation at the current time instant and the angle of rotation at the previous time instant are different, calculate the coordinates (xn, Yn) of the target at the current time n equaling them with the coordinates (Xn.1, Yn. 1) of the objective in the previous instant n-1; -if the first image frame and the second image frame are the same, calculate the coordinates (xn, Yn) of the target at the current instant equal to the coordinates (Xn_l, Yn-l) of the objective at the previous instant; -si no, in another case, it obtains the image coordinates of, at least one detected marker, and its radius, to compare the radii at the current instant of time n and at the previous instant of time n-1 and:

-

if the radii are equal and there is a plurality of markers, the coordinates (Xn. Yn) of the target at the present time are obtained by triangulation using the first image frame and the second image frame;

-

if the radii are different and there is also more than one marker, the coordinates (xn, Yn) of the target at the current moment are obtained by triangulation but using a single image frame, the one captured at the current moment;

-

if the radii are different and there is a single marker detected, the coordinates (xn, Yn) of the target at the present time are obtained by the stereo geometry algorithm known in the state of the art;

-

if the radii are equal and there is a single marker detected, the coordinates (xn, Yn) of the target at the current time are obtained by an algorithm that reminds the stereo geometry but using the image coordinates of the marker at the current time instant and in the previous instant of time, instead of a left and right image of the same instant of time

Another aspect of the invention relates to a system for locating an objective, which can be an object or an individual, from at least one reference marker in a 3D space or environment, comprising the following means:

a stereo camera to capture image frames in which one or more markers are detected;

an angle measuring device to obtain the angle of rotation of the objective at each instant of time; a signal processor, with access to a storage device (a memory), configured to perform the steps of the method described above to obtain at its output the coordinates (Xn, Yn) of the target calculated at the current time instant, using, according to each case indicated above, the data obtained in the previous instant of time stored in the memory.

A light source, identifiable in the environment of use, is used as a reference marker.

In a possible field of application, the invention described can be used for Simulated Reality applications. For this, glasses are incorporated into the system

Virtual reality. Both the stereo camera and the glasses can be part of a helmet

or fastening equipment that is placed on the user's head and connecting the camera with the glasses. The system can additionally incorporate an accelerated meter, which measures the displacement made in a finite time, which would reduce the cumulative errors.

The present invention has a series of differentiating characteristics with respect to the existing solutions discussed in the prior art that have technical advantages such as the following:

With respect to US 7,231,063 82, the present invention solves the computation time problem of existing systems such as that described in US 7,231,063 82, because contrast enhancement algorithms and / or specific markers stored in a base are required of data, because in the present invention light markers are used that work in the visible or infrared spectrum, such as light emitting diodes (LEOs With respect to WO 2013/120041 A1, one of the differences of the present invention is that uses fixed light sources and comes to solve the problem that occurs in environments where there are light sources with a luminance much greater than the marker itself.To solve this problem, the present invention uses an element that prevents the light conditions of an environment significantly affect the use of a background after the light source.

BRIEF DESCRIPTION OF THE FIGURES

A series of drawings that help to better understand the invention and that expressly relate to an embodiment of said invention which is presented as a non-limiting example thereof is described very briefly below.

FIGURE 1.- Shows a schematic block diagram of the spatial location system of individuals or objects, according to a preferred embodiment of the invention.

FIGURE 2.- Shows a simplified representation of a type of luminous marker, which the system of Figure 1 can use.

•

FIGURE 3.- Shows an environment of use of the markers of Figure 2 and in which the system of Figure 1 is applicable, according to a possible embodiment.

FIGURES 4A-4B.-They show a scheme of the markers and parameters that the system uses to locate in the environment individuals or objects that move vertically and when only a single marker is detected.

FIGURES 5A-5B.-They show a scheme of the markers and parameters that the system uses to locate in the environment individuals or objects that move vertically and when more than one marker is detected.

FIGURE 6A.- Shows a scheme of the markers and parameters that the system uses to locate in the environment individuals or objects that move horizontally and when only a single marker is detected.

FIGURE 68. - Shows a scheme of the markers and parameters that the system uses to locate in the environment individuals or objects that move horizontally and when more than one marker is detected.

FIGURE 7.- It shows a scheme of the operation of the method, it is merely an example of data flow.

PREFERRED EMBODIMENT OF THE INVENTION

Next, possible embodiments of the obtaining system are proposed, from the use of one or more luminous markers, of the position and orientation of a user, in different possible environments, which can be indoors or outdoors, within a controlled scenario.

Figure 1 shows a schematic diagram of the block architecture of the system to locate in space the objects or individuals that constitute a target (10) in a three-dimensional environment (11) under any environmental condition defined by a number m ~ 1

of light sources (fLt, fL2, fL3, ..., fLm), having one or more light markers (20) such as those shown in Figures 2-3, 4A-4B, 5A-58 and 6A-6B. The system comprises a stereo camera (12) to detect the luminous markers (20) and an electronic angle measuring device (13), for example, a gyroscope or electronic compass, with which the rotation angles of the objective are obtained ( 10). In addition, the system comprises a digital signal processor (14) that calculates the positional coordinates in the space of each luminous marker (20) in time and stores them in a memory or storage device (15). The digital signal processor (14) uses the stored coordinates and the output parameters it obtains from the stereo camera (12) and from the angle measuring device (13) to determine at its output (16) the position of the target user (10 ).

Figure 2 shows a type of reference marker (20) of those used, which is a light marker and comprises two main elements: a light source (21) and a contrast surface (22). The preferred light source (21) is a LEO diode that emits in the visible range: 400-700 nm. This type of source is a point light source that achieves ranges greater than 50 m for powers greater than 1 W. In addition, a LEO diode can be considered as a non-hazardous product due to the optical powers in which it works since in the worst of the cases the exposure time is very low (aversion reaction time = -250 ms). Even so, the system can use luminous markers (20) with other types of light sources (21), because the device that detects the luminous marker (20), that is, the stereo camera (12) used as light receiver, detects both light sources (21) that work in the visible and infrared spectrum. The image sensor of the stereo camera (12) has a spectral curve which, for the wavelength of the LEO used, indicates a spectral response with a value greater than 0.005 ANV. Filament bulbs are another example of light sources (21), although they are diffuse sources with emitted optical powers lower than those attainable with a LEO diode. Another possible light source (21) can be a laser diode, although it is a collimated source capable of focusing the light on a very small point and, for most cases, all optical powers greater than 1 mW can be dangerous. The last type of light source (21) that can be used is an infrared LEO diode, although due to the scope it presents, the drawback is that the usual one is not able to perceive it and could cause eye damage. On the other hand, the contrast surface (22) is of a color - for example, black - and dimensions that allow distinguishing between the light marker (20) and any external light source. The contrast screen or surface (22) allows you to apply the

method proposed here in environments with little or a lot of light, and over long distances. The shape of the contrast surface (22) can be any, for example, square as in Figure 2. The dimensions of the contrast surface (22) depend on the surrounding light conditions (11), on the luminous flux of the light source (21) And the maximum distance between the objective (10) and the light markers (20). The template or contrast surface (22) is located on the outside of the light source (21), specifically at the rear, the light source (21) being visible to the user. In the case that the environment or the background behind the light source (21) is dark enough, it is not necessary to add the contrast surface (22). The system supports the use of other types of luminous markers (20), such as white printed markers with a black border, although these cannot be used in any type of environment.

Figure 3 shows a possible system application scenario, in which the distribution of the markers (20). However, the markers (20) can be placed at different distances from each other, which the system must know in advance. The height between each light marker (20) and the ground is not preset, but it is recommended that it be that which allows direct vision between the stereo camera (12) and the light sources (fL "fb, f ~, ..., fLn ) of the luminous markers (20) In the case of outdoor environments, the markers (20) are placed on vertical supports to achieve the necessary height.In the case of indoor environments, the luminous markers (20) can also go on supports verticals or subjects in the surrounding walls or objects.The relationship between the number m of markers (20) and the distance between them (d "d2) depends on the opening angle (2 <p) of the stereo camera (12 ), of the emission angle (29) of the light source (21), for example the LEO, of the light marker (20) and of the minimum distance (L) that must be between the target (10), user of the system, and light sources (fL "fL2, fL3, .., tLm),

i.e. , the LEOs, so that at least one pair of sources can be visualized; as reflected in the following in the equation m / d ~ _ 1_.

tg9 ~ L

The scenario where the method is applied does not present any predefined characteristics with respect to distribution, plant, obstacles, so that the system adapts to it. The type of environment, as explained above, can be indoor or outdoor. The only restriction that it has is the maximum dimensions of this environment, being limited by the scope of the light sources (fL "fL2, fL3, ..., fLn) chosen. This range is measured according to the intensity and luminous flux of light sources (fL "fL2, fL" .., fLm) And of the sensitivity of the image sensor of the stereo camera (12), from the images captured by the stereo camera (12) and of the coordinates image of the luminous markers (20) calculated in a previous capture, as described below, by the digital signal processor (14) of the system, this system allows to locate specific reference points in the image by means of a detection algorithm of luminous markers (20), such as the one described below.The method to be described is not unique, other variants can be used, returning as image output parameters (u, v) and the diameter of the markers light (20) detec In the image coordinates (u, v) in 2 dimensions, the first coordinate or denotes the coordinate along a horizontal axis and the second coordinate v denotes the coordinate along a vertical axis, in the 2D plane of the image where they are detected the movements. The detection of light markers (20) is divided into the following steps:

Conversion of image to grayscale to reduce considerably the size of the image, since thus it goes from having three channels, red, green and blue, to only one black and white. That is, each pixel in the image reduces its value from 3 byles to 1 byle. Noise filtering to eliminate erroneous pixels and noise from images captured by cameras. The type of filter depends on how clear the images are desired and the delay time that can be introduced into the system. Localization of neighboring pixels with strong contrasts, analyzing the image by windows and looking for those regions where the contrasts between neighboring pixels are greater. This algorithm makes sense because the light sources (21) have pixel values in the image around 255 and the black template (22) has values around O. Obtaining the coordinates of the luminous markers (20), a once the regions that can correspond to light sources (21) are located, verifying that they really are. The first thing that is checked is the shape of the light sources (21), which approximates a circle or ellipse, and the image coordinates (u, v) of its central point as well as its radius are obtained. In addition, these regions have to be checked against each other, verifying that they are all in very similar rows of pixels and that they have intensity values

"

similar, since it is assumed that they are all sources of light (21) with the same luminance. Final verification, comparing the coordinates of the luminous markers (20) calculated with those obtained is an earlier capture. Once the regions that have been checked correspond to luminous markers (20), a final check is made. In this case, comparing the positions of the current markers with those of a previous instant; taking into account that being consecutive moments, the coordinates do not change very significantly from one site to another.

To locate in the image of an environment (11) the reference points that give the location of the target user (10), it is necessary to know the following information: a) the image coordinates (u, v) and radius of each marker ( 20) detected by the algorithm for obtaining markers described above from the image captured by the stereo camera (12); b) the value in degrees é, of the rotation of the target user, returned by the angle measuring device (13), at the time of capture by the stereo camera (12) of each image; and c) the data stored in the memory (15) such as: previous position, actual distance between markers, focal length of the cameras, camera opening angle, distance between cameras ('baseline', in English), image frame ('' frame '' in English), previous radius of the markers, previous position vectors of the markers and anterior angle of rotation.

Considering the particular case of a continuous, unobstructed and square-shaped environment (11), for example, as the scenario depicted in Figure 3, the position of the target user (10) depends on the turns and the type of movements he performs -vertical: up or down, horizontal: left or right-; or if it does not make any movement.

The methods for calculating the position described below are summarized in Figure 7, implemented in different ways, illustrated in Figures 4A-48, 5A-58 and 6A-68, depending on the kind of displacement that is has registered and the number of markers detected, being valid for any type of marker, both bright and printed.

As Figure 7 shows, the first thing is to check the value, in degrees, returned by the angle measuring device (13) to determine if there is a significant turn, which occurs if the angle obtained at the present time is ~ (n) other than the previous moment i "i (n-1); however, if i5 (n) = 0 (n-1) indicates that the target user (10) has not turned. If there is rotation, the coordinates are the same despite the fact that the images captured by the stereo camera (12) change.When the angle of rotation, obtained by the angle measuring device (13), is constant over time, the image frame is compared captured at the current moment, frame (n), with the immediately previous frame (n1) And if they coincide, it is interpreted as no user movement.

if there is no displacement, the method returns the same user coordinates as in the previous moment (: <n.l, Yn.l); otherwise, the position is calculated with all the information, a) -c), mentioned above. In this way, redundant and unnecessary operations are avoided. When position change is detected, the marker detection algorithm is applied. By knowing the radius values of the markers detected at the current instant, r (n) and those of the previous instant r (n-1), the type of displacement of the target user (10) can be identified:

If these radii are different, r (n-1) t r (n), the displacement is up or

down. To know the position of the objective (10) it is necessary to know the distance

between it and the markers, that is, knowing the displacement made

vertically

If those radii are equal, r (n-1) = r (n), displacement is to the right or

left. To know the position of the objective (10) it is necessary to know how much

It has moved horizontally. Once the type of movement performed by the objective (10) has been identified, it can be located in the environment (11) according to the following methods, which depend on the type of movement and the number m of markers (20) detected.

Figures 4A-4B show the case in which it has been determined that there is a vertical movement of the lens (10) and when only a single marker (20) is detected in the binocular image (40) captured by the stereo camera (12). In this case, a triangulation algorithm cannot be used, because the pixels cannot be related to a real distance; Therefore, the stereo vision technique must be used and the following parameters are required:

the binocular disparity ('disparity') of stereo vision given by

Ul and UR coordinates, rectified and undistorted respectively, of the marker

(20) obtained from the two image components, left (41) and right (42), captured by the stereo camera (12); the values of baseline B distance and focal length focaLlength f of the camera stereo (12); Y the angle of rotation (C: »of the target user (10).

In order to transform the image coordinates to depth, the projective geometry is calculated at the current time n according to the equation:

10 Once the Marker distance (n) between the objective (10) and the marker (20) is calculated, its position on stage can be obtained. As it has moved vertically, the only thing that has apparently changed is its coordinate and, but it is necessary to take into account the angle of rotation li to obtain the coordinates

15 absolute. The coordinates (xn • Yn) in the current instant are equal to the coordinates in the previous instant (Xn.1, Yn.,) Plus the sum of the displacement made:

If r (n-1) <r (n) = x n_ 1 + without (or ')' # IL mcr, aaor n -L m.crccdor I

xn

; ¡_-; ¡

20 Figures 5A-5B show the case in which it has been determined that there is a vertical movement of the lens (10) and two or more markers (20, 20 ', 20 ") are detected in the image

(50) captured by the stereo camera (12). In this case, triangulation can be applied, since more than one marker is available, the actual distance (d / m) between markers (20, 20 ', 20 "), the angle of rotation (é), the angle opening (2 ",) of the chamber (12) and the

25 number of pixels (AxB) of the image (50). Knowing the horizontal or image coordinates of the markers (20, 20 ', 20 "), it is calculated, in pixels, the distance in pixels q between them, q: ;;: U2-U1, than in the real world equals ad / m meters, m being the number of

markers, therefore the actual meters of distance Marker (n) between the target

(10) and one of the markers, marker (20), at the present time n is:

L marcCdor (n) = the

tg (2 ", * 2- · u,)

5 In Figures 4A-4B and 5A-5B, the angle is shown that refers to half the opening angle (2q "of the camera (12). Known the distance to the marker and the distance there was in the previous instant n-1, the meters traveled are calculated as the difference between the two, based on that value and the user's previous position, coordinates of the target (10) in the previous instant (Xn_1, Yn_,),

10 its new coordinates (> <or, Yn) can be calculated at the current time:

In this case, stereo vision can also be used to obtain depth to the markers. But it is necessary to apply stereo correspondences, that is, to relate the

15 markers of the left image with their equivalents of the right image. Once the correspondences have been obtained, projective geometry can be applied, as in the case of a single marker to obtain the real distance to each marker.

Figures 6A ~ 6B show the case in which it has been determined that there is a horizontal movement 20 of the target (10).

Figure 6A refers to the case in which only a single marker (20) is detected in the image (61, 62). An algorithm that can be reminded of stereo geometry is applied, but in this case two images of the same moment taken from two different angles are not used, 25 but two images of contiguous instants and the same perspective will be used: the image captured in the instant current (61) And the one captured in a moment immediately before 62).

Likewise, it has the horizontal coordinates of the marker at the current moment (a) and those obtained from the previous frame (Un_,), as well as the previous distance between the marker and the user (Marker) and the focal distance (focaLlength ) of the camera (12), to calculate the horizontal displacement (D) made by the target user (10) according to the

5 following expression:

D = Lmo.rcac.orn_ ~ *! U: ocI_t -un I

f ocaCleng ht

Once the displacement D, in meters, that the target user (10) has made is known,

10 its real coordinates can be obtained, which depend on its position in the previous instant n-1 and on the type of displacement, left or right, performed:

Yes Un-l <Un

x ~ =} '= PI X n -1 -cosCó) "" D V ....... s in (or) '"D ~ PI -1"

v .. =. ' .. V _ -without (o) "- D. N 1

15 Figure 68 refers to the case where more than one marker (20, 20 ', 20 ") is detected in the image. A technique similar to that of triangulation explained in the case of a vertical movement of the user with a vertical plurality of markers detected, but in this case two images (63, 64) captured by the same image sensor are used consecutively in time, having the current image (63) and the image captured instantly

Previous 20 (64). Knowing the actual distance between markers and the pixels between them, p pixels in the current instant n and q pixels in the previous instant n ~ 1, can be extrapolated to the length that the user has moved. This requires knowing, in addition to the distance between markers (d / m), the angle of rotation (15) and the image coordinates (Un. ') Of the markers (20, 20', 20 ") in the image previous (64).

25 d q d IU-u,

D ::: cos Có ') * _ Jio._ = cos (or') * _;,. ' 1 n- ~ "-1

m p m 1 "2 ,, -ud

As in the case of a single marker, once the offset O is known, the actual coordinates of the target user (10) can be obtained:

x,., = X. ., _ l -cosCS) ... D YIl =) '' '-1 i-5111 (0) ~ D

) ',., =)',., - 1 - without (or).;:. D

In this case, the previous case of a single marker detected can also be applied to obtain the displacement (O) made by the target user (10). That is, from the coordinates of the same marker in two contiguous images, neglecting the rest of the markers detected, and with the previous distance between the marker and the user,

10 calculate the displacement D.

Claims (9)

1. Method for spatial location of a target (10) using at least one marker

(twenty)

luminous identifiable in the reference use environment, which in a moment of time i calculates coordinates (Xi, y¡) of the objective (10), characterized in that it comprises: -capturing through a stereo camera (12) a first frame of image in an instant of current time and a second image frame in an instant of previous time, detecting at least one marker (20) in the first and second image frame; - obtaining a radius in an instant of current time and a radius in the previous instant of the at least one, marker (20) detected in the first image frame and second image frame; - obtain an angle of rotation of the objective (10) by means of an angle measuring device

(13)

in the instant of current time and in the instant of previous time; -if the angle of rotation at the current time instant and the angle of rotation at the previous time instant are different, calculate the coordinates (Xn, Yn) of the target (10) at the current time equal to the coordinates (Xn_l, Yn-l) of the objective (10) in the previous instant; -if the first image frame and the second image frame are the same, calculate the coordinates (Xn, Yn) of the objective (10) at the current time equal to the coordinates (Xn.l, Yn-l) of the objective (10 ) in the previous instant; -if not, compare the radios at the current time instant and at the previous time instant of the at least one, marker (20) detected and:

-

if the radii are equal and there is more than one marker (20, 20 ', 20n) detected, the coordinates (xn, Yn) of the target (10) at the present time are obtained by triangulation using the first image frame and the second image frame;

-

if the radii are different and there is more than one marker (20, 20 ', 20 ") detected, the coordinates (Xn, Yn) of the target (10) at the present time are obtained by triangulation using a single image frame that is the first image frame;

-

if the radii are different and there is a single marker (20) detected, the coordinates (Xn, Yn) of the target (10) at the present time are obtained by stereo geometry;

-

if the radii are equal and there is a single marker (20) detected, the coordinates (xo, Yn) of the target (10) at the current time are obtained by calculating image coordinates of the marker (20) at the current time instant in the first image frame and image coordinates of the marker (20) obtained at the previous time in the second image frame.

2.

Spatial location method according to claim 1, characterized in that it uses a luminous marker (20) comprising a light source (21) and a contrast surface (22).

3.

Spatial location method according to claim 2, characterized in that it uses a luminous marker (20) comprising a light source (21) that is a LEO diode.

Four.

Spatial location method according to claim 1, characterized by

10 that, if the radii are equal and there is more than one marker (20, 20 ', 20 ") detected, obtain the coordinates (xn, Yn) of the target (10) at the present time comprises: -for each marker (20 , 20 ', 20 "), obtain horizontal coordinates An of the image in the first image frame at the current time instant and in the second image frame obtain horizontal image coordinates Un.l at the time of 15 previous time; -measure a displacement O of each marker (20, 20 ', 20ft by the expression: ) D ~ 005 (0), '!: .'I ~ 00. (0). ,!:. 1-u1,. -You  m p m lu, "-u, I ., where p is a number of pixels in the current instant n and q is a number of pixels in the previous instant n-1, m is a total number of markers, d / m is a real distance 20 between markers (20, 20 ', 20 ") and' is the angle of rotation of the target (10) obtained; -calculate the coordinates (X n, Yn) of the target (10) at the current time using the equation:

x "-: '1:" _ 1 -cosCó'),. D) '' 1 =) "" 1 -: - without (o), .. D

Yes Un-l> Un

X1'l -X) I _1 + cosCó) .. D \ '= \' _, -sin (6) "D_ n ~ n

Method of spatial location, according to claim 1, characterized in that, if the radii are equal and there is a single marker (20) detected, obtain the coordinates (xn • Yn) of the objective (10) in the instant current includes:

-

for the marker (20) obtain coordinates in the first image frame

horizontal An image at the current time instant and in the second image frame obtain horizontal Un_1 image coordinates at the previous time;

-

measure a displacement O of the objective (10) by the expression:

L merchant '"' 1" 1.l "_1 -tLn l D = 1I-! focal _lel1g ht where the camera (12) has a focal distance focaL / ength and Lmarcadorn_l is a distance between the marker (20) and the objective (10) measured at the previous time, -calculate the coordinates (Xn, Yn) of the objective ( 10) at the current moment through the equation:

\ t = ~) '¡

v ... l..Yes71 (6). D. ' one'! -one

_'t0,., _ v = • 1'1

X ,, _l + cosCo) • D and -s¡.n (8) '"D1'1 -1

6. Method of spatial location, according to claim 1, characterized in that, if the radii, which are the radius at the current instant r (n) and the radius at the previous instant r (n-1), are different and there is more than one marker (20, 20 ', 20 ") detected, get the 15 coordinates (xn, i) of the objective (10) at the present time comprises: - obtaining in the first image frame first horizontal image coordinates Ul of a first marker (20, 20 ', 20H And second horizontal coordinates U2 ) of image of a second marker (20 ') -measure in the instant of current time a distance Lmarcadofn between the objective (10) And the first 20 marker (20) by means of the expression: : 1-u I, ", .E !. .. CO S O" 1 1Z 'm lu, u, l Lmo.r ~ Cl.t! Dr (n) = lA I tg (2 ",. 2" - ",) where the camera (12) has an opening angle 2q>, AxB is a number of two-dimensional image pixels at the present time, m is a total number of markers, d / m is a real distance between the markers (20, 20 ') And o is the angle of rotation of the target (10) 25 obtained; - obtain a distance Lmarcadorn_, measured at the previous time between the objective (10) and the first marker (20); -calculate the coordinates (xn, Yn) of the target (10) at the current time using the equation: If r (n-1) <r (n) If r (n-1)> r (n) x "-x" _ ~ -sin (or '), -ILmarccdOr: "i._ <' - Marker ~. 1 7. Spatial location method according to claim 1, characterized in that. If the radii, which are the radius at the current instant r (n) and the radius at the previous instant r (n-1), are different and there is a single marker (20) detected, obtain the coordinates (Xn, 10 and,) of the objective (10) in the current instan1e comprises: - obtaining UL and UR rectified marker coordinates (20) in a Left image component (41) And a right image component (42) captured by the stereo camera (12), a binocular disparity

-

measure a Lmarcadorn distance between the target (10) and the marker (20) at the current time, using the expression:

8> Lmercadcr (n) = .- ,, - "- L UR where the stereo camera (12) has a binocular disparity equal to UL -U R "a reference distance B and a focal length f, - obtain a distance Lmarcadorn_l measured at the previous time between the 20 objective (10) and marker (20); -calculate the coordinates (xn, Yn) of the objective (10) at the current time using the equation, where <5 is the angle of rotation of the objective (10) obtained at the current time: If r (n-1) <r (n) XII -X "_ 1 + without (or) · I '' mark.:! Orn_ ~ -L rnarcadl" 'n I y ", = .....' '' '-1. + CosCó) ~ ILmarc ..dOr1: _ ~ -Lmarcc. C.or ,, 1 8.-Spatial location system of an objective (10) in a three-dimensional environment (11) comprising at least one reference light marker (20) and a digital processor of signals (14) to calculate coordinates (Xi, y¡) of the target (10) in an instant of time i, characterized in that it comprises: -a stereo camera (12) to capture a first image frame in an instant of current time and a second image frame in an instant of previous time; -a device for measuring angles (13) to obtain an angle of rotation of the objective (10) in the instant of current time and in the instant of previous time; -the signal processor (14) with access to a memory (15) that stores a radio in an instant of current time and in a radius in the previous instant of time of the at least one, marker (20) detected in the first image frame and second image frame; he signal processor (14) configured to:

-

If the angle of rotation at the current time instant and the angle of rotation at the previous time instant are different, calculate the coordinates (x ", Yn) of the target

(10) in the current instant equaling them with the coordinates (Xn_1, Yn-1) of the target (10) in the previous instant;

-

If the first image frame and the second image frame are the same, calculate the coordinates (xn, Yn) of the target (10) at the current time equal to the coordinates (Xn_1, Yn_1) of the objective (10) in the previous instant ;

-

if not, compare the radios at the current time instant and at the previous time instant of the at least one, marker (20) detected and:

-

if the radii are equal and there is more than one marker (20, 20 ', 20 ") detected, calculate the coordinates (Xn, Yn) of the target (10) at the current time by triangulation using the first image frame and the second image frame;

-

if the radii are different and there is more than one marker (20, 20 ', 20 ") detected, calculate the coordinates (xn, Yn) of the target (10) at the current time by triangulation using a single image frame that is the first image frame;

-

if the radii are different and there is a single marker (20) detected, calculate the coordinates (xn, Yn) of the target (10) at the current time using stereo geometry;

-

if the radii are equal and there is a single marker (20) detected, calculate the

5 coordinates (Xn, Yn) of the target (10) at the current time using image coordinates of the marker (20) at the current time in the first image frame and image coordinates of the marker (20) obtained in the previous time in the second image frame. 10. A spatial location system according to claim 8, characterized in that the light marker (20) comprises a light source (21) that is a LEO diode. 10.-Spatial location system according to any of claims 8-9, characterized in that the luminous marker (20) which includes a contrast surface 15 (22) 11.-Spatial location system according to any of claims 810, characterized in that the environment (11) is interior. 12. A spatial location system according to any of claims 810, characterized in that the environment (11) is external.