BR102014010722B1 - SYSTEM AND METHOD FOR LOCATION OF SPATIAL OBJECTS COORDINATES AND USE OF SUCH SYSTEM FOR LOCATION OF OBJECTS DISPOSED IN INTERNAL ENVIRONMENTS - Google Patents

Abstract

SYSTEM AND METHOD FOR LOCATION OF SPACIAL COORDINATES OF OBJECTS AND USE OF THE SAID SYSTEM FOR LOCATION OF OBJECTS DISPOSED IN INTERNAL ENVIRONMENTS The present invention describes a system and method for locating spatial coordinates of an object of interest, using the combination of two or more types of technologies and/or sensors, defining a hybrid system and returning the spatial position of an object of interest in the x, y and z axes. The present invention also consists of the use of said system for locating objects arranged indoors. The present invention lies in the fields of electrical engineering and automation and control.

Description

Field of Invention

[0001] The present invention describes a system and method for locating spatial coordinates of an object of interest, using the combination of two or more types of technologies and/or sensors, defining a hybrid system and returning the spatial position of an object of interest. interest in the x, y, and z axes. The present invention also consists of the use of said system for locating objects arranged indoors. The present invention lies in the fields of electrical engineering and automation and control.

Background of the Invention

[0002] Object location and/or tracking systems are increasingly present in the daily lives of people across the globe. Such systems can have different applications, depending on the need, the object to be located/tracked, the necessary precision and, mainly, the environment in which such system will be used.

[0003] Location systems in external environments (outdoor), commonly operate based on satellite monitoring, by GPS technology (Global Positioning System) and its use is growing more and more, becoming, at present, the most used system in the world. concerning the location, tracking and navigation of people, vehicles and other objects. However, such technology is not suitable when the monitoring object is located indoors, such as buildings, sheds, houses, underground environments or equivalent, because in these conditions the targeted and direct communication to the satellites is interrupted.

[0004] Tracking and localization systems in indoor environments (indoor) have been developed with greater emphasis nowadays, however, indoor environments are more complex when compared to outdoor environments, due to the large number of obstacles and interfering phenomena existing in a density reduced space. Therefore, several technologies have been proposed over the years, generally using different types of technologies and possible methods, mostly aimed at locating people and objects. The most used technologies are: Radio-frequency Identification (RFID - Radio-Frequency Identification), Wi-fi (IEEE 802.11), radio frequency (2.4 GHz) and ultra-wideband (UWB). Such technologies, in general, are called IPS (Internal Positioning System)

[0005] IPS systems have a high potential to become as popular as GPS systems, as they can be used in various applications, such as: providing navigation in indoor environments of buildings, and can be used to help the visually impaired to find locations and/or products of interest or to enable guide systems for the general public, in places such as museums, fairs or equivalent; location of objects in inventories or books in libraries; traceability of people with special needs, the elderly or children; locate equipment and drugs in hospitals and find specific items in distribution centers; or any other similar application. In public safety and military use, such systems are needed to track detainees and aid in navigating police, firefighters, and soldiers on their missions inside buildings.

[0006] The current library management systems only inform if the book is borrowed or not. Other systems maintain in a static database the shelf number where the book should be present. However, such systems cannot provide the precise and dynamic location of the book in the collection. In museums, tourists can benefit from navigation services to check the location of pieces and create a more attractive sequence for visitation.

[0007] Many applications require information with high precision of the location of objects, that is, there is a demand for systems capable of determining the position of objects with precision around 5 cm or less. In this application, a human or robot needs to find the location of a certain item, and such location must be precise enough for the item to be collected correctly, even if there are other objects nearby or any other type of interfering phenomenon.

[0008] Another application, considered to be a variant of item collection, consists of ensuring the location of objects, such as systems that must verify whether products, materials or medical equipment are in fact where they should be.

[0009] Some systems already used for these purposes are described below. The Wi-Fi technology (IEEE 802.11) has as its main advantage the infrastructure already established in several places, but the accuracy of the location based on this technology is affected by the interfering elements existing in closed environments, in addition to depending on the strength of the signal and the standard of the received signal. Thus, the movement and orientation of objects, the human body, devices, walls or doors influence the result of the accuracy of the location.

[0010] RFID technology has the great advantage of low manufacturing cost of tags affixed to monitored objects, in addition to easy attachment to objects and high reading rate, however, location systems based exclusively on RFID have some limitations, such as: accuracy better than one meter; lack of standardization; intolerance to certain types of material (mainly water and metals); and low scalability, that is, the larger the area, the more readers and tags are needed.

[0011] Ultra-wideband (UWB) technology has the main advantage of greater accuracy compared to previous technologies, but represents a high cost of implementation, which makes its use unfeasible for a series of applications.

[0012] Radiofrequency technology (2.4 GHz) has some limitations and problems regarding the influence of interfering phenomena that can decrease performance, thus having results affected by the movement of people, objects, furniture, walls and doors.

[0013] It is noted, therefore, that the use of a single technology to solve such a technical problem is not advantageous due to cost limitations. On the other hand, in the current state of the art, it is noted that few solutions are aimed at the use of hybrid systems to solve the problem of locating and/or tracking objects in indoor environments. Currently, wireless systems are combined with other technologies, such as optical, inertial and ultrasonic systems. Some problems for most of the proposed systems consist of their dependence on new components that must be manufactured in their entirety, raising the cost of producing the new technology product, and on various materials and additional resources. In this way, the use of some systems can also facilitate the implementation process, by using devices already available on the market and distributed on a large scale.

[0014] In the search for the state of the art in scientific and patent literature, some documents were found that deal with the subject, without, however, anticipating or suggesting the teachings of the present technology.

[0015] Document PI0701740-5 reveals a network-based location system of radio frequency transmitters and receivers, ultrasound, thermal sensors and/or video cameras. The system offers the two-dimensional location of a person or object through the characteristics of difference and/or delay of the signals, such as frequency, time, power or heat. The difference and delay values are applied to triangulation models of the signals, providing the location of the person or object. The solution presented by document PI0701740-5 has the disadvantage of requiring a high number of antennas/receivers to optimize the accuracy of the position of the observed object, given that such positioning is obtained through triangulation.

[0016] Document PI0601824-6 reveals a system that uses electronic tag sensitized through remote control. The location of the object is provided through the sound and light signal on the tag, and must be provided by the user (human) and not through the automated system that informs the position of the object in a computerized interface. The document does not mention the technology used in remote communication.

[0017] Document CN101782652 reveals an IPS system based on RFID technology, which uses active RFID tags associated with objects of interest. Such tags send a signal to an RFID reader, which communicates with a computer equipped with an intelligent positioning system, which analyzes the signal strength and estimates the position of the object of interest. Such a solution has the disadvantage of using a single sensing system (RFID) that does not have accurate accuracy and also because it uses active sensors, which have a higher market cost than passive RFID sensors.

[0018] Document CN102306264 discloses an IPS system based on RFID technology, which has an equipment system, an event processing system, a real-time positioning system, a track tracking system and a mobile services system, based on RSSI reference of tags based on a weighted average, adopting logarithmic values of environment variables. Such a solution has the disadvantage of using a single sensing system (RFID) that does not have accurate accuracy.

[0019] Thus, from what can be seen from the researched literature, no documents were found anticipating or suggesting the teachings of the present invention, so that the solution proposed here has novelty and inventive activity compared to the state of the art.

[0020] In this way, we can consider that the state of the art lacks technologies that use a hybrid approach in relation to the constant technical problems for the area of operation, in order to optimize the location accuracy, without such improvement impacting the increase in the cost of manufacturing such systems.

Summary of the Invention

[0021] The present invention solves the constant problems in the state of the art from the hybrid approach to the problem, in the development of a functional system, which combines technologies, such as RFID technology, with RFID localization, computer vision and systems infrared. The system also stands out for its reduced cost compared to the state of the art, as it is based on equipment already available on the market, in addition to contemplating the precision required by the system, and can be used for small objects, that is, around 5 cm .

[0022] It is, therefore, a first object of the present invention a spatial coordinate location system of objects comprising at least one observation point and at least one observed object, using two subsystems for determining the location of at least one object of interest , namely: image recognition system using computer vision; and system for measuring distances and defining spatial coordinates, being: a. The image recognition subsystem through computer vision, preferably defined by a camera controlled by image recognition software; B. The subsystem for defining spatial coordinates by means of an RFID signal, where preferably given by the object of interest comprising a passive RFID tag (RFID receiver) (1) and the RFID emitter being an antenna (3) emitting RFID signals and receiving signals response by passive RFID tags (1) (RFID receiver) associated with the objects of interest; and c. The distance measurement subsystem by means of an infrared emitter and receiver, preferably for measuring linear distances between the sensor and the object of interest.

[0023] In a preferred embodiment, the observation point is equipped with: camera controlled by image recognition software, antenna (3) emitting and receiving RFID signals; and infrared sensor and emitter.

[0024] In a preferred embodiment, the observed object is provided with a passive RFID tag (1) and fiducial marker (2) arranged on a visible surface, when associated with the observed object.

[0025] It is a second object of the present invention a method of locating spatial coordinates of objects using a system provided with at least one observation point and at least one observed object, the observation point comprising: at least one emitting and receiving antenna of RFID signals (3); at least one infrared sensor and emitter; and at least one computer vision system and the observed object comprising at least one RFID receiver (1) (passive tag) and at least one visible region, provided with a fiducial marker (2), said method comprising at least one of the defined steps for D. RFID signal emission (I) through the antenna (3); and. reception of the signal emitted in step (a) by the RFID receiver (1) associated with the object of interest, finding the tag (II); f. sending a response to the signal of step (a) by the RFID receiver (1) to the RFID antenna (3); g. computer vision mapping (III) of the region pointed by the RFID system to determine the spatial coordinates (x, y, z) of the location of the RFID receiver (1), within a previously referenced region, based on the response signal received by the antenna (3); H. directing the computer vision system to an analysis region, indicated by the RFID system as the region where the object of interest is located, defined by the spatial coordinates (x, y, z) obtained in step (d); f. computer vision mapping (III) of the analysis region by obtaining and superimposing at least one region of interest (ROI), based on the fiducial marker (2) (IV) corresponding to the object of interest; g. definition of x and y coordinates in the perpendicular plane to the sensor (V) and to the observation point, based on the overlapping of the regions of interest (ROI); H. measurement using infrared (VI) to determine the linear distance (z) between the sensor of the observation point and the object of interest (VII), using the infrared emitter and receiver.

[0026] In a preferred embodiment, the method of locating spatial coordinates of objects comprises the predecessor steps of calibrating the system, comprising the steps of: a. arrange RFID receivers (1) in locations with known coordinates, from the point of observation; B. emitting a signal through the RFID antenna (3); ç. collect the response signals by means of the RFID antenna (3) from the RFID receivers (1); d. associate values collected from each RFID receiver (1) to the coordinates (x, y, z) of the respective RFID receivers (1) and store such values in a database; and is. adopt the values stored in step (d) as references in determining the distance between the RFID antenna (3) and RFID receivers (1).

[0027] In a preferred embodiment, the determination of the spatial coordinates (x, y, z), based on the response signals from the RFID receivers (1), is given by means of probabilistic analysis, by comparing the values obtained by the response signal of the RFID receiver (1) associated with the object of interest, with the reference values obtained in the system calibration. In a preferred embodiment, the method also comprises a step of measuring the linear distance (z) between the point of observation and the object of interest through the infrared emitter and receiver, preferably through estimation by artificial neural networks (ANN) and/or SVR (Support Vector Regression).

[0028] It is a third object of the present invention the use of said system for determining the spatial coordinates of objects in indoor environments, operating through said method of locating spatial coordinates as defined above.

[0029] These and other objects of the invention will be immediately valued by those versed in the art and by companies with interests in the segment, and will be described in sufficient detail for reproduction in the following description.

Brief Description of Figures

[0030] In order to better define and clarify the content of this patent application, this figure is presented:

[0031] Figure 1 shows the flowchart of the activity steps of the proposed merger method.

[0032] Figure 2 shows an example of the scenario and infrastructure used by the proposed system, namely: (1) RFID tags and (2) fiducial markers.

[0033] Figure 3 shows the infrastructure and configuration necessary to carry out the data collection of the RFID system, considering the space (axis x, y, z) and being: (1) RFID tags; (2) fiducial markers; (3) antenna and (4) RFID reader.

[0034] Figure 4 shows MROI (Multiple regions of interest - ROI1, ROI2 and ROI3) obtained through the probabilistic model based on RFID, with 3 different ROIs (regions of interest): ROI1, ROI2 and ROI3 and the presence of (1 ) RFID tags and (2) fiducial markers in ROI2.

[0035] Figure 5 illustrates evaluated scenarios and the displacement used in training and system estimates, considering: coordinates x, y, z, scenarios C1, C2, C3 and C4 and linear distance between the camera and C2(dz).

[0036] Figure 6 shows an example of an artificial neural network (ANN) structure with scalability for two dimensions - 2D.

[0037] Figure 7 shows a simplified example of an observed object Figure 8 illustrates a scenario contemplating the reference tags (RFID receivers) (squares in positions 1 to 13) and the target markers of the online phase (triangles in positions 1, 5, 9, 14, 15 and 16).

[0038] Figure 9a shows a photo of an observed object and figure 9b shows a screenshot of the running prototype, through computer vision. 3D location of fiducial marker (2) at 70 cm distance.

[0039] Figure 10 shows the system view displaying the positions inferred by the RFID subsystem (triangles) and the final position of the target (stars), being: (1) RFID tags and (2) fiducial markers.

Detailed Description of the Invention

[0040] The present invention comprises a system and method for locating spatial coordinates of objects, especially for indoor environments, defining its use as an IPS system. The common inventive concept among the objects of the present invention consists in the use of a hybrid system, that is, equipped with different types of sensing, to determine, with high precision and low cost, the position of objects of interest within a controlled three-dimensional region. . More precisely, such types of sensing are of the RFID type; computer vision and infrared, however not limited to these, and use a probabilistic analysis, based on data previously calibrated to the operation of the system/method, on the location of an object of interest in space.

[0041] Thus, the different objects described here are interrelated because they present the same inventive concept, and, therefore, present unity of invention among themselves. Such objects will be described ahead with a level of detail sufficient to allow their total reproduction by a technician in the subject.

[0042] An indoor positioning system (IPS) is defined as a system that can determine, continuously and in real time, the position of something or someone in a physical space, such as a hospital , a school, a gym, etc. An IPS must provide up-to-date location information on the target object, estimate its positions within a maximum delay to be respected, and cover the area expected and required by the system user. In this way, the localization system for indoor environments (IPS) of the present invention is a hybrid model, with computer vision technique operating in conjunction with RFID technology, fiducial marker (2) and infrared.

[0043] Figure 2 illustrates an embodiment of the present invention, where RFID tags (1) with fiducial markers (2) are affixed to the objects that you want to recognize/locate in the controlled environment.

[0044] The system is able to obtain the 3D location/position (x,y,z axes) of each object to be recognized present in the scenario. The location of the object is performed through the relative distance between the executor and the object, thus determining the position between the x axis (horizontal distance in relation to the edge of the image captured by the camera), the y axis (vertical distance in relation to the edge bottom of the image) and the z-axis (depth distance).

[0045] In figure 1, the process of locating the monitored object is illustrated, starting from its location, which starts by issuing an RFID signal (I) through the RFID antenna (3). Finding an RFID tag (II), mapping is performed by computer vision (III) of the region indicated by the RFID system as the region of interest (ROI), in which the object of interest is located. Once the marker (2) (IV) is detected, the X and Y coordinates are determined in the plane perpendicular to the sensor (V), then the measurement is performed using infrared (VI) to determine the distance between the sensor and the object desired (VII), distance in Z, the infrared detection step being optional, according to the Z distance detection method adopted.

[0046] According to the summarized flowchart in figure 1, the RFID location (I) is the detection of the region of interest (Region Of Interest - ROI) that indicates the most accurate 3D location of the object of interest. ROI is defined as an area of limited size and less than the dimensions of the full scenario. Since other localization techniques can be applied to this reduced area, always looking for the best performance in the global localization process.

[0047] Probabilistic models are used to infer the location proximity of objects in the scenario in the RFID location. Thus, an initial data collection, for system calibration, causes the statistical model to be stored.

[0048] For the system calibration step, a fiducial marker (2) must be affixed together with the RFID tag (1), which may be in only one type of material, among several possible ones, such as paper, plastic, among others .

[0049] This RFID location step requires an RFID reader (4), an RFID antenna (3) and an RFID tag (1), therefore, the RFID antenna (3) is positioned in front of the tags, with the reader (4) activated for a certain fixed period of time. The data collected by the reader (4) are stored in a database and thus store all measurements and data performed from the RFID tag (1). Some of them are: the Received Signal Strength Indication (RSSI) level, the number of readings, and the phase of the electromagnetic waves.

[0050] In one embodiment of the invention, the received signal (RSSI) is the signal used by the RFID system during training and execution of object location. The RSSI value used in location systems is based on the power received by the reader, as shown in figure 6. Readers currently available on the market, in compliance with the LLRP standard, provide the RSSI value for the readings performed.

[0051] As can be seen in figure 3, an example of the necessary infrastructure for data collection is demonstrated, also illustrating each axis of the 3D position, which are described as: • Axis x: horizontal distance between a certain limit of the scenario up to the position of the label (1). • Y axis: vertical distance (height) between the floor and the label (1). • Z axis: straight line distance between the antenna (3) of the RFID reader (4) and the tag (1).

[0052] Data collection requires some factors, especially that the equipment is kept static, in order not to influence the results. Therefore, a statistical analysis must verify the correlation between the measurement variables (explanatory) and the x position, y position and z position variables (response). A statistical inference technique and a probabilistic model makes the 3D position of a target tag, based on the measurements performed, estimated. Therefore, after the system configuration phase, the execution phase follows, which applies the developed model.

[0053] In the execution phase, the RFID reader (4) measures the tags to be located. The model must be able to obtain the location estimate using labels and positions different from those used during the configuration phase.

[0054] Given the low precision of IPS's based on radiofrequency, a tolerance area is preferably adopted, based on the center of the location estimated by the probabilistic model. In this way, a region of interest (ROI) is estimated for the possible location of the tag, which is subsequently used by the sensor fusion method.

[0055] Due to the low performance of RFID technology in IPS's, the model response results in more than one ROI simultaneously, all of these regions being explored by the fusion method. Thus, this response can be termed as “Multiple Region Of Interest” (MROI). The MROI is composed of the list of possible locations (3D) of the target tag.

[0056] The computer vision (III) of the proposed system, preferably consists of optical localization systems for indoor environments, using cameras as sensory technology, video cameras or digital cameras may be used in the image capture process. Most optical localization systems are based on the visual analysis technique using marker-type algorithms. Marker-like algorithms are based on tags with unique patterns in their content. Generally, elements are created to receive tags having such characteristics, thus receiving the name of fiducial tags. Fiducial markers are arranged in a physical environment to support object identification, location, and tracking. A fiducial marker (2) is designed to solve the following problem: for a given input image (still image or streaming video frame), provide the list of markers found in the image.

[0057] The proposed visual localization model is based on the detection and reading of fiducial markers (2), encoded in images captured from the environment. Fiducial markers (2) encoded are used in systems that require greater accuracy of location and environments where there may be variation in lighting. This visual location approach provides distinction between each object in the scenario, as it uses a unique identifier for each marker.

[0058] In this proposal, the objective of the visual location is to detect and indicate the location of a certain fiducial marker (2) that has a specific number stored in its codification. The choice of the barcode was due to the fact that it is a fiducial marker (2) encoded, that is, it is a marker that has a code associated with the image. Such a system has the advantage of ease of implementation and available software libraries, where the bar code is a bitonal fiducial marker (2), bringing better performance to the detection algorithm.

[0059] Preferably, a digital camera is used in order to capture the image of the scenario where the objects are located, which may be an integral part of a smartphone equipment or a video game sensor or its equivalent, among others. As can be seen in figure 5, the system was evaluated in four scenarios, namely C1, C2, C3 and C4. In each scenario, the distance dz between the camera and the frames are 100, 140, 180 and 220 cm, respectively.

[0060] From the capture performed by the camera, the system applies an algorithm to detect, read and locate the fiducial bar code (2) on the image. Each of these phases can be described as: • Detection: phase of the algorithm that tries to find a possible barcode - fiducial marker (2) - in the image content. • Reading: phase that analyzes each bar and verifies which code (identifier) is stored in the set of bars. • Location: phase that, after detection, must return to the position (x, y) of the barcode - fiducial marker (2) - inside the image. Only bar codes in which the detection and reading phases were successful return their location information.

[0061] The detection and reading phases were implemented from the BarcodeImaging library, which has its source code open and is licensed under “The Code Project Open License (CPOL)”.

[0062] According to the steps in figure 4, the fusion method proposed for the present invention solves most of the problems with similar proposals, and is based on the application of the visual localization method on the result obtained by RFID localization, making that RFID location becomes even more accurate for objects. MROI supports location refinement by the visual location method. Fusion causes an infrared projector to be associated with a specific camera to obtain the distance in depth (z or dz axis) of the target object.

[0063] The proposed fusion method is initiated by combining RFID location (MROI) with visual analysis. Therefore, for each ROI provided by the RFID, a visual analysis is performed to find the fiducial marker (2) in the image. It makes a comparison between the code of the RFID tag (1) detected by the RFID system (antenna (3)) and the code stored in the fiducial marker (2). Once the marker's x and y coordinates are determined, the stream proceeds to the infrared location, which will include the depth z coordinate, and the final result of the target object.

[0064] In order to obtain the distance in depth z, an infrared device is connected to the computer, which runs the localization system. Currently, infrared devices are already sold on a large scale, mainly as accessories for video game consoles or even in robotics teaching kits. Such a device is capable of obtaining distance information in depth for each point of the scenario, as it has an infrared projector combined with an infrared camera in its structure. In this part of the system, infrared localization is employed in order to obtain an estimate of the distance in depth between the infrared device itself and the target object.

[0065] The present development is based solely on the use of devices already on the market and distributed on a large scale, both for the necessary infrastructure and for the location devices themselves. In addition, the components used have an affordable cost and part of them are already part of the day-to-day activities of organizations in general. The reuse of equipment infrastructure that the organization already has is also taken into account, enabling a faster, easier and financially advantageous implementation.

[0066] One of the differentials of the development of the present invention concerns the reuse of the infrastructure of organizations, in addition to the use of devices (sensors) already available on the market on a large scale, bringing the benefit of reducing costs associated with the acquisition, deployment and maintenance of the system.

[0067] Another economic advantage of the invention is the use of low-cost fiducial markers, using a visual marker printed on paper and a passive RFID tag (1) for each object to be located. In addition, the markers do not require any type of battery, the main characteristic of passive tags, thus further reducing the cost of acquiring and maintaining the inventive object.

[0068] The high precision achieved by the system also stands out as a differential in relation to competitors. The location of small objects (around 5 cm) increases the range of applications that can benefit from the system, bringing greater possibilities and ideas for the use of the invention.

Example 1. Preferred Realization

[0069] The examples shown here are intended only to exemplify one of the many ways to carry out the invention, without, however, limiting its scope.

[0070] In one embodiment, the location system for indoor environments (IPS) of the present invention is a hybrid model, with computer vision technique operating in conjunction with RFID technology, fiducial marker (2) and infrared. Passive RFID tags (1) and paper fiducial markers (2) are affixed to the object to be 3D located.

[0071] For the RFID location (I) that detects a ROI region of interest, at first the size of the ROI coverage area is manually configured by the system user.

[0072] The system preferably requires that the objects spread around the environment with RFID tags (1) and fiducial markers (2) are in the form of a grid, which is how the prototype of the present invention was made.

[0073] For the proposed system, during data collection in the configuration phase, some factors were fixed and kept static, such as the frequency fixed at 915Mhz and the power at 30dBm. In the execution phase, the RFID reader (4) is activated, carrying out measurements of the tags to be located. The RFID reader (4) settings (for example, frequency and power) must keep the same values used in the configuration phase.

Para a localização visual (III) foi utilizada uma câmera digital fotográfica empregando algoritmo do tipo marker, sendo os marcadores fiduciais (2) utilizados para o protótipo do tipo papel, fazendo assim com que haja a leitura da codificação presente no marcador fiducial (2), a partir de uma base de dados existente com cada identificador único de cada marcador. A fase de localização passou por alterações no código-fonte da biblioteca BarcodeImaging, para assim fornecerem as coordenadas (x, y) de cada código de barras detectado e lido.Para isto, foram adicionadas estruturas de dados onde, a cada código de barras cujo número natural fosse obtido com sucesso, a posição da banda corrente (x) e a posição central horizontal do código de barras (y) fossem armazenadas e fornecidas como saída do algoritmo.[0074] Figure 4 shows an environment where the regions of interest (ROI) returned by the MROI (Multiple regions of interest) model are displayed. In Table 1 below, an MROI of a given label is presented, containing the possible locations for each axis and the code of the detected label.

For the visual location (III) a digital camera was used using a marker-type algorithm, with the fiducial markers (2) being used for the paper-type prototype, thus making it possible to read the coding present in the fiducial marker (2) , from an existing database with each unique identifier of each marker. The localization phase underwent changes in the source code of the BarcodeImaging library, in order to provide the coordinates (x, y) of each bar code detected and read. For this, data structures were added where, to each bar code whose natural number was successfully obtained, the position of the current band (x) and the horizontal central position of the barcode (y) were stored and provided as output of the algorithm.

[0075] For the proposed system of the present invention, a developed prototype demonstrated a good functioning between the visual localization result and the infrared localization process. Figure 9 demonstrates, in a comparative way, an object and the screen capture image of the system running 70 cm away from the fiducial marker (2) to be located, and figure 9a shows the image captured by the digital camera used in visual localization, while figure 9b represents, in gray tones, the depth distance obtained by the infrared sensors (so that, the farther, the lighter the tone). The fiducial marker (2) is located in the two-dimensional image, indicated by the yellow dot drawn on the interface.

[0076] From this location, the 3D location is informed, which in turn obtains the third coordinate (z axis) that is displayed in the status bar, and corresponds to the depth distance, which in this case is seen with the result of 69.6 cm, with an error of only 0.4 cm from the ideal distance. Example 2. Preferred Realization

[0077] In one embodiment, a known edge detection algorithm is implemented and defined to detect the position of the visual marker in the subimage, using canny edge detection in evaluating photographs of scenery regions in search of visual markers and a simple visual marker represented by a black square on a white background, whose side measures 4.5 cm (20.25 cm2), as shown in figure 10.

[0078] This type of marker has the advantage of ease of creation, low cost and fast detection (less than 1s). Because the marker is black and white, problems resulting from poor ambient lighting are mitigated.

[0079] Due to restrictions of federal regulations, RFID equipment uses different operating frequencies. In order to overcome these limitations, the data collected in both phases are partitioned by operating frequency. This proposal aims to separate the RSSI (Receive Signal Strength Indicator) values of different frequencies, avoiding combining the results of different frequencies. Thus, it is possible to employ location models based on pattern analysis with greater probability of success using commercial equipment that meets radiofrequency regulations.

[0080] The system was implemented in its entirety, being developed in the C# programming language. Statistical models were developed and integrated into the system using the MATLAB language.

[0081] Figure 10 shows the system view showing the positions inferred by the RFID subsystem (triangles) and the final position of the target (star). The graphical interface helps users to easily identify the location of the marker. The processing time for locating each marker is less than 1 second, excluding the RFID collection time (approximately 3 seconds).

[0082] Experiments were conducted in the laboratory in order to evaluate the performance of the location and compare the developed models. The results in two-dimensional space demonstrated an accuracy of 8.9 cm in the best case and the hybrid approach was up to 32% better than the location that uses only one technology (RFID). These results demonstrated that the system can be applied to a location of objects with reduced size, although the proposed approach still has some limitations in scenarios where many items are close to each other.

onde os valores (Xt,yt,Zt) são as coordenadas estimadas em cada localização realizada pelo sistema, os valores (xt, yt, zt) representam as coordenadas reais do marcador e k representa o número de estimativas.[0083] In order to synthesize the results of several repetitions and combinations of experiments, the accuracy of the location is demonstrated through the Root Mean Square Error (RMSE). The RMSE is calculated by the difference shown in equation (i) below:

where the values (Xt,yt,Zt) are the estimated coordinates in each location performed by the system, the values (xt, yt, zt) represent the actual coordinates of the marker and k represents the number of estimates.

[0084] On the z axis, the unit of measure used in the coordinates is the distance in centimeters between the sensor (camera) and the marker affixed to the target object. On the x and y axes, the reported distance is relative to the side and top edges of the scenario photograph, respectively. As can be seen in figure 5, the system was evaluated in four scenarios, namely C1, C2, C3 and C4. In each scenario, the distance dz between the camera and the frame is 100, 140, 180 and 220 cm, respectively.

[0085] The realization uses, in a comparative way, models of ANN (Artificial Neural Networks) and SVR (Support Vector Regression) to estimate these distances. Artificial neural networks (ANN) are efficient models for location problems, since they make it possible to infer positions through the interpolation of known coordinates. One of its main characteristics lies in the fact that no prior knowledge of the geometry of the environment is required (position of walls, furniture and other obstacles), without this impacting on the quality of measurements, thus making the system more versatile.

[0086] In general, in neural networks, the most common learning paradigm is supervised, associated with a multilayer direct network. For the present invention, a feedforward network with back propagation was modeled using four layers, such network consisting of “n” neurons in the input layer, 24 neurons and 12 neurons in each hidden layer, and “k” neurons in the output layer. . The number of neurons in the input layer must be equal to the number of antennas used. The number of neurons “k” in the output layer depends on the spatial dimension of the problem (k=2 for 2D, k=3 for 3D). Figure 6Error! Reference source not found. presents the network configuration in 2D scalability, which is a preferred performance and uptime value.

[0087] The modeling of the location problem by SVR (Support Vector Regression) is defined similarly to neural networks, with the model training set having the RSSI values received by each antenna present in the environment. The desired output is one of the spatial coordinates of the RFID tag position. As the SVR technique allows only one output in each formulation, the process is performed once for each coordinate. For example, in a three-dimensional location problem, three formulations are performed, one for the x coordinate, another for the y and the last one for the distance in depth (z). Regardless of which coordinate is observed, the RSSI values of all antennas are considered in the model's input space vector.

[0088] Figure 7 shows a simplification of the proposed modeling, where a given RSSI is measured and related to the distance in depth of the target tag. In the example, the SVR tries to find a function f(x) that reaches at most one deviation ε in relation to the z position of the target label. In the complete modeling, there may be several input spaces x, one for each antenna present in the environment.

[0089] In the online phase, the objective of the experiments is to locate six markers distributed in the scenario. Three markers being in positions already used in the offline phase and three in unknown positions. Figure 8 Error! Reference source not found.illustrates the markers that must be located (triangles) and their respective identifiers (EPC). In this case, markers 1, 5 and 9 are in positions already used in the offline phase, while markers 14, 15 and 16 are in places never trained before. During the online phase, reference tags are not present in the scenario.

[0090] Three validation tests were performed, the first considering “only RFID”, that is, the visual subsystem was not used. In the “dense” test, in addition to the six target markers, another ten visual markers were placed on the whiteboard. These additional visual markers were not targeted for location, and were only present to simulate a more complex environment. In this test, the camera captured the image with all 16 visual markers simultaneously. In the last test, called “clean”, only one visual marker was present in the environment in each execution of the hybrid system.

[0091] In the comparison between the ANN and SVR approaches, the ANN model performed 30% better than the SVR model. The results of the ANN model in the hybrid approach exhibit an error between 8 and 24 cm. The ANN model was inferior to the SVR only in the “RFID only” mode of the C3 scenario. In a comparison targeting the “clean” test, the ANN model demonstrated a 1.7 times better accuracy compared to the SVR.

[0092] Table 2, below, summarizes the performance of the system against the scenarios, models and tests evaluated and the square root of the mean squared error (RMSE) of the measurements for all scenarios. Table 2 - Localization performance (RMSE error in cm) for each scenario, validation test and machine learning approach.

[0093] Table 3 summarizes the system performance in the 3D scenario. Unlike the 2D scalability results, the SVR model performed 5.3% better than the ANN model. In general, the results of the hybrid system and RFID subsystem validation tests were very similar. In the ANN model, the hybrid approach increased the error by 1 cm compared to the RFID subsystem. Table 3 - Location performance (RMSE error in cm) under three-dimensional scaling.

[0094] Table 4, below, summarizes the performance of the system in the 3D scenario using the infrared sensor to provide the estimate of the Z coordinate. In this case, the infrared device was positioned next to the camera. Table 4 - Location performance (RMSE error in cm) under 3D scalability with the aid of the infrared sensor.

[0095] Those versed in the art will value the knowledge presented here and will be able to reproduce the invention in the presented modalities and in other variants, covered in the scope of the appended claims.

Claims (9)

1. Object spatial coordinate location system comprising at least one observation point and at least one observed object, the system being characterized by the fact that it uses two subsystems to determine the location of at least one object of interest, being a system of image recognition through computer vision; and system for measuring distances and defining spatial coordinates, being: a. the computer vision subsystem is defined by a camera, controlled by image recognition software; B. the subsystem for defining coordinates by means of an RFID signal, given by the object of interest, comprising a passive RFID tag (1) (RFID receiver) and the RFID emitter being an antenna (3) emitting RFID signals and receiving response signals by the passive RFID tags (1) (RFID receiver) associated with the objects of interest; ç. the distance measurement subsystem by means of sensor and infrared emitter to measure linear distances between the sensor and the object of interest. 2. System for locating spatial coordinates of objects, according to claim 1, characterized in that the observation point is equipped with: camera controlled by image recognition software, antenna (3) emitting and receiving RFID signals; and infrared sensor and emitter. 3. Object spatial coordinate location system, according to claim 1, characterized in that the observed object is equipped with a passive RFID tag (1) and fiducial marker (2) arranged on a visible surface, when associated with the observed object . 4. Method of locating spatial coordinates of objects, according to any one of claims 1 to 3, characterized in that it uses said system with at least one observation point and at least one observed object, i. the observation point comprising: at least one antenna (3) for emitting and receiving RFID signals; at least one infrared sensor and emitter; and/or at least one computer vision system (camera); and ii. the observed object comprising at least one RFID receiver (1) and/or at least one visible region, provided with a fiducial marker (2); said method comprising the steps defined by: a. RFID signal emission (I) through the antenna (3); B. reception of the signal emitted in step (a) by the RFID receiver (1) associated with the object of interest, finding the tag (II); ç. sending a response to the signal of step (a) by the RFID receiver (1) to the RFID antenna (3); d. computer vision mapping (III) of the region pointed by the RFID system to determine the spatial coordinates (x, y, z) of the location of the RFID receiver (1), within a previously referenced region, based on the response signal received by the antenna (3); and. directing the computer vision system to an analysis region, indicated by the RFID system as the region where the object of interest is located, defined by the spatial coordinates (x, y, z) obtained in step (d); f. computer vision mapping (III) of the analysis region by obtaining and superimposing at least one region of interest (ROI), based on the fiducial marker (2) (IV) corresponding to the object of interest; g. definition of x and y coordinates in the perpendicular plane to the sensor (V) and to the observation point, based on the overlapping of the regions of interest (ROI); and h. measurement using infrared (VI) to determine the linear distance (z) between the sensor of the observation point and the object of interest (VII), using the infrared emitter and receiver. 5. Method of locating spatial coordinates of objects, according to claim 4, characterized by the fact that it comprises the predecessor steps, of system calibration, defined by: a. arrange RFID receivers (1) and fiducial markers (2) in known coordinate locations, from the observation point; B. emitting a signal through the RFID antenna (3); ç. collect the response signals by means of the RFID antenna (3) and an RFID reader (4) from the RFID receivers (1); d. associate values collected by the RFID reader (4) of each RFID receiver (1) to the coordinates (x, y, z) of the respective RFID receivers and store such values in a database; and is. adopt the values stored in step (d) as references in determining the distance between the RFID antenna (3) and RFID receivers (1). 6. Method of locating spatial coordinates of objects, according to claims 4 and 5, characterized by the fact of determining the spatial coordinates (x, y, z), based on the response signals (RSSI) of the RFID receivers (1 ), be given by means of probabilistic analysis, by comparing the values obtained by the response signal (RSSI) of the RFID receiver (1) associated with the object of interest, with the reference values obtained in the system calibration. 7. Method of locating spatial coordinates of objects, according to any one of claims 4 to 6, characterized in that it comprises, yet a step of measuring the linear distance (z) between the point of observation and the object of interest by middle of the infrared emitter and receiver. 8. Method of locating spatial coordinates of objects, according to any one of claims 4 to 6, characterized in that it comprises, yet a step of measuring the linear distance (z) between the observation point and the object of interest by means of estimation by artificial neural networks (ANN) and/or SVR (Support Vector Regression). 9. Use of a system for determining the spatial coordinates of objects indoors, characterized in that said system is as defined in claims 1 to 3 and for making use of said method of locating spatial coordinates as defined in claims 4 to 8.

Images