RU2597517C1 - Method of in-room navigation using mobile device and graphic marks - Google Patents

Abstract

FIELD: navigation.

SUBSTANCE: invention relates to navigation. Method of in-room navigation comprises placing marks in the form of monochromatic or colour two-dimensional images with preset format, which are graphic codes of their individual numbers and represent encoded two-dimensional codes of their number, on any horizontal or vertical surfaces inside premises, preferably so, that in any direction of view the camera lens of the mobile device can get at least one mark, composing a diagram of marks arrangement in each specific room, composing a list of indicators with assigning an identifier to each indicator, composing a table containing two columns of the marks numbers, which compose unique combinations of the marks numbers in rows of the first and the second columns, including those marks numbers combinations, where the number of a mark from the first column is equal to the number of a mark from the second column, in the third column - identifier of the indicator pointing at the direction of the shortest path from the initial point in the room associated with a mark with a number from the first column to the target point associated with a mark with a number from the second column, herewith if the numbers in the first and the second columns coincide, the indicator identifier is used, which denotes achieving the target, installation of software on the user mobile device equipped with a video camera, selection of a destination point from the list of all points of the room, in which there is at least one mark, overview of the room by means of the mobile device with the software running and receiving an image from the mobile device camera providing output of the obtained image on the display, processing the obtained image, detection of graphical marks of the preset format compensating the perspective, decoding marks, obtaining the numbers of recognized marks, the graphical indicator output on the display of the user mobile device by overlaying the image obtained from the mobile device an image of the graphical indicator pointing at the direction of the shortest path to the point in the room where the target mark is.

EFFECT: technical result is efficient in-room navigation.

1 cl, 10 dwg

Description

Technical field

The present invention relates to the field of navigation, and in particular to a method for navigating indoors and in limited open areas.

State of the art

A known method for determining the location of an object using global navigation satellite systems is that they receive signals from the first GPS and second GLONASS orbital constellations of navigation satellites; primary digital processing of navigation signals with the allocation of ephemeris and pseudorange; corrected pseudo-ranges are calculated; predicting the coordinates of the satellites according to the actual ephemeris at time instants of radiation of navigation signals; determine the coordinates x, y, z, the time parameter a1 corresponding to the divergence of the clock of the navigation receiver from the system time of the first orbital group, and the time parameter a2, corresponding to the difference of the clock of the navigation receiver from the system time of the second orbital group using the least squares method, using information from all satellites with the same weight of the first and second orbital groupings, by analytically solving a system of nonlinear equations with minimizing residuals; verify the validity of the condition a1 <ε, where ε is the permissible time difference between the scale of the navigation receiver and the system time of the global navigation satellite system (GNSS); bring the navigation receiver clock to the GNSS system time; smooth the results of the Kalman filter determination; write down the results of determining x, y, z, a1, a2 and display them for the user (patent RU 2 492 499, publ. 09/10/2013).

Known satellite navigation system for confined space, including: a) at least one station for receiving satellite signals located outside the confined space, designed to receive satellite signals transmitted by a set of satellites, and determine the information corresponding to the transmitted signals related to synchronization time; b) at least one local transmitting station located within a given confined space, designed to transmit a local signal compatible with transmitted satellite signals; and c) a communication channel for transmitting information related to self-consistency between each station and at least one other station; wherein each local transmitting station uses information related to self-consistency to form said local signal; and wherein at least one local signal provides navigation information suitable for use by a receiver of a satellite navigation system (patent application RU 2013 105 411, publ. 08.20.2014).

Satellite navigation systems allow you to expand the functionality, but are not applicable in rooms where satellite signals do not reach properly, and is not economically feasible to solve the problem.

A known method of magnetic navigation along geomagnetic sections, containing a reference map of the Earth’s magnetic field of the investigated area, measuring the Earth’s magnetic field from the carrier, the coordinates of the starting point and a regular system of approximate coordination of the carrier, characterized in that geomagnetic sections are constructed from the reference and measured fields, according to the selected parameters section (coordinates of objects, magnetic masses and distribution in the context of the similarity function of theoretical anomalies of objects to real ones) in a given observation interval and the specified depth interval exclude false and interfering objects in the specified intervals, compare the sections obtained from the measured and reference magnetic fields of the Earth by calculating a certain functional such as the correlation function and search for the extremum of this function and determine the exact coordinates of the carrier (patent RU 2530778, publ. 10.10.2014) . The described method, in the framework of the task of providing navigation in the room, is characterized by high complexity of implementation and low accuracy, since there are always many dynamically changing magnetic anomalies in the room (for example, wiring).

Closest to the invention in technical essence and the achieved result in the field of the method is the method described in the article "Using two-dimensional barcodes to create a positioning and navigation system in the room" (Abdrakhmanova AM, Namiot D.E. Using two-dimensional barcodes for creating a positioning and navigation system in the room. Abstracts of the XX International Scientific Conference of Students, Graduate Students and Young Scientists "Lomonosov-2013". M: Moscow State University named after MV Lomonosov, 2013). According to the method, the system is implemented in the form of a website storing data about each tag. The user adds information about a particular tag using the site’s content management system, noting its location on the layout of the room. Information about each tag is represented by a separate web page, the address of which is encoded by a two-dimensional bar code. The barcode is located at that point in the room, which is indicated on the diagram by the appropriate label. The user, using the software on his mobile device, reads the bar code and goes to the page with information about the tag, where its position on the diagram is indicated. The method is simple and easy to implement and maintain, it can be used on all models of modern mobile phones, however, it requires an Internet connection, cannot be used for continuous navigation, because for barcode recognition it means stopping the user, does not mean calculating the optimal route to destination

Disclosure of invention

The objective of the present invention is to develop a simple, economical way to navigate in a room that does not use radio signals, does not require an Internet connection, can be used for navigation during movement and solves the problem of choosing the optimal route.

The technical result of the present invention is a navigation method that does not require the detection of the current position of the object by means of radio signals, but solves the problem of choosing a further direction of movement and finding the shortest path. The introduction of the system requires the preliminary development of a table-diagram based on the layout of the room, as well as the manufacture and placement of labels, the position of which can remain unchanged throughout the functioning of the room.

The technical result is achieved by the fact that indoors and on limited equipped areas, on any horizontal or vertical surfaces, place special labels, make a label placement scheme, a list of pointers with an identifier for each pointer, a table that contains unique combinations of label numbers - the original and target, as well as identifiers of pointers, indicating the direction of the shortest path from the current position in the room to the target, set a special program e software on a user’s mobile device equipped with a video camera launches the application on a mobile device, selects a destination from the list of all the locations in which at least one label is located, review the room using a mobile device with the software turned on, which receives the image from camera of a mobile device, displays the received image on the screen, processes the received image, the detection of the original graphic labels of a given shape that, perspective compensation, decoding of marks, obtaining the numbers of recognized marks, overlaying on top of the image on the screen of a mobile device received from the camera, an image of a graphic pointer indicating the direction of the shortest path to the place in the room where the target mark is located.

Embodiments of the invention

The method is as follows.

In a room or in limited equipped areas, labels are placed on any horizontal or vertical surfaces. A label is a monochrome or color two-dimensional image, which is a graphic code of its individual number (for example, a two-dimensional bar code). Labels are printed on paper or any other medium, or applied directly to the surface. The size and number of tags placed empirically is selected based on the area of the room and the distance required for tag recognition. The best result is achieved when at least one mark falls into the camera lens for any direction of view. The layout of labels in each particular room is drawn up.

Next, they compile a list of direction indicators, assigning each identifier its own identifier.

Next, make up a table containing three columns: in two columns (column 1 and column 2) indicate label numbers, in the third (column 3) - identifier of the pointer. Each row in columns 1 and 2 contains a unique combination of label numbers, including combinations where the number in column 1 is equal to the number in column 2. Column 3 contains a pointer identifier indicating the direction of the shortest path from the place in the room where the label with the number is located from column 1, to the place in the room where the label with the number from column 2 is located. In the rows where the number in column 1 is equal to the number in column 2, the identifier of the pointer indicating the achievement of the goal is indicated. The numbering of the columns in the table does not matter.

Special software (hereinafter referred to as software) is installed on a user's mobile device equipped with a video camera. The user launches the application on a mobile device, selects a destination from the list, which contains all the locations in which at least one label is located.

The user reviews the room using a mobile device with the software turned on. The software receives the image from the camera of the mobile device and displays it on the screen. The software performs processing of the received image, detects graphic marks of a given format, compensates for the perspective, performs decoding of marks, receives the numbers of recognized marks. The software overlays the image of the graphic pointer on top of the image received from the camera of the mobile device. A graphic pointer is displayed on the screen of the user's mobile device, indicating the shortest path to the place in the room where the target mark is located.

In the process of the software, the user, inspecting the room using a mobile device and following the signs on the screen, reaches the goal.

A brief description of the drawings.

FIG. 1 - floor plan.

FIG. 2 - a plan for placing tags in a room.

FIG. 3 - table of identifiers of pointers.

FIG. 4 - table of directions.

FIG. 5 - label with position cells.

In the drawing, the lines separating the cells are plotted to demonstrate the method. When printing, line marks were not displayed.

FIG. 6 - image from a smartphone camera.

FIG. 7 - binarized decoded area.

FIG. 8 is the result of label recognition.

FIG. 9 - image from a smartphone with a cursor over it.

FIG. 10 - image from a smartphone with a pointer over it when it reaches the goal.

The invention is further described in the following non-limiting examples of the invention.

Example 1

For the experimental premises with an area of 560 square meters. (Fig. 1) compiled a label placement scheme (Fig. 2), compiled a list of pointers, assigned an identifier to each pointer (Fig. 3), prepared a table (hereinafter referred to as the Direction Table) containing 3 columns: in the columns 1 and 2, the label numbers were placed , in column 3 placed the identifier of the pointer. The table was compiled in such a way that each row in columns 1 and 2 contains a unique combination of label numbers, including combinations where the number in column 1 is equal to the number in column 2. Column 3 contains the identifier of the pointer indicating the direction of the shortest path from the room , where the label with the number from column 2 is located, to the place in the room where the label with the number from column 1 is located. In the rows where the number in column 1 is equal to the number in column 2, the identifier of the pointer indicating achievement of the goal was indicated (Fig. 4) .

Next, we prepared graphic labels, which are an encoded two-dimensional code of their number, namely, a square matrix consisting of square cells, each of which is uniformly painted black or has the color of the surface on which the label is applied. The cell shaded in black was assigned the value “1”, the cell not shaded was assigned the value “0”. We chose the binary dimension of the label number: 15 bits, which allows you to prepare 2 ¹⁵ different labels. For noise immunity of the graphic code of the label, the label number was encoded according to the Hamming method.

Given the redundancy of the Hamming code, the total length of the label number code was 30 bits. Added the checksum used during decoding to verify the correctness of the recognized code - 6 bits. The checksum was also encoded using the Hamming method. Given the redundancy of the Hamming code, the final length of the checksum code was 12 bits. We got the total code length (label number + checksum) - 42 bits.

The selected size of the graphic label was 11 * 11 cells, while the outer cells were painted black and formed a closed loop. Then placed the inner row of unfilled cells. Received the internal field of the graphic label size 7 * 7 cells.

The color of seven cells (position cells) was fixed on the internal field to determine the orientation of the label (indicated by the “x”, Fig. 5).

Each label was printed on a separate sheet of A4 format, while maintaining the proportions of the label. The tags were placed according to the layout of tags on the vertical surfaces of the experimental room. Just posted 32 tags.

On a mobile device (smartphone) Sony Xperia with installed Google Android 4.4 operating system installed software (software). While in room No. 7, we launched the application on a mobile device, chose a destination from the list, which contains all the places where at least one label is located, specifically room No. 9, in which the label with number 1105 is located. We started the navigation process to the target label. Surveyed using a mobile device with the software turned on. The software received the image from the camera of the mobile phone and displayed it on the screen, processed the image received from the camera: it switched the image mode in grayscale. After changing the mode on the received frame, a search for closed loops of the "closed square" type was performed (Fig. 6). Then they removed the perspective of the decoded area (the part of the frame limited by the found closed loop), binarized the decoded area, namely, the threshold method (50% threshold) replaced the color of each pixel of the decoded area with one of two colors: white or black (Fig. 7). The decoded area was divided into cells: 11 vertically and 11 horizontally. The values of the code bits and positional cells of the recognized label were obtained: for each cell, the number of black pixels was calculated. If this number exceeds half of the total number of pixels in the cell, then we take the cell value equal to "1", otherwise - equal to "0" (Fig. 8). We determined the orientation of the decoded area by position cells - the orientation is basic, you do not need to rotate the decoded area. We built the code from the values of the cells of the decoded area. Got the following code:

"001011101101001011111000000000111000100110".

The code was decoded according to the Hamming method with error correction, and the corrected label number was obtained in binary form: "00001111101111"; corrected checksum value: "101111". The checksum was checked: the checksum was calculated (the remainder of dividing by 26) and compared with the corrected checksum value of the correct code:

00001111101111/01000000 = 101111

The value of the calculated checksum coincided with the value of the recognized part of the code corresponding to the checksum, therefore, it was assumed that the label number was recognized correctly.

We converted the binary code of the number to decimal, got the number 1007. In the Direction Table, we searched for a line where the value in the first column is equal to the number of the target label 1105, the value in the second column is equal to the recognized label number 1007. By the value of the third column, we got the identifier of the pointer: 3.

A graphic image of the pointer corresponding to the identifier found was superimposed on the original image from the camera in the place where the original closed loop was obtained (Fig. 9).

We followed the premises in the direction of the sign.

Continued to inspect the room for the following signs.

Further frame-by-frame image analysis and display of pointers on the screen occurred in a similar way.

Followed the signs on the screen of the mobile device in the room, until they received a pointer on the screen indicating the achievement of the goal (Fig 10).

Industrial applicability

The method of indoor navigation using graphic labels is widely used in navigation in any buildings with many indoor facilities, such as shopping and entertainment centers, airports, exhibition centers, car parks, as well as in limited equipped areas, such as amusement parks, zoos.

Claims (1)

A method of navigating indoors, including placing labels in the form of monochrome or color two-dimensional images that have a given format, are graphic codes of their individual numbers and represent a coded two-dimensional code of their number, on any horizontal or vertical indoor surfaces, preferably so that for any direction of view, at least one mark got into the camera’s lens of the mobile device, drawing up the layout of the marks in each specific room nii, compiling a list of pointers with assigning an identifier to each pointer, compiling a table containing the label numbers in two columns that form unique combinations of label numbers in the rows of the first and second columns, including those label number combinations in which the label number from the first column is equal to the label number from the second column, in the third column - the identifier of the pointer indicating the direction of the shortest path from the source location in the room, indicated by a label with the number from the first column, to the target location, marked with a label with a number from the second column, and if the numbers in the first and second columns coincide, the identifier of the pointer is used to indicate the achievement of the goal, installation of software on the user's mobile device equipped with a video camera, selection of a destination from the list of all the locations in which at least one label, viewing the room using a mobile device with the software turned on, receiving the image from the camera of a mobile device producing displaying the received image on the screen, processing the received image, detecting graphic labels of a given format, performing perspective compensation, decoding the labels, obtaining the numbers of recognized labels, displaying a graphic pointer on the screen of the user's mobile device by overlaying on top of the image received from the camera of the mobile device a graphic image a pointer indicating the direction of the shortest path to the place in the room where the target mark is located.

Images