TWI668464B - Positioning method for mobile electronic apparatus and positioning system - Google Patents

Abstract

A positioning method and positioning system for a mobile electronic device. The positioning tag is photographed through an image capturing device provided in the mobile electronic device. The positioning tag includes at least three designated points. According to the positioning label, at least three coordinate information of the specified point in the world coordinate system is obtained. The target position of the image capturing device in the world coordinate system is calculated based on the coordinate information of at least three specified points and the image obtained by capturing the positioning tag through the image capturing device.

Description

Positioning method and positioning system of mobile electronic device

The present invention relates to a positioning mechanism, and in particular to a positioning method and positioning system for a mobile electronic device.

It is only necessary to quickly arrange the machine, equipment station and actual use in a strange factory environment, and then carefully measure and calculate various possibilities. The traditional layout method relies on the engineering team to design the layout according to the factory blueprint, and to pull, pipe and arrange the machine position. With the increasing number of machines configured by the manufacturers, the entire plant is full of various equipment, and any part of the adjustment will involve other machines.

The factory is an indoor space, so the traditional Global Positioning System (GPS) cannot be used as a positioning technology in the factory. Although there is an ultra-wideband (UWB) indoor positioning technology, because the factory includes various sophisticated machines, if UWB's indoor positioning technology is adopted, its wireless signal will affect the operation of the machine in the factory.

The invention provides a positioning method of a mobile electronic device, which can be accurately positioned by simply configuring a positioning tag and an image capturing device.

The positioning method of the mobile electronic device of the present invention comprises: capturing a positioning tag through an image capturing device disposed in the mobile electronic device, wherein the positioning tag includes at least three designated points; and obtaining at least three designated points in the world according to the positioning tag Coordinate information in the coordinate system; and calculating the target position of the image capturing device in the world coordinate system based on the coordinate information of at least three specified points and the image obtained by capturing the positioning tag through the image capturing device.

In an embodiment of the invention, the positioning method further includes: detecting whether a positioning tag exists in a visible range of the image capturing device.

In an embodiment of the invention, the step of detecting whether the positioning tag exists in the visible range of the image capturing device comprises: capturing the target image; and using the object detection algorithm to frame the candidate object in the target image; Whether the feature data of the candidate object meets the preset feature data; and when the feature data meets the preset feature data, it is determined that the positioning tag exists.

In an embodiment of the invention, the step of obtaining the coordinate information of the at least three specified points according to the positioning label comprises: connecting to the server according to the positioning label, and obtaining at least three corresponding to the positioning label from the server Specifies the coordinate information of the point.

In an embodiment of the invention, coordinate information of at least three designated points is recorded in the positioning tag.

In an embodiment of the present invention, the positioning method further includes: scanning a specified space by using a stereo modeling system to establish a spatial model; determining a plurality of label positions in the designated space, wherein the label positions are respectively used to set a plurality of positioning labels Scanning a plurality of positioning tags to obtain a relative position of at least three specified points included in each of the positioning tags; and querying the world coordinate system to set coordinate information of at least three specified points.

In an embodiment of the present invention, the step of scanning a specified space by the stereo modeling system to generate a spatial model includes: dividing the designated space into a plurality of regions; and scanning the regions by using a stereo modeling system to establish a plurality of small block models; And merging these small block models to produce a spatial model.

In an embodiment of the invention, the step of calculating the target position of the image capturing device in the world coordinate system according to the coordinate information of the at least three specified points comprises: utilizing the image and at least three specified points based on a camera imaging principle Coordinate information, calculating a transformation matrix; calculating the length of the side length based on the actual side length of the positioning label and the imaging side length of the positioning label imaged in the image; using coordinate information, side length ratio, and image capturing of at least three specified points A focal length of the device to calculate a relative coordinate of the imaging device relative to the positioning tag; and converting the relative coordinate to a target position of the world coordinate system based on the transformation matrix.

In an embodiment of the invention, the positioning tag is a graphic code.

The positioning system of the present invention comprises: a positioning tag comprising at least three designated points; and a mobile electronic device comprising: an image capturing device and a processor. The image capture device is used to capture the positioning tag. The processor is coupled to the image capturing device, and the positioning label is photographed by the image capturing device, and the coordinate information of at least three specified points is obtained according to the positioning label, and the coordinate information according to at least three specified points in a world coordinate system and the transmission information are obtained. The image obtained by the positioning camera is imaged by the device to calculate the target position of the image capturing device in the world coordinate system.

Based on the above, setting a positioning tag in the indoor space to assist positioning can solve the problem that the conventional Global Positioning System (GPS) cannot be used indoors, and the use of the image capturing device and the positioning tag can solve the problem of using the wireless signal. Positioning affects the operation of the machine.

The above described features and advantages of the invention will be apparent from the following description.

1 is a block diagram of a positioning system in accordance with an embodiment of the present invention. Referring to FIG. 1 , the positioning system 10 includes a mobile electronic device 100 , a positioning tag 150 , and a server 160 . The mobile electronic device 100 is an electronic device having computing power, such as a robot. The mobile electronic device 100 can be applied to a drone or an unmanned vehicle to implement the transportation automation of the factory. The following positioning method can avoid the interference caused by the use of the radio signal in the factory and affect the production. In addition, in other embodiments, the mobile electronic device 100 can also be a notebook computer, a tablet computer, a digital camera, a smart phone, or a smart watch. The mobile electronic device 100 includes an image capturing device 110, a processor 120, a storage device 130, and a communication device 140. The processor 120 is coupled to the image capturing device 110, the storage device 130, and the communication device 140.

Here, the mobile electronic device 100 is connected to the server 160 via the Internet 170 using the communication device 140. The server 160 and the communication device 140 are selectively configured as needed. That is, in other embodiments, the server 160 may not be included in the positioning system 10, and the mobile electronic device 100 may not include the communication device 140. The communication device 140 is, for example, a network card, a WiFi chip, a mobile communication chip, or the like.

The image capturing device 110 may be a camera, a camera, or the like using a charge coupled device (CCD) lens or a complementary metal oxide semiconductor transistor (CMOS) lens.

The processor 120 can be a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a programmable microprocessor (Microprocessor), It is implemented by an embedded control chip, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or the like.

The storage device 130 can adopt any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory, and security. A Digital Card (Secure Digital Memory Card, SD), a hard disk, or other similar device or a combination of these devices is implemented. The storage device 130 stores a module consisting of one or more code segments, and the processor 120 executes the module to implement the following positioning method.

2 is a flow chart of a method for locating a mobile electronic device 100 in accordance with an embodiment of the present invention. Referring to FIG. 1 and FIG. 2 , in step 205 , the mobile electronic device 100 captures the positioning tag 150 through the image capturing device 110 . In this embodiment, the positioning tag 150 can be implemented by using a sticker manner, and the positioning tag 150 can be a graphic code. The graphic code may be a two-dimensional barcode. For example, the positioning tag 150 is a Quick Response Code (QR code). In other embodiments, the positioning tag 150 is not limited to the QR code, and a Han Xin Code may also be used.

Next, in step S210, the mobile electronic device 100 obtains coordinate information of three designated points in the world coordinate system according to the positioning tag. Here, the positioning tag 150 includes at least three designated points. The specified point may be a specific graphic included in the positioning tag 150. Taking the QR code as an example, three square points printed on three corners of the QR code can be used as the designated points. In addition, at least three specific patterns may be additionally set on the positioning tag 150 as designated points. The number of designated points is at least three. In other embodiments, four or more specified points may be set, which is not limited herein.

Then, in step S215, the mobile electronic device 100 calculates the target position of the image capturing device 120 in the world coordinate system based on the coordinate information of the three designated points and the image obtained by the image capturing device 110 capturing the positioning tag 150. For example, the processor 120 calculates the relative coordinates of the image capturing device 120 with respect to the positioning tag 150 using the coordinate information of at least three specified points, the aspect ratio, and the focal length of the image capturing device 120. The side length ratio is obtained by the processor 120 capturing the positioning label 150 through the image capturing device 110 to obtain an image, and then calculated according to the imaging side length imaged by the positioning label 150 in the image and the actual side length of the positioning label 150.

After obtaining the relative coordinates of the image capturing device 120 with respect to the positioning tag 150, for example, using the intrinsic parameter, the extrinsic parameter of the image capturing device 110, and the coordinate information of at least three specified points, the calculation can be performed. The coordinate position (world coordinates) of the image capturing device 120 is taken out. That is, based on the principle of camera imaging, the conversion matrix is calculated by capturing the image obtained by the positioning tag 150 by the image capturing device 120 and the coordinate information of at least three specified points. The relative coordinates are then converted to the target position of the world coordinate system based on the transformation matrix.

Generally, the world coordinates are converted into the camera coordinates of the image capturing device 110 by the external parameters of the image capturing device 110, and the camera coordinates of the image capturing device 110 are converted into the captured images by the projection of internal parameters. The pixel coordinate of the image. Accordingly, the processor 120 can use the captured image to derive the world coordinates.

First, the image capturing device 110 is used to capture an object (for example, the positioning tag 150) positioned in the three-dimensional world coordinates. The image capturing device 110 converts the world coordinates into camera coordinates using the following formula (1).

Equation (1) P _c = R (P _w - C) where C = (c _x , c _y , c _z ), which is the origin in the world coordinate system.

In formula (1), P _w is the world coordinate of the object, P _w = (X, Y, Z); P _c is the camera coordinate of the object, P _c = (X _camera , Y _camera , Z _camera ), R is A 3 × 3 rotation matrix (r ₁₁ to r ₃₃ ), t is a 3 × 1 displacement matrix (t _{x ,} t _{y ,} t _z ). The rotation matrix R and the displacement matrix t constitute an external parameter matrix. Equation (1) can be converted into the following formula (2).

Formula (2)

After converting the object into an object world coordinate P _W P _C camera coordinates, the resulting object coordinate P _C is the position of the camera focal length f 120 of the apparatus with respect to imaging, this can be another object of the camera coordinate P _{c is} converted to the imaging plane coordinates of the image capturing device 120, that is, the screen coordinates (x, y) of the image. In the pinhole camera model (Pinhole Camera Model), the camera coordinates P _c (X _camera , Y _camera , Z _camera ) are projected on the screen coordinates (x, y) of the imaging plane, as shown in equation (3).

Formula (3)

Next, the two-dimensional pixel coordinates (x, y) are represented by three-dimensional coordinates (x', y', z'), as shown in the formula (4), where x=x'/z'; y=y' /z'.

Formula (4)

Since the origin of the screen coordinates is at the center, but the origin of the pixel coordinates is at the upper left corner, the offset is corrected by the following formula (5). Where (o _x , o _y ) is the center coordinates of the screen, and f is the focal length of the image capturing device 120.

Formula (5)

Next, the camera coordinates are converted to pixel coordinates as described below. Where (s _x , s _y ) represents the actual size of the pixel, the unit of which is, for example, mm, and f _x and f _y are focal lengths in units of pixels. Among them, the internal parameter matrix = ; external parameter matrix = .

Based on the above, it can be known that the camera coordinates of the object = external parameter matrix × world coordinates of the object; pixel coordinates = internal parameter matrix × camera coordinates of the object. Therefore, the pixel coordinates = internal parameter matrix × external parameter matrix × world coordinates of the object.

Accordingly, to obtain the world coordinates from the pixel coordinates, the pixel coordinates are first converted to screen coordinates, then the screen coordinates are converted to camera coordinates, and then the camera coordinates are converted to world coordinates. To convert from pixel coordinates to camera coordinates, the pixels of the image must be undistorted using internal parameters and distortion coefficients to return them to the camera coordinates. Next, map the camera coordinates back to the world coordinates.

The external parameter matrix [R|t] is: The conversion matrix describing the posture of the image capturing device 120 is [R _c |C]. Based on the above:

According to this, by using the above-described conversion matrix [R _c |C] to inversely calculate the relative coordinates of the image capturing device 120, the target position of the image capturing device 120 in the world coordinate system can be obtained.

3 is a schematic diagram of a QR code as a positioning tag in accordance with an embodiment of the present invention. Referring to FIG. 3, the QR code is square, and has three square points in three corners of the QR code, that is, designated points A1 to A3. After obtaining an image by capturing the QR code through the image capturing device 110, the processor 120 obtains coordinate information of the designated points A1 to A3 based on the QR code. In the present embodiment, the coordinate information of the designated points A1 to A3 is stored in the remote server 160, and the connection address of the server 160 is written in the QR code. After the processor 120 parses the QR code to obtain the connection address of the server 160, it is connected to the server 160 through the communication device 140 to obtain the coordinate information of the designated points A1 to A3. In other embodiments, the coordinate information of the specified points A1 to A3 may be directly written in the QR code.

The positioning method further includes detecting whether a positioning tag exists in the visible range of the image capturing device 110. The processor 120 continuously captures the target image through the image capturing device 110 and performs analysis to determine whether a positioning tag exists in the visible range. When it is determined that there is a positioning tag, positioning can be performed.

FIG. 4 is a flow chart of a method for detecting a positioning tag of the mobile electronic device 100 according to an embodiment of the invention. Referring to FIG. 4, in step S405, the mobile electronic device 100 frames the candidate object in the target image by using the object detection algorithm. Next, in step S415, the mobile electronic device 100 finds the feature data of the candidate object. For example, the feature data is a frame, a color, a size, a feature point, and the like.

Thereafter, in step 315, the database is read. A plurality of preset feature data are stored in the database. Moreover, in step S320, the mobile electronic device 100 recognizes whether the feature data of the candidate object meets the preset feature data. Here, the preset feature data is, for example, feature data of a QR code. When the feature data of the candidate object meets the preset feature data, as shown in step S425, the mobile electronic device 100 determines that the positioning tag exists in the visible range, and then performs steps S205 to S215, thereby obtaining the image capturing device 110. (ie, mobile electronic device) coordinate position. When the feature data does not meet the preset feature data, as shown in step S430, it is determined that there is no positioning tag in the visible range.

On the other hand, before positioning, the specified space can be scanned with a stereo modeling system to create a spatial module. Taking the internal space of the factory as the designated space as an example, a three-dimensional modeling system is used to scan multiple areas inside the plant one by one, and then a small block model is established. The small block model includes existing wiring, equipment and devices. After obtaining the small block models corresponding to each region, these small models are merged into a large model, that is, a spatial model corresponding to the entire plant.

After the spatial model is established, the object detection algorithm can be used to find the rough position of the object, and then the pattern match is used to find the real position, and the outer frame of the object is marked to establish the corresponding stereo. model. In this way, non-animal and movable objects in the plant can be marked. Use the non-movable object as the layout base, configure the objects that can be moved based on the layout, and simulate the possible results through the simulator to find the optimal space configuration and maximum capacity.

After that, multiple label locations are determined in the designated space (for example, the entire plant area of the plant). These tag positions are used to individually set a plurality of positioning tags. Thereafter, the positioning tags are scanned through the mobile electronic device having the image capturing device to obtain the relative positions of at least three specified points included in each of the positioning tags, and then the world coordinate system is queried to set the coordinates of at least three specified points. Information (world coordinates). Accordingly, the coordinate positions of the plurality of positioning tags in the designated space can be set in the modeling stage, and the coordinate positions are written into the server 160. Then, if the location of the positioning tag is changed, the corresponding data in the server 160 can be modified, and the positioning tags can be reused.

In summary, the present invention can be accurately positioned by simply configuring the positioning tag and the image capturing device. Based on the development of the positioning label, the world coordinate system of the designated space is designed, and the world coordinates of at least three specified points of the positioning label are set, and the difference between the three points and the imaging position of the camera is visually observed to complete the positioning system. In addition, the integration of modeling, object detection, and positioning systems breaks through the limitations of traditional manufacturing line arrangements, and the remaining capacity is concentrated to maximize resource use. The layout of the space arrangement and the use of the factory can be used to achieve an evaluation of the results through simulation of the model, achieving unprecedented results.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

10‧‧‧ Positioning System

100‧‧‧Mobile electronic devices

110‧‧‧Image capture device

120‧‧‧ processor

130‧‧‧Storage device

140‧‧‧Communication device

150‧‧‧ Positioning label

160‧‧‧Server

170‧‧‧Internet

A1~A3‧‧‧ designated points

S205～S215‧‧‧ positioning method steps

S405～S430‧‧‧Methods for detecting positioning tags

1 is a block diagram of a positioning system in accordance with an embodiment of the present invention. 2 is a flow chart of a positioning method in accordance with an embodiment of the present invention. 3 is a schematic diagram of a QR code as a positioning tag in accordance with an embodiment of the present invention. 4 is a flow chart of a method for detecting a positioning tag according to an embodiment of the invention.

Claims (13)

A method for positioning a mobile electronic device includes: capturing a positioning tag through an image capturing device disposed on the mobile electronic device, the positioning tag includes at least three designated points; and obtaining the at least three designations according to the positioning tag Pointing on the coordinate information in a world coordinate system; and calculating the image capturing device in the world coordinate system according to the coordinate information of the at least three designated points and an image obtained by capturing the positioning label through the image capturing device a target position, wherein the step of calculating the target position of the image capturing device in the world coordinate system according to the coordinate information of the at least three specified points and the image obtained by capturing the positioning label by the image capturing device comprises: Calculating a conversion matrix using the image and the three coordinate information of the at least three specified points based on a camera imaging principle; based on an actual side length of the positioning label and an imaging side length of the positioning label imaged in the image Calculating the length ratio of one side; using the coordinate information of the at least three specified points, the ratio of the length of the side, and the taking Means to calculate a focal length of the imaging means with respect to a coordinate of the relative position of the tag; and based on the transformation matrix to convert the relative coordinates in the world coordinate system of the target position. For example, the positioning method described in claim 1 of the patent scope further includes: Detecting whether the positioning tag exists in a visible range of the image capturing device. The positioning method of claim 2, wherein the step of detecting whether the positioning tag exists in the visible range of the image capturing device comprises: capturing a target image; using an object detection algorithm in A candidate object is framed in the target image; whether the feature data of the candidate object meets a preset feature data; and when the feature data conforms to the preset feature data, determining that the positioning tag exists. The positioning method of claim 1, wherein the step of obtaining the three coordinate information of the at least three designated points according to the positioning label comprises: connecting to the server according to the positioning label, and the servo is obtained from the servo Coordinate information of the at least three specified points corresponding to the positioning tag is obtained in the device. The positioning method according to claim 1, wherein the positioning label records coordinate information of the at least three designated points. The positioning method of claim 1, further comprising: scanning a specified space by using a stereo modeling system to establish a spatial model; determining a plurality of label positions in the designated space, wherein the label positions are respectively used Setting a plurality of the positioning tags; scanning the positioning tags to obtain relative positions of the at least three specified points included in each of the positioning tags; and querying the world coordinate system to set the at least three designated points Coordinate information. The positioning method of claim 6, wherein the step of scanning the designated space by the stereo modeling system to generate the spatial model comprises: Dividing the designated space into a plurality of regions; scanning the regions by the stereo modeling system to create a plurality of tile small models; and merging the small block models to generate the spatial model. The positioning method according to claim 1, wherein the positioning tag is a graphic code. A positioning system comprising: a positioning tag comprising at least three designated points; and a mobile electronic device comprising: an image capturing device for capturing the positioning tag; and a processor coupled to the image capturing device, The processor captures the positioning tag through the image capturing device, and obtains coordinate information of the at least three designated points in a world coordinate system according to the positioning label, and according to the coordinate information of the at least three specified points and through the image capturing The device captures an image obtained by the positioning tag to calculate a target position of the image capturing device in the world coordinate system, wherein the processor uses the image and the coordinate information of the at least three specified points based on a camera imaging principle Calculating a conversion matrix; the processor calculates a side length ratio based on an actual side length of the positioning tag and an imaging side length of the positioning tag imaged in the image; the processor utilizes coordinates of the at least three specified points Information, the length ratio of the side, and a focal length of the image capturing device to calculate a relative seat of the image capturing device relative to the positioning tag ; The processor converts the relative coordinates to the target position of the world coordinate system based on the conversion matrix. The positioning system of claim 9, wherein the mobile electronic device detects whether the positioning tag exists in a visible range of the image capturing device. The positioning system of claim 9, further comprising: a server storing coordinate information of the at least three specified points; wherein the mobile electronic device further comprises a wire for connecting to the server And a communication device, wherein the processor uses the communication device to connect to the server according to the positioning tag, and obtain coordinate information of the at least three specified points from the server. The positioning system of claim 9, wherein the positioning label records coordinate information of the at least three designated points. The positioning system of claim 9, wherein the positioning tag is a graphic code.