TWI728469B - Teaching path module for mobile vehicles - Google Patents

Abstract

A teaching path module for a mobile vehicle. The mobile vehicle is a movable robot or a vehicle body. The module includes a visual sensing element and a receiver, an arithmetic unit, and a storage. The receiver receives The instruction command sent by the remote end teaches the path of the mobile vehicle's walking instructions. The visual sensing element is used as a shooting image, and the path of the mobile vehicle is used to continuously shoot images, and the continuous shooting image data is transmitted to the computing unit The arithmetic unit calculates the path to provide a guidance path to the storage, and the storage provides the guidance path to the moving vehicle, thereby providing the moving vehicle for the mover.

Description

Teaching path module for mobile vehicles

The present invention, as a teaching path module for a mobile vehicle, refers to a module that can be installed on a mobile vehicle. The mobile vehicle includes a mobile robot or a vehicle body. The module is used as a mobile vehicle to move to a destination. The guidance module of the location path. The data calculation method of the module is to calculate the characteristic points using the images continuously captured by shooting, and then the computing unit uses the characteristic points to calculate the walking path to the destination, thereby providing the displacement of the mobile vehicle Direction path.

At present, in factories, golf courses, restaurants, etc., it is necessary to use mobile vehicles to replace humans to carry goods or patrol and monitor tasks. At present, the walking path modules of mobile vehicles used in these areas are equipped with sensing elements. There are multiple sensors or corresponding components such as infrared or QR Code on the turning key position of the path, so as to provide the moving path of the mobile vehicle. The sensor components and the corresponding components are matched with information to carry out the displacement of the fixed path. However, such corresponding elements for guiding the path must be located at all necessary turning positions. Therefore, there are a large number of corresponding elements required. If one of the corresponding elements fails, the mobile carrier cannot be moved.

And, when these work areas encounter a fire, or the environment path of the work area changes, the corresponding elements cannot be matched with the sensing elements of the mobile vehicle; and these sensing elements must be set in a fixed position. If the location of the execution environment is changed, these sensing elements must be re-established at the corresponding locations.

GPS is also used for route navigation, but the positioning accuracy of GPS is about a few meters, and if the robot moves indoors, or even shuttles inside and outside buildings or between floors, these GPSs cannot accurately provide mobile vehicles to move and execute. jobs.

Therefore, in order to improve the conventional shortcomings, this creation uses photography to obtain images to create feature point images to obtain the navigation path, and use the image feature points to calculate the guiding path. It does not require powerful calculations and large storage, and can meet the needs of fast calculations. .

The present invention uses visual images to establish teaching path data. The stored data is a fixed path generated by the feature points of the visual image, and only the path data generated by the feature points are stored for the purpose of minimizing the amount of data information.

The position confirmation module of the mobile vehicle of the present invention adopts a guide path to locate the destination, which facilitates quick storage and quick comparison after reading; and provides a calculator to quickly compare the old data in the storage Yes, to update the storage to generate the latest path update.

The teaching path module of the mobile vehicle of the present invention is provided with a driving device and a controller, and the teaching path module is connected to the controller assembled on the mobile vehicle by means of electric circuit or wireless transmission; The teaching path module includes: a visual sensing element, a receiver, an arithmetic unit, and a memory, the receiver is for receiving the message of the teaching path sent by the remote end, and providing the traveling direction of the mobile vehicle to move; And, the visual sensing element circuit is connected to an arithmetic unit, the arithmetic unit is connected to the storage, and the storage system is used for storing and retrieving record path data; and, the visual sensing element includes a camera lens; and, the The visual sensing element is arranged on any surface of the mobile carrier; and, the visual sensing element uses the camera lens to receive and guide the receiver to guide the walking path of the mobile carrier to continuously shoot images, and provide multiple images to be obtained Transmit to the arithmetic unit, the arithmetic unit calculates the feature points obtained from multiple images to obtain the guidance path, and stores the guidance path in the memory. The specific process of this calculation guidance path includes FAST, HARRIS, GFTT, SIFT, etc. Appropriate method, by using this method to take out the two-dimensional feature points in the photo, and use the Fundamental Matrix projection geometry method to use the parallax of the two-dimensional feature points in different photos to calculate the position of the feature points in three dimensions, which is called three-dimensional The feature point and the pose where the mobile vehicle is located, so that the pose of the mobile vehicle and the current two-dimensional feature point and the three-dimensional feature point corresponding to the two-dimensional feature point will construct a corresponding relationship, and the relationship path The data is stored in the memory and will be available for subsequent use; and the memory only records the two-dimensional feature points, three-dimensional feature points generated by the image outline and the path relationship data of the moving vehicle, so it is stored The data space is small, because the path relationship data is more intuitive and simple than the holographic image data, so when reloading, the current position of the mobile vehicle can be quickly compared, so that the mobile vehicle can quickly obtain the location, and therefore follow The guiding path is positioned and taught to proceed to the next walking path to reach the destination; in addition, the storage space required for the establishment of the guiding path data is less, so the reading of the storage is fast, so the current comparison of the mobile vehicle is carried out When positioning, it can have the fastest search and guidance path and more accurate positioning accuracy.

1‧‧‧Visual sensor

2‧‧‧Receiver

3‧‧‧Computer

4‧‧‧Storage

P1, P2‧‧‧Feature points

Pi‧‧‧Three-dimensional feature points

O1, O2‧‧‧Position

The first picture is the operation flow of the module for this creation.

The second picture is a block diagram for the creation and implementation of the game.

The third picture is the corresponding relationship diagram of the creative feature points.

For the teaching path module of the mobile vehicle of the present invention, please refer to the first figure. The mobile vehicle A includes a mobile robot or a conveying machine, and the teaching path module B is assembled in the mobile vehicle A. Built in electronic components, it provides teaching functional components for position recognition and posture of the mobile carrier A. The teaching path module B includes a visual sensor component 1 and a receiver 2 as well as an arithmetic unit 3 and a storage 4, The electronic circuit of the visual sensing element 1 connects the receiver 2 and the arithmetic unit 3, and the electronic circuit of the arithmetic unit 3 is connected to the storage 4; and the connection method can be a direct circuit connection to a circuit on a circuit board, or a network Wireless connection, or other communication connection; the implementation of the wireless connection means that the arithmetic unit 3 can be installed outside the module, as shown in the second figure, the arithmetic unit 3 is disposed on a control host terminal C, by By wireless transmission or network cloud, the calculation path information of the arithmetic unit 3 is transmitted to the receiver 2, and the receiver 2 stores the path information in the storage 4, so that the storage 4 provides access path information. When the mobile vehicle wants to move, the receiver 2 obtains the predetermined path information through the storage 4 and provides it to the mobile vehicle to store the predetermined path information, so that the mobile vehicle A can move according to the designated path; and, the visual sensing Component 1 includes a lens, the photographic lens is installed at any surface position of the mobile carrier A, as a function of shooting the environment where the vehicle is located, the photographic lens continuously shoots several images, and the resulting image is transmitted to The arithmetic unit 3, the arithmetic unit 3 extracts discrete two-dimensional feature points from the two-dimensional image taken by the visual sensor element 1, and finds out the same spatial point in different locations according to the feature of the continuously acquired image The two-dimensional coordinates on the image, due to the parallax caused by the moving image of the moving vehicle, the two-dimensional coordinates of these two-dimensional feature points on each image will be changed accordingly, and the continuously obtained two-dimensional feature point data will be used The projection method calculates the true three-dimensional space position of the feature point in space and the pose of the mobile vehicle itself. The pose refers to the direction the mobile vehicle faces in the three-dimensional space. Commonly used mathematical expressions include Euler angles. , Three-dimensional orthogonal matrix, unit quaternion; and, when the camera module of the visual sensing element is working, it will continuously shoot the image of the current environment of the mobile vehicle, and transmit the shooting data to the computing unit, and repeat the above The method calculates new two-dimensional feature points and three-dimensional feature points and the new pose of the mobile vehicle, and stores these data and relational data in the storage for updating, without the need to rebuild the entire storage 4.

The calculator 3 analyzes that if there are a large number of similar or repeated feature points near a location, after linking the data of the memory, it will automatically compare the partially similar or overlapping feature points or pose relations and other information Delete from storage to reduce data occupying storage space.

The visual sensing element can obtain image data and transmit it to the computing unit through circuits, networks, mobile communications and other methods.

The above arithmetic unit 3 obtains image information, including: the description of the feature point, the two-dimensional coordinates of the feature point in the image, the three-dimensional coordinate of the feature point in space, the position where the mobile vehicle has walked and the current posture, etc., the computing unit 3 will The information is combined into a path file and pre-stored in the memory 4. When the mobile vehicle A is suspended or restarted or loses its position, the computing unit 3 can compare the two-dimensional features extracted from the current captured image according to the contents of the memory 4 Point to calculate the current path position of the mobile carrier A, and send the path position information to the driving device of the mobile carrier A for it to continue moving.

In addition, there may be more than one visual sensor element 1 and the receiver 2 as well as the arithmetic unit 3 and the storage 4.

In addition, the storage 4 is provided in the teaching path module B, and can also be installed in the network cloud or externally connected to the memory of a computer host; and, the data in the storage 4 can be directly used by the mobile vehicle A, or transmitted For use by another mobile vehicle A, the storage 4 can provide files to be transferred to another storage for storage.

The calculation method of the arithmetic unit 3 implementing the module of the present invention is as described in the following examples. First, the optical imaging of the camera taken by the camera will more or less produce aberrations, and these differences will cause errors in the calculation of the spatial position; even the calculation results may diverge; Therefore, before calculating the position, the arithmetic unit 3 should correct the aberration of the image according to the optical characteristics of the mounted camera (for example: the pixel size of the image sensor, the pixel aspect ratio, the paraxial focal length of the optical lens, the difference between vertical and horizontal focal lengths, distortion, etc.) , To restore the true projection position of the feature point on the image.

Continue to implement the calculation and pairing method. At the beginning, the user operates the mobile vehicle at the proximal end to move. According to the path to be traveled, the mobile vehicle is remotely controlled to walk the designated path. When the mobile vehicle starts to move, the visual The sensing element 1 is activated, so continuous shooting starts, and the continuously shot images are provided to the arithmetic unit 3, and the arithmetic unit 3 finds two-dimensional feature points. The two-dimensional feature points of each two images are found to be the same Use the fundamental matrix and multiple sets of feature point matching, usually 8 groups but not limited, to restore the robot pose and the three-dimensional position of the paired feature points, and use this geometric relationship to calculate all the feature points in the image Out of the position in space.

Also, as shown in the second figure, the user can wirelessly send a control signal to the receiver 2 at the remote control host terminal C, and the receiver 2 provides the control information to the built-in driving element of the mobile vehicle A (Not shown in the figure), and then the driving element drives the mobile carrier A to move according to the remote control, whereby the mobile carrier A walks the specified path once, and the visual sensor element 1 is activated when the mobile carrier starts to move. Started, so the continuous shooting starts, and the continuously shot images are provided to the calculator 3, and the calculator 3 calculates the path data file from the image data, and stores the path data file in the storage 4.

亦可簡化表示如下：[x,y,z,ψ,θ,φ] As shown in the third figure, among the many feature point pairs found in each two images I1 and 12, when a feature point Pi in space is seen by the two images I1 and 12, the camera center is in place at this time. For pose O1 and pose O2, when the camera is in pose O1, you can see that the projection of the 3D feature point Pi is on the feature point p1; and when the camera is in pose O2, the projection of the 3D feature point Pi is on the feature point p2, and the second The projection when the camera center of the image l2 is projected to the first image l1 is called the pole (epipole) e1, on the contrary, the pole of the pose O1 projected on 12 is e2, and any paired feature points P1 and P2 can correspond The three-dimensional feature point Pi in the real space, the three-dimensional coordinates of the feature point are [x,y,z]t, the pose O1 and the pose O2 of the robot can be calculated by using multiple three-dimensional feature point projections, the relationship is: O2=O1 . [(R‧f)] where,

It can also be simplified as follows: [x,y,z,ψ,θ,φ]

Among them, x, y, z are the three-dimensional position coordinates, ψ , θ , and φ represent the three rotation angles of yaw, pitch, and roll; in order to express the posture of the vehicle in space, the order of operations is first to yaw ( ψ ), followed by pitch( θ ), and finally roll( φ )

In the continuously shot images, any two images can be matched with a certain number of feature points through the above methods, and the calculated spatial position of the feature points, as well as the relative positions and poses of the modules when these two images were taken, these data A structured data file can be constructed continuously and placed in the memory 4 to be stored in the form of a file, which can be reused in the future.

The usual usage method is as follows: the visual sensor 1 transmits the continuously captured scene image to the arithmetic unit 3, and the arithmetic unit 3 resolves it into one or more two-dimensional feature points, usually at least 10 feature points, and then Compare the two-dimensional feature points with the two-dimensional feature points in one or more storages to obtain the closest two-dimensional feature points stored in the storage, and then obtain the position and attitude of the vehicle at that time. It is used to initially locate the current pose, and then calculate and analyze the two-dimensional feature points extracted from the newly captured image, the two-dimensional feature points in the storage and the three-dimensional space position of the feature points in the storage, and the movement can be obtained The current position of the path where the vehicle A is located. For this reason, the mobile vehicle A can continue to move based on this information, and the guidance path file in the storage 4 can also be sent to any mobile vehicle A to let others The mobile vehicle A can use this storage 4 file repeatedly to facilitate work, or upload the guidance path file in the storage 4, when there is another mobile vehicle A that needs to work on the same site, it can be used Remotely or near-end transfers the guidance path file in the storage 4 to the storage 4 in the teaching path module of another mobile vehicle A, thereby providing another mobile vehicle A with the same guidance path that can be walked .

Another implementation of this method is that it can be combined with other sensors as the auxiliary pose data to obtain the path position data source, so as to integrate into a more reliable position estimation, for example: when the image features of the environment are scarce, the result calculated by the above method The error is large. If the path position data can be obtained from other sensors, it will help to correct the error of the image calculation pose; its advantages are:

1. In a specific area, you can create a guide path file on the storage 4, the guide path file is more than one file folder, or the file folders are scattered and built on different storage 4.

2. The memory 4 can be stored as a guide path file, which can be used directly by other mobile vehicle A in the future or transferred to another mobile vehicle A for use.

3. When the environment changes, the guide path file in the storage 4 can be updated.

4. When the guidance path file of the mobile vehicle A exceeds the storage range of the original storage 4, the storage 4 can be expanded.

5. When there are quite similar features or poses in the storage 4, the arithmetic unit 4 will remove some feature point information from the storage 4 after analysis.

6. The arithmetic unit 3 will calculate the feature point data of different feature changes caused by different time, light and shadow, environment..., etc., and will be accessed by the memory 4 to recalculate the feature points at any time.

In summary, these are only preferred embodiments of the present invention and are not used to limit the scope of implementation of the present invention. That is, all equivalent changes and modifications made in accordance with the scope of the patent application of the present invention are covered by the patent scope of the present invention.

1‧‧‧Visual sensor

2‧‧‧Receiver

3‧‧‧Computer

4‧‧‧Storage

Claims (11)

A teaching path module of a mobile vehicle, the mobile vehicle including a mobile robot or a transportation machine, the teaching path module is assembled in a built-in electronic component of the mobile vehicle, and the teaching path module includes a visual sense A sensing element and a receiver and an arithmetic unit and a storage, the visual sensing element electronic circuit connects the receiver and the arithmetic unit, the arithmetic circuit is connected to the storage; and the visual sensing element includes a A photographic lens, which is installed at any surface position of the mobile vehicle as a function of photographing the scenery of the environment where the mobile vehicle is located, and transmits the resulting image to the arithmetic unit. The arithmetic unit obtains the visual perception from the In the two-dimensional image taken by the measuring element, the discrete two-dimensional feature points are extracted, and the two-dimensional coordinates of the same spatial point on different images are found according to the characteristics of the continuously acquired image. Because the moving vehicle is moving The position image produces parallax, and the two-dimensional coordinates of the two-dimensional feature points of the moving vehicle in the moving position image on each image will be changed accordingly, and the two-dimensional feature point data is obtained continuously and the feature is calculated by the method of projection. The real three-dimensional position of the point in space and the pose of the mobile vehicle itself. The pose refers to the direction the mobile vehicle faces in the three-dimensional space. Commonly used mathematical expressions include Euler angles, three-dimensional orthogonal matrix, Unit quaternion; and, the camera module of the visual sensor element will continue to capture images of the current environment of the moving vehicle, and transmit the captured data to the arithmetic unit, and repeatedly calculate the new two-dimensional feature points and The three-dimensional feature points and the new pose of the mobile vehicle are stored, and the data and relational data are stored in the storage for updating, without the need to re-create the entire storage; and, the receiver is a remote one for receiving The control host side wirelessly sends a control signal to the receiver, and the receiver provides the control information to the built-in driving element of the mobile vehicle, and then the driving element drives the mobile vehicle to move according to the remote control, thereby moving the vehicle Walk the designated path, the visual sensing element is activated from the moment the mobile vehicle starts to start, so continuous shooting starts, and the continuously shot images are provided to the arithmetic unit, and the arithmetic unit then takes the image data of the designated path Calculate the path data file, and save the path data file in the memory. For the teaching path module of the mobile vehicle described in the first item of the patent application, the connection method between the receiver and the arithmetic unit can be a direct circuit connection with a circuit on a circuit board, a wireless connection with a network, or other communication connections; The implementation of the wireless connection means that the arithmetic unit can be a control host terminal located outside the module, and the calculation path information of the arithmetic unit is transmitted to the receiver through wireless transmission or network cloud, and then the The receiver stores the path information in the memory, and the memory provides access path information. When the mobile vehicle wants to move, the receiver obtains the stored path information through the memory and provides it to the mobile vehicle. , For the mobile vehicle to move according to the specified path. For example, the teaching path module of the mobile vehicle described in item 1 of the scope of patent application, in a specific area of the storage, more than one guidance path file can be created on the storage, or the file folders can be scattered and built in different On the storage. For example, the teaching path module of the mobile vehicle described in the first item of the scope of patent application, the memory can store multiple guidance path files, which can be directly used by other mobile vehicles in the future or transferred to another mobile vehicle for use . For example, in the teaching path module of the mobile vehicle described in the first item of the scope of patent application, the guidance path file in the storage can be updated when the environment of the storage is changed. For example, the teaching path module of the mobile vehicle described in item 1 of the scope of patent application, the memory can be expanded when the guidance path file of the mobile vehicle exceeds the storage range of the original memory. For example, the teaching path module of the mobile vehicle described in the first item of the scope of patent application, when the memory has quite similar features or poses in the memory, after analysis, the arithmetic unit will convert some feature point information from Remove from storage. For example, the teaching path module of the mobile vehicle described in item 1 of the scope of patent application, the arithmetic unit will calculate the feature point data of different feature changes caused by light and shadow, environment, etc. at different times, and it will be carried out by the memory at any time Access to recalculate feature points. For the teaching path module of the mobile vehicle described in item 1 of the scope of the patent application, there can be more than one visual sensing element and the receiver, and the arithmetic unit and the storage. A teaching path module of a mobile vehicle, the mobile vehicle including a mobile robot or a transportation machine, the teaching path module is assembled in a built-in electronic component of the mobile vehicle, and the teaching path module includes a visual sense The sensing element and an arithmetic unit and a memory, the visual sensing element electronic circuit connects the receiver and the arithmetic unit, the arithmetic circuit is connected to the memory; and the visual sensing element includes a photographing lens, the The photographic lens is installed at any surface position of the mobile vehicle as a function of photographing the environment in which the mobile vehicle is located, and the resulting image is transmitted to the arithmetic unit, which captures the image from the visual sensor element In the two-dimensional image, the discrete two-dimensional feature points are extracted, and the two-dimensional coordinates of the same spatial point on different images are found according to the characteristics of the continuously acquired image. Parallax occurs, the two-dimensional coordinates of these two-dimensional feature points on each image will be changed accordingly, and the two-dimensional feature point data is continuously obtained using the projection method to calculate the true three-dimensional space position of the feature points in the space and The pose of the mobile vehicle itself. The pose refers to the direction the mobile vehicle faces in three-dimensional space. Commonly used mathematical expressions include Euler angles, three-dimensional orthogonal matrices, and unit quaternions; and, the visual sense The camera module of the test component will continue to capture images of the current environment of the mobile vehicle, and transmit the captured data to the arithmetic unit, and repeatedly calculate the new two-dimensional feature points and three-dimensional feature points and the new position of the mobile vehicle according to the above method. Posture, and store these data and relational data in the storage for updating, without the need to re-create the entire storage; from the moment when the mobile vehicle starts to start, the visual sensing element is activated, so it starts to continue. Shoot, and provide the continuously shot images to the calculator, and the calculator calculates the path data file from the image data shot by the specified path, and stores the path data file in the memory. For example, the teaching path module of the mobile vehicle described in item 10 of the scope of patent application, the teaching path module is provided with a receiver, which connects the visual sensing element and the arithmetic unit; and, the receiver can be connected externally A driving element assembled in the mobile vehicle; and a control host terminal that receives the remote sends a control signal to the receiver wirelessly, and the receiver provides the control information to the built-in driving element of the mobile vehicle, and then drives the element The mobile vehicle is driven to move according to the remote control, whereby the mobile vehicle is remotely controlled to walk a designated path, and the image captured by the visual sensor element is transmitted to the computing unit.

Images