JP7025276B2 - Positioning in urban environment using road markings - Google Patents

Description

The present invention relates to position identification in an urban environment using road markings.

With cognitive technology, motion planning technology, control technology and / or new sensing technology, the development of self-driving cars is remarkable. Accurate localization is used to realize autonomous navigation. When using the Global Positioning System (GPS), the multi-path effect in an urban environment can cause inconvenience. Other techniques may be used for positioning in environments where GPS is inconvenient.

In the position identification, the sensor observation result is collated with a known map in advance. Maps are created by human surveying or robotic mapping using different sensors. Cameras and photodetection and range-finding (LiDAR) devices are two common cognitive sensors. LiDAR is generally used for mapping because it can perform accurate distance measurement. A commonly used technique is to use LiDAR for both the mapping process and location.

However, the cost of LiDAR is extremely high for widespread use. On the other hand, although the camera is low cost and lightweight, visual mapping is difficult because it cannot directly measure the distance. The challenge is to collate the measurements against maps constructed by different sensing methods.

Considering the above, there is a need for a method that can more accurately identify the position of the autonomous traveling vehicle. The following disclosures will reveal further effects and new features.

This section is provided to provide a simplified introduction to the set of concepts further described in the detailed description below. This section is not intended to identify important features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

In one aspect of the invention, a positioning method is provided. In the method, an image is generated from the camera in a certain posture, a pre-made map is received, features are determined from the generated image based on edge detection, and at least based on the pre-made map. , The posture of the camera is predicted, and the feature is determined from the predicted posture of the camera. The method further determines the chamfer distance based on features determined from the image and the predicted camera orientation, and the determined chamfer distance is optimal based on odometry information and epipolar geometry. Is made. When the optimization is performed, a camera attitude estimate is generated based on the optimization.

In another aspect of the invention, a positioning system is provided. The system includes a camera that produces an image in a certain posture, a memory, and a processor connected to the memory. The system receives a pre-made map, determines features from the generated image based on edge detection, and predicts the attitude of the camera based on at least the pre-made map. The system further determines features from the predicted camera pose and determines the chamfer distance based on the image and the features determined from the predicted camera pose. The system further optimizes the determined chamfer distance based on odometry information and epipolar geometry, and generates camera attitude estimates based on the optimized data.

In another aspect of the invention, a non-transient computer readable recording medium is provided. The non-transient computer-readable recording medium stores a program. When the program is executed by a circuit of the system, the system produces an image from the camera in a certain posture. The system receives a pre-made map, determines features from the generated image based on edge detection, and predicts the attitude of the camera based on at least the pre-made map. The system further determines features from the predicted camera pose and determines the chamfer distance based on the image and the features determined from the predicted camera pose. The system further optimizes the determined chamfer distance based on odometry information and epipolar geometry, and generates camera attitude estimates based on the optimized data.

These and other aspects of the present disclosure will be better understood by reviewing the detailed description below.

New features that are believed to be features of the present disclosure are described in the appended claims. In the following description, similar parts are designated by the same number throughout the specification and drawings, respectively. Drawings are not always drawn to a constant scale, and some are shown in exaggerated or generalized form for clarity and brevity. However, the disclosure itself, and preferred modes of use, additional objectives and effects, when read in conjunction with the accompanying drawings, will be best understood by reference to the following detailed description of exemplary embodiments of the disclosure. ..

FIG. 1 is a schematic diagram of an exemplary operating environment of a positioning system based on optimization according to an embodiment of the present invention. FIG. 2 is a diagram showing an example of a map element used for position identification according to an embodiment of the present invention. FIG. 3 is a diagram showing an example of a map element stored in a text format according to an embodiment of the present invention. FIG. 4 is a conceptual diagram showing the overall architecture of the positioning system based on the optimization according to the embodiment of the present invention. FIG. 5 is a flowchart showing an example of an object detection method and a position specifying method based on optimization according to the embodiment of the present invention. FIG. 5A is a diagram showing an example of a map projected on a camera view that executes a camera posture obtained by the optimization according to the embodiment of the present invention. FIG. 6 is a system diagram illustrating various hardware components and other features for use according to an embodiment of the present invention. FIG. 7 is a block diagram of various exemplary system components used according to embodiments of the present invention.

Below are definitions of the selected terms used herein. Their definitions include various examples and / or forms of components that are within the scope of the term and can be used to implement them. These examples are not limited to.

As used herein, a "processor" processes a signal and performs common computational and arithmetic functions. Signals processed by the processor can include digital signals, data signals, computer instructions, processor instructions, messages, bits, bitstreams, or other arithmetic results that are received, transmitted, and / or detected.

As used herein, "bus" refers to an interconnect architecture that is operably connected to transfer data between computer components within a single or multiple system. The bus can be, in particular, a memory bus, a memory controller, a peripheral bus, an external bus, a crossbar switch, and / or a local bus. The bus may be, in particular, a vehicle bus that interconnects elements within the vehicle using protocols such as Controller Area Network (CAN), Local Interconnect Network (LIN).

As used herein, "memory" can include volatile memory and / or non-volatile memory. The non-volatile memory may include, for example, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable PROM) and EEPROM (electrically erasable PROM). Volatile memories include, for example, RAM (random access memory), synchronous RAM (SRAM), dynamic RAM (RAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM) and / or direct rambus RAM (DRRAM). ) Can be included.

As used herein, an "operable connection" is a connection in which the substance is "operably connected", in which a signal, a physical communication, and / or a logical communication is used. Can be sent and / or received. Operable connections can include physical interfaces, data interfaces and / or electrical interfaces.

As used herein, "vehicle" refers to any mobile vehicle powered by any form of energy. Vehicles can carry human occupants or cargo. The term "vehicle" includes, but is not limited to, vehicles, trucks, vans, minivans, SUVs, motorcycles, scooters, boats, surface bikes, aircraft, for example. In some cases, a vehicle may include one or more engines.

Positioning is performed in autonomous navigation of self-driving cars. The essence of the locating process is to collate the sensor measurements with a given map. Maps can be generated by human surveying or robotic mapping with different sensors. In other words, the positioning technique may be classified according to the measurement method and the map representation.

The first type of locating technique is to utilize the same type of sensor for both locating and mapping. By using the same type of sensor for both processes, the matching problem can be greatly simplified. For example, 3D-LiDAR (eg Velodyne) can be used for both processes. This is because 3D-LiDAR performs high-precision distance measurement. 3D-LiDAR is first performed to map the road surface and then can locate the vehicle by correlating ground reflectance. In another example, 3D-LiDAR can be used to generate a 3D map represented by Gaussian mixing, and location identification is done by registering 3D point clouds in the map.

The second type of positioning technology uses a low-cost, lightweight camera to visually locate using visible landmarks. For example, a database of visible landmarks can be created from scale-invariant feature transformations (“SIFT”) points. Then, the camera may be positioned by SIFT matching. Another example is to use bag-of-words technology to locate the camera by collating the current image with an image database.

There are disadvantages to using a camera for both location and mapping. First, since the camera does not have the ability to see distance information, it is difficult to obtain high accuracy in visual mapping and at the same time positioning and mapping techniques (SLAM). Second, the quality of visual coincidence in location can be easily influenced by changes in time, viewpoint and lighting.

The above limitations can be overcome by using different detection methods in locating and mapping. For example, by adopting a low-cost sensor (for example, a camera) for position identification and using a high-cost sensor (for example, LiDAR) for map production, it is possible to improve the estimation of the camera posture.

The rationale for using different detection methods for locating and mapping is that the map needs to be very accurate, but not as often as locating / updating. Therefore, a monocular camera can be used to locate the camera itself within the map generated by the highly accurate LiDAR.

FIG. 1 is a schematic diagram of an exemplary operating environment 100 of a position identification system 110 for performing position identification in an urban environment using road markings according to an embodiment of the present disclosure. The position identification system 110 can be resident in the vehicle 102. The components of the relocation system 110, as well as the components of the other systems, hardware architectures, and software architectures described herein, can be combined, omitted, or configured in various forms.

The vehicle 102 generally has an electronic control unit (ECU) 112 that controls the operation of a plurality of vehicle systems. The vehicle system is not limited, but includes, for example, a position identification system 110 including a monocular position identification system 140, and particularly a vehicle HVAC system, a vehicle audio system, a vehicle video system, a vehicle infotainment system, a vehicle telephone system, and the like. include. The monocular positioning system 140 includes a monocular camera 120, or other imaging device (eg, a scanner) that can be connected to the ECU 112 to provide an image of the environment surrounding the vehicle 102 described in detail below. Can be done.

The monocular positioning system 140 can also include a LiDAR sensor data unit 122 captured by a mapping company and composed of various elements such as road markings, curbs, traffic signs and the like. Further, as described in more detail below, the epipolar geometry unit 146, the odometry data unit 144, and the chamfer distance unit 142 are utilized for optimization. You can also do it.

The monocular positioning system 140 is also described in detail below for estimating and evaluating the camera orientation based on the inputs of the epipolar geometry unit 146, the odometry data unit 144 and the chamfer distance unit 142, but the optimization unit 148. Can be included.

The positioning system 110 can also include a camera 120, a LiDAR sensor data unit 122, a communication device 130, and a processor 114 and a memory 116 that communicate with the autonomous driving system 132.

The ECU 112 includes an internal processing memory, an interface circuit, and a bus line for transferring data, transmitting commands, and communicating with the vehicle system. The ECU 112 can include an internal processor and memory (not shown). The vehicle 102 may also include a bus for internally transmitting data between various components of the locating system 110.

The vehicle 102 is a communication device 130 for providing wired or wireless computer communication using various protocols for internally transmitting / receiving electronic signals to features and systems within the vehicle 102 as well as external devices. Further includes (eg, wireless modem). These protocols include radio frequency (RF) communications (eg, IEEE802.11 (Wi-Fi), IEEE802.5.1 (Bluetooth®)), Near Field Communication (NFC) (eg, ISO13157), and more. Local area networks (LANs), wireless wide area networks (WWAN) (eg, cellular) and / or point-to-point systems can be included. Further, the communication device 130 of the vehicle 102 is an internal computer via a bus (eg, CAN or LIN protocol bus) to facilitate data input and output between the electronic control unit 112 and the vehicle features and systems. It may be operably connected for communication. In one embodiment, the communication device 130 may be configured for vehicle-to-vehicle (V2V) communication. For example, V2V communication can include radio communication over a reserved frequency spectrum. As another example, V2V communication can include vehicle-to-vehicle ad hoc networks configured using Wi-Fi or Bluetooth®.

The vehicle 102 has at least one camera 120. The camera 120 may be a digital camera capable of capturing one or more images or image streams, or may be another image capture device such as a scanner. The camera 120 can provide an image of the space directly in front of the vehicle 102. Other cameras can provide an image of another space surrounding the vehicle 102. For example, the rear camera may be placed on the bumper of the vehicle. The camera 120 may be a monocular camera or may provide an image in 2D.

The vehicle 102 includes an automatic driving system 132 for controlling the vehicle 102. The autonomous driving system 132 may include, in particular, a lane keeping support system, a collision warning system, or a fully autonomous driving system. The automatic driving system 132 can receive object position and direction information from the position specifying system 140. In one embodiment, the positioning system 140 may be a component of the automated driving system 132.

FIG. 2 shows an example of map elements generated from a map used for locating. The map may be provided by a map making company (eg, Google) and is composed of various map elements including road markings, curbs, traffic signs and the like. For example, for simplification, two types of map elements have been determined and used as road markings (solid and dashed lines (“road markings”)). Yet another type of map element can be used as a pavement marking. The present disclosure is not limited to only two types of map elements.

As shown in FIG. 2, when determining road markings, solid lines arise from map elements such as lane or pedestrian crossing boundaries, and dashed lines usually exist between lanes. For example, selecting this subset of map elements (eg, solid and dashed lines) as road markings can be beneficial to the system for a variety of reasons. First, this subset of map elements is observed more often than other map elements (eg, speed limit signs, directional arrows, etc.). Second, this subset of map elements is relatively easy to detect from the image due to its characteristic appearance (as opposed to curbs) and its large size (compared to traffic signs).

As shown in FIG. 2, the map is not generated by a camera located in the vehicle 102, but by another detection device (eg, 3D-LiDAR). The LiDAR sensor data obtained by the vehicle 102 may be stored in the LiDAR sensor data unit 122 as shown in FIG. Maps can be constructed by manually labeling landmarks within a 3D environment created by recording a 3D-LiDAR point cloud. As shown in FIG. 2, the map consists of sparse 3D points representing road markings.

FIG. 3 shows an example of road markings stored as text. As shown in FIG. 2, road markings as sparse points are briefly stored in a text file and grouped by geographic location. As shown in FIG. 3, the pavement marking is represented by a set of 3D sparse points sampled along the center line, along with other information such as width and color. These text files are stored in the LiDAR sensor data unit 122, as shown in FIG.

FIG. 4 is a conceptual diagram showing the overall architecture of the location identification system 110. As described in detail below, FIG. 4 shows a camera view detected by the camera 120 and a 3D map acquired by the detector and stored in the LiDAR sensor data unit 122. The monocular position specifying system 140 detects edges and features from the camera view, and the camera attitude prediction is determined from the 3D map.

As mentioned above, chamfer matching is performed to register the edges and map elements detected from the camera view on a lightweight 3D map where the road markings are represented as a set of sparse points. .. In addition, vehicle odometry and epipolar geometry constraints may be considered. Further, a nonlinear optimization problem may be formulated in order to estimate and evaluate the attitude of a 6 degree of freedom (“DoF”) camera.

Further, the positioning system 110 can also detect a matching failure and reset the system after the matching fails, as described in detail below.

FIG. 5 is a flowchart showing an example of a position specifying method according to the embodiment of the present disclosure. In step 502, the relocation system 110 is initialized, as system initialization is described in detail below. At step 504 (time k), the edges of the map element are detected in the image I _k obtained from the camera 120. At the same time (time k), in step 505, the camera orientation P'k is estimated / predicted using the information in the last frame P _{k -1} and the odometry data D _k .

In step 506, edges are detected by extracting contours obtained from the camera view. However, a typical edge detector has too many irrelevant edges (ie, false positives) and trains a random forest-based edge detector based on the camera view.

Random forests are a group of independent decision trees. Each tree is given the same input sample, which is classified by propagating the tree from the root node to the leaf node. By giving a large number of input and output mappings to the initial unlearned decision tree, the parameters of its internal partitioning function gradually evolve to generate similar input-output mappings. This learning process is made possible by defining information gain criteria. The parameters that lead to maximum information acquisition are rewarded. In this way, as shown in FIG. 4, the road marking is detected from the camera view through the feature detection based on the random forest.

In step 509, matching is performed based on the 3D map. As mentioned above, road markings are represented by a small set of 3D points. At time _k , the camera attitude P'k is predicted from the odometry information. As shown in FIG. 4, a small set of 3D points of road markings is projected onto the image space.

In step 510, chamfer matching is performed to evaluate how well the projection points determined in step 509 match the features detected in step 506, and the camera posture is evaluated.

Chamfer matching basically associates each projection point with the nearest edge pixel. As described below, the chamfer distance can be efficiently calculated from the following chamfer distance conversion (1). To take the orientation into account, the edge pixels are divided into different groups according to their gradient direction and the distance transformation is calculated accordingly.

C _k is a distance conversion calculated from the edges of the image. For any point x on I _k , the chamfer distance C _k (x) is queried by completion from C _k . π (P, X) is a projection function that projects the 3D point X from the frame to the image of the attitude P. M _k is a set of road marking points in the camera view with the expected camera attitude _P'k .

Further, for example, when there is a straight solid line in the view, the road marking does not always impose sufficient restrictions on the evaluation of the camera posture. Therefore, optimization may be necessary.

ただし、Ｆは、基礎行列であり、チルダ（～）付きｘは、ｘの同次座標である。キャリブレーションされたカメラ１２０に対して、Ｆは、以下のように、２つのビュー間の相対姿勢によって決定される。

In step 512, the epipolar constraint is determined by the equation (2) described below. x _{i, k-1} ← → x _{i, k} is a pair of image points from I _k-1 to I _k . They correspond to the same 3D point. Epipolar constraints are as follows.

However, F is a basic matrix, and x with a tilde (~) is a homogeneous coordinate of x. For the calibrated camera 120, F is determined by the relative orientation between the two views as follows.

ただし、［^k-1ｔ_k］× は、^k-1ｔ_kとの外積の行列表示である。 Verification is performed so that { _k ^-1 R _k , ^k-1 tk} is a relative rigid transformation between P _k-1 and P _k . The basic matrix is calculated as follows.

However, [ ^k-1 tk] _× is a matrix display of the outer product with _k ^-1 tk.

Given the set of point correlations between I _k-1 and I _k {x _{i, k-1} ⇔ x i _{, k} , i = 1, ...}, the epipolar constraint is defined as follows: To.

Accelerated robust feature (SURF) points are used in epipolar constraints. As mentioned above, since the physical scale cannot be observed with a monocular camera, the equation (5) only imposes a constraint on the five dimensions of the camera posture. Therefore, odometry is used for additional constraints.

At step 514, the odometry constraint is determined. D _k is the rigid transformation between I _k-1 and I _k measured by an odometer. Since the epipolar constraint already covers the 5th dimension, it is only necessary to use the magnitude of the translational movement of D _k as the constraint, as described above. d _k indicates the magnitude of the translational movement component of D _k . The odometry constraint is defined as follows.

In step 516, the optimum conditions are formulated. Given P _k-1 , P _k are evaluated by minimizing the following cost function.

The cost function (7) determines the optimum conditions and is solved using the Lebenberg-Marcat algorithm.

In step 518, the optimized data is used to determine the evaluation of the camera posture. The evaluation of the camera posture is performed on the map.

FIG. 5A shows an example of a map projected onto a camera view that performs estimation of camera orientation obtained by optimization according to an embodiment of the present invention.

In order to initialize the monocular position specifying system 140, an initial estimated value of the camera posture is determined. The estimated camera orientation is obtained from GPS or another type of source. Initial camera pose estimates may be far from their true position in the optimization that should be performed properly. Therefore, a thorough search is used to determine more accurate estimates. To do so, the monocular positioning system 140 randomly samples a large set of candidate poses around the initial estimates in the parameter space. The monocular positioning system 140 finds updated camera attitude estimates that minimize C (P _k ).

When the monocular positioning system 140 is initialized with the optimum candidate as the initial solution, the system further minimizes C (P _k ) as described above.

Further, the monocular position specifying system 140 may monitor the position specifying performance by checking the chamfer distance. A large chamfer distance represents a position-specific assessment. If a large chamfer distance is continuously generated, the monocular positioning system 140 is determined to have failed (eg, improper determination of camera orientation). If the determined monocular positioning system fails, the system performs a reset for initialization using the same strategy as described above.

As mentioned above, the difference between reset and initialization is that the monocular positioning system 140 does not start from an unknown state, but samples candidates around the current pose estimate.

The embodiments of the present disclosure can be performed using hardware, software, or a combination thereof, and can be performed on one or more computer systems or other processing systems. In one embodiment, the disclosure is performed on one or more computer systems capable of performing the functions described herein. FIG. 6 shows an exemplary system diagram of various hardware components and other features for use in accordance with embodiments of the present disclosure. The embodiments of the present disclosure can be performed using hardware, software, or a combination thereof, and can be performed on one or more computer systems or other processing systems. In one variant, embodiments of the present disclosure are directed to one or more computer systems capable of performing the functions described herein. An example of such a computer system 600 is shown in FIG.

The computer system 600 includes one or more processors (eg, processor 604). Processor 604 is connected to the communication infrastructure 606 (eg, communication bus, crossover bar, or network). Various aspects of the software are described in terms of this exemplary computer system. Reading this description will reveal to those skilled in the art how to implement the embodiments of the present disclosure using other computer systems and / or architectures.

The computer system 600 includes a display interface 602 that transfers graphics, text, and other data from the communication infrastructure 606 (or a frame buffer (not shown)) for display on the display unit 630. The computer system 600 also includes a main memory 608, preferably a random access memory (RAM), and may also include a secondary memory 610. The secondary memory 610 includes, for example, a hard disk drive 612 and / or a removable storage drive 614 (floppy disk drive, magnetic tape drive, optical disk drive, etc.). The removable storage drive 614 reads from the removable storage unit 618 and / or writes to the removable storage unit 618 by a well-known method. The removable storage unit 618 is a floppy disk, a magnetic tape, an optical disk, or the like, and is read by the removable storage drive 614 and written to the removable storage drive 614. As will be appreciated, the removable storage unit 618 includes computer software and / or a computer-usable storage (recording) medium that stores the data.

In another aspect, the secondary memory 610 may include a computer program or other similar device to allow other instructions to be loaded into the computer system 600. Such devices can include, for example, a removable storage unit 622 and an interface 620. Such examples include program cartridges and cartridge interfaces (such as those found in video game devices), removable memory chips (erasable programmable read-only memory (EPROM), or programmable read-only memory (PROM), etc.). , Corresponding sockets, and other removable storage units 622, and an interface 620 that allows software and data to be transferred from the removable storage unit 622 to the computer system 600.

The computer system 600 may also include a communication interface 624. The communication interface 624 allows software and data to be transferred between the computer system 600 and an external device. Examples of the communication interface 624 include a modem, a network interface (Ethernet card, etc.), a communication port, a personal computer memory card International Association (PCMCIA) slot, a card, and the like. The software and data transferred via the communication interface 624 take the form of a signal 628 such as an electronic signal, an electromagnetic signal, an optical signal or another signal that can be received by the communication interface 624. These signals 628 are supplied to the communication interface 624 via a communication path (eg, channel) 626. This path 626 carries the signal 628 and is carried out using wires or cables, fiber optics, telephone lines, cellular links, radio frequency (RF) links and / or other communication channels. In this specification, the terms "computer program medium" and "computer-usable recording medium" are used to collectively refer to media such as hard disks and signals 628 installed in removable storage drives 680 and hard disk drives 670. These computer program products provide software to the computer system 600. Embodiments of the present invention are directed to such computer program products.

Computer programs (also called computer control logic) are stored in main memory 608 and / or secondary memory 610. The computer program can also be received via the communication interface 624. When such a computer program is executed, the computer system 600 implements various features according to embodiments of the invention, as described herein. In particular, the computer program causes the processor 604 to perform such features when executed. Therefore, such a computer program corresponds to the controller of the computer system 600.

In a variant in which the embodiments of the present invention are implemented using software, the software is stored in a computer program product and uses a removable storage drive 614, a hard drive 612, or a communication interface 620 into a computer system 600. Loaded. When executed by the processor 604, the control logic (software) causes the processor 604 to perform the functions according to the embodiments of the present invention described herein. Another variant is implemented primarily in hardware, using hardware components such as, for example, application specific integrated circuits (ASICs). Implementation of a hardware state machine to perform the functions described herein will be apparent to those of skill in the art.

Yet another variant is implemented using a combination of both hardware and software.

FIG. 7 is a block diagram of various exemplary system components that may be used according to embodiments of the present invention. For example, various components may be present in the vehicle 102, or only some of the components may be in the vehicle 102 while the other components may be distant from the vehicle 102. The system 700 includes one or more accessors 760,762 (also referred to herein as one or more "users") and one or more terminals 742,766 (such terminals are eg, eg. , Which may or may be various features of the object detection system 110). In one embodiment, the data for use in accordance with embodiments of the invention is via a network 744 (eg, the Internet or intranet, etc.) and coupling devices 745, 746, 764, a processor and a data storage location and / or data storage. Terminals 742,766 (eg, personal computer (PC), minicomputer, main) coupled to a server 743 (eg, PC, minicomputer, mainframe computer, microcomputer, or other device, etc.) that has a connection to a location. It is input and / or accessed, for example, by a person 760, 762 who accesses it via a frame computer, a microcomputer, a telephone device, a wireless device such as a personal digital assistant (“PDA”) or a handheld wireless device). Examples of the coupling device 745, 746, 764 include a wired, wireless, optical fiber link, and the like. In another variant, the methods and systems according to embodiments of the invention operate in a stand-alone environment such as a single terminal.

The embodiments described herein may be described and practiced in the context of a computer-readable storage (recording) medium for storing computer executable instructions. Computer-readable storage media include computer storage media and communication media. For example, a flash memory drive, a digital versatile disc (DVD), a compact disc (CD), a floppy disk, a tape cassette, or the like. Computer-readable storage media include volatile and non-volatile, removable media and non-removable media implemented by any method or technique for storing information such as computer-readable instructions, data structures, modules or other data. Possible media are listed.

Various other embodiments of the features and functions disclosed above, and other features and functions thereof, or alternatives or variants thereof, may be optionally combined to form many other different systems or applications. Will be understood. Also, various substitutions, modifications, changes, or improvements that are intended to be included in the following claims, but are not currently predicted or expected, shall continue to be made by those skilled in the art. Can be done.

Claims (21)

The steps to generate an image from the camera in a certain posture,
Steps to receive a pre-made map and
A step of determining features from the generated image based on edge detection,
A step of predicting the posture of the camera based on at least the pre-made map and determining a feature from the predicted posture of the camera.
A step of determining a chamfer distance based on the feature determined from the image and the predicted camera posture.
A step of optimizing the determined chamfer distance based on odometry information and epipolar geometry.
A step to generate a camera attitude estimate based on the optimization,
Positioning method with. The method according to claim 1, wherein the image is generated from a monocular camera. The method according to claim 1, wherein the pre-made map is made based on LiDAR detection. The method according to claim 1, wherein the optimization is performed by minimizing the cost function defined below.

However, P _k is the estimated posture of the camera, _Codm is the odometry information, C _epi is the epipolar geometry, and P _k-1 is the posture of the camera at the previous time. Yes, k is time. The method according to claim 4, wherein the odometry information is defined by the following formula.

Where d _k is the magnitude of the translational movement component of D _k , D _k is the rigid transformation between the image of the previous time and the image, and t _k is the translational movement vector. The method according to claim 4, wherein the epipolar geometry is defined by the following equation.

However, x with a tilde (-) is the homogeneous coordinate of the point on the image, and F is a basic matrix. The method according to claim 1, wherein the edge detection is performed by an edge detector based on a random forest. A camera that produces an image in a certain posture,
With memory
With the processor connected to the memory
It is a positioning system that has
The processor
Receive a pre-made map
Based on edge detection, features are determined from the generated image and
At least the posture of the camera is predicted based on the map prepared in advance, and the feature is determined from the predicted posture of the camera.
A chamfer distance is determined based on the features determined from the image and the predicted camera orientation.
Optimize the determined chamfer distance based on odometry information and epipolar geometry.
A positioning system characterized by generating camera attitude estimates based on the optimized data. The position identification system according to claim 8, wherein the camera is a monocular camera. The position identification system according to claim 8, wherein the map prepared in advance is created based on LiDAR detection. The location identification system according to claim 8, wherein the optimization is performed by minimizing the cost function defined below.

However, P _k is the estimated posture of the camera, _Codm is the odometry information, C _epi is the epipolar geometry, and P _k-1 is the posture of the camera at the previous time. Yes, k is time. The position identification system according to claim 11, wherein the odometry information is defined by the following equation.

Where d _k is the magnitude of the translational movement component of D _k , D _k is the rigid transformation between the image of the previous time and the image, and t _k is the translational movement vector. The position identification system according to claim 11, wherein the epipolar geometry is defined by the following equation.

However, x with a tilde (-) is the homogeneous coordinate of the point on the image, and F is the basic matrix. The position identification system according to claim 8, wherein the edge detection is performed by an edge detector based on a random forest. If the non-transient computer readable recording medium is a non-transient computer readable recording medium that stores a program and the program is executed by a circuit of the system, the system will:
Generate an image from the camera in a certain posture and
Receive the pre-made map and receive
Based on edge detection, features are determined from the generated image and
At least the posture of the camera is predicted based on the map prepared in advance, and the feature is determined from the predicted posture of the camera.
A chamfer distance is determined based on the features determined from the image and the predicted camera orientation.
Optimize the determined chamfer distance based on odometry information and epipolar geometry.
A non-transient computer-readable recording medium characterized by generating camera attitude estimates based on the optimized data. The non-transient computer-readable recording medium according to claim 15, wherein the image is generated from a monocular camera. The non-transient computer readable recording medium according to claim 15, wherein the preliminarily prepared map is produced based on LiDAR detection. The non-transient computer readable recording medium according to claim 15, wherein the optimization is performed by minimizing the cost function defined below.

However, P _k is the estimated posture of the camera, _Codm is the odometry information, C _epi is the epipolar geometry, and P _k-1 is the posture of the camera at the previous time. Yes, k is time. The non-transient computer-readable recording medium according to claim 18, wherein the odometry information is defined by the following equation.

Where d _k is the magnitude of the translational movement component of D _k , D _k is the rigid transformation between the image of the previous time and the image, and t _k is the translational movement vector. The non-transient computer-readable recording medium according to claim 18, wherein the epipolar geometry is defined by the following equation.

However, x with a tilde (-) is the homogeneous coordinate of the point on the image, and F is a basic matrix. The non-transient computer-readable recording medium according to claim 15, wherein the edge detection is performed by an edge detector based on a random forest.

Images