WO2020042349A1

WO2020042349A1 - Positioning initialization method applied to vehicle positioning and vehicle-mounted terminal

Info

Publication number: WO2020042349A1
Application number: PCT/CN2018/113669
Authority: WO
Inventors: 李天威; 谢国富
Original assignee: 初速度（苏州）科技有限公司; 北京初速度科技有限公司
Priority date: 2018-08-31
Filing date: 2018-11-02
Publication date: 2020-03-05
Also published as: CN110148170A

Abstract

A positioning initialization method applied to vehicle positioning and a vehicle-mounted terminal. The method comprises: constructing a surrounding environment of a vehicle by means of a target image photographed by an image collection device, so as to obtain a local map; matching the local map with a pre-constructed global map to obtain a position of the local map in the global map; and on the basis of the position of the local map in the global map, mapping the position of the vehicle in the local map into the global map to obtain an initial position of the vehicle in the global map. According to the technical solutions, the initial position of the vehicle in the global map can be determined by utilizing image data photographed by the image collection device under the condition that position prior information such as a GPS signal misses, so that initialization of vehicle positioning is completed.

Description

Positioning initialization method and vehicle terminal applied to vehicle positioning

Technical field

The invention relates to the technical field of automatic driving, in particular to a positioning initialization method and a vehicle-mounted terminal applied to vehicle positioning.

Background technique

In the technical solution of automatic driving, when the vehicle is started, the operation of vehicle positioning initialization needs to be performed, that is, determining the initial position of the vehicle in the electronic driving navigation electronic map. Generally, the initial position can be obtained based on prior information of a position such as a Global Satellite Positioning System (GPS) signal. However, in some special environments (such as GPS equipment failure or the vehicle is in an underground garage or tunnel), it is not possible to use position prior information such as GPS signals when initializing vehicle positioning, and it is difficult to perform positioning initialization.

Summary of the Invention

The embodiment of the invention discloses a positioning initialization method and a vehicle-mounted terminal applied to vehicle positioning, which can implement vehicle positioning initialization when a priori information such as GPS signals is missing.

The first aspect of the embodiments of the present invention discloses a positioning initialization method applied to vehicle positioning, the method includes:

Use the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map;

Matching the local map with a pre-built global map to obtain the position of the local map in the global map; wherein, as the vehicle travels, the range of the local map gradually increases. When the range of the local map is increased to be able to match from the global map to the only area that is the same as the local map, the local map is determined to be in the global map according to the position of the area in the global map s position;

Based on the position of the local map in the global map, map the position of the vehicle in the local map to the global map to obtain the initial position of the vehicle in the global map.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, constructing a surrounding environment of the vehicle by using a target image captured by an image acquisition device to obtain a local map includes:

Acquire multiple target images captured by multiple image acquisition devices at the same time; the multiple image acquisition devices include image acquisition devices installed in the front, rear, left, and right directions of the vehicle, each The framing range of the image acquisition device includes at least the ground below the image acquisition device;

Stitching a plurality of said target images to obtain a top-view mosaic image;

Identify image semantic features in the top-view mosaic image, and build a local map based on the image semantic features.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, matching the local map with a pre-built global map to obtain a position of the local map in the global map includes :

Detecting a specific feature in the local map, where the specific feature is a semantic feature of the image whose appearance probability in the local map is lower than the non-specific feature;

Identifying target features in the pre-built global map that match the specific features;

Determining a position of the local map in the global map according to a position of the target feature in the global map.

As an optional implementation manner, in the first aspect of the embodiment of the present invention, before the using the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map, the method Also includes:

Determining whether to record the end position of the positioning at the end of the last positioning calculation, where the last positioning calculation is a previous positioning calculation that occurred before the vehicle was started;

If the positioning end position is recorded, determining the positioning end position as an initial position of the vehicle in the global map;

If the positioning end position is not recorded, the target image captured by the image acquisition device is used to construct the surrounding environment of the vehicle to obtain a local map.

Determine whether a satellite positioning signal is received, and if the satellite positioning signal is received, determine an initial position of the vehicle in the global map based on the satellite positioning signal;

If the satellite positioning signal is not received, executing the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map.

A second aspect of the embodiments of the present invention discloses a vehicle-mounted terminal, including:

A construction unit, configured to construct a surrounding environment of the vehicle by using a target image captured by an image acquisition device to obtain a local map;

A matching unit configured to match the local map with a pre-built global map to obtain a position of the local map in the global map; wherein, as the vehicle travels, the range of the local map gradually increases Large, when the range of the local map is increased to be able to match from the global map to the only area that is the same as the local map, determining the location of the local map based on the location of the area in the global map A position in the global map;

A first determining unit, configured to map a position of the vehicle in the local map to the global map based on a position of the local map in the global map to obtain the vehicle in the global map The initial position in the map.

As an optional implementation manner, in a second aspect of the embodiment of the present invention, the construction unit includes:

An acquisition subunit, configured to acquire multiple target images captured by multiple image acquisition devices at the same moment; the multiple image acquisition devices include image acquisitions respectively installed in front, rear, left, and right directions of the vehicle Device, the framing range of each said image acquisition device includes at least the ground below the image acquisition device;

A stitching sub-unit for stitching a plurality of said target images to obtain a top-view stitching image;

A construction subunit is used for identifying the image semantic feature in the top-view mosaic image, and constructing a local map based on the image semantic feature.

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the matching unit includes:

A detection subunit, configured to detect a specific feature in the local map, where the specific feature is a semantic feature of the image whose appearance probability in the local map is lower than the non-specific feature;

A recognition subunit for identifying a target feature in a pre-built global map that matches the specific feature;

A determining subunit, configured to determine a position of the local map in the global map according to a position of the target feature in the global map.

As an optional implementation manner, in the second aspect of the embodiments of the present invention, the vehicle-mounted terminal further includes:

A first determining unit is configured to determine whether the positioning end position at the end of the last positioning calculation is recorded before the construction unit uses the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map. The last positioning calculation is a previous positioning calculation that occurred before the vehicle was started;

A second determining unit, configured to determine the end position of the positioning as the initial position of the vehicle in the global map when the first determination unit determines that it is recorded to the positioning end position;

The construction unit is specifically configured to use the target image captured by the image acquisition device to construct the surrounding environment of the vehicle when the first determination unit determines that the positioning end position is not recorded, to obtain a local map .

As an optional implementation manner, in the second aspect of the embodiment of the present invention, the specific feature includes a zebra crossing, a lane arrow, and a storage location.

A third aspect of the embodiments of the present invention discloses a vehicle-mounted terminal, including:

Memory storing executable program code;

A processor coupled to the memory;

The processor calls the executable program code stored in the memory to execute any method disclosed in the first aspect of the embodiments of the present invention.

A fourth aspect of the present invention discloses a computer-readable storage medium that stores a computer program, wherein the computer program causes a computer to execute any one of the methods disclosed in the first aspect of the embodiments of the present invention.

A fifth aspect of the embodiments of the present invention discloses a computer program product, and when the computer program product runs on a computer, the computer is caused to execute any method disclosed in the first aspect of the embodiments of the present invention.

Compared with the prior art, the invention and its beneficial effects are as follows:

1. Use the target image captured by the image acquisition device to construct a local map, and match the local map with a pre-built global map to obtain the position of the local map in the global map, so that the position of the local map in the global map Determined as the vehicle's initial position in the global map. It can be seen that, in the implementation of the embodiment of the present invention, the initial position of the vehicle in the global map can be determined by using the image data captured by the image acquisition device when the prior information of the position such as the GPS signal is missing, thereby completing the initialization of the vehicle positioning.

2. Using the top-view mosaic as the input object of the neural network to extract the semantic features is more accurate than the traditional map matching using the forward graph or forward side graph or ring view, so that the positioning is accurate. In addition, stitching the target images first, and then extracting the semantic features of the images from the top-view mosaic, can also improve the extraction efficiency of the semantic features of the images.

3. According to the probability of occurrence in different situations, specific semantic features (such as arrows, storage locations, etc.) in the garage scene are optimized, and the local map is constructed to improve the accuracy of matching the local map with the global map.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. Those of ordinary skill in the art can obtain other drawings according to the drawings without paying creative labor.

1 is a schematic flowchart of a positioning initialization method applied to vehicle positioning disclosed in an embodiment of the present invention;

2 is a schematic flowchart of another positioning initialization method applied to vehicle positioning disclosed in an embodiment of the present invention;

3 is an exemplary diagram of a partial map of a parking lot constructed by a vehicle terminal disclosed in an embodiment of the present invention;

FIG. 4 is another exemplary partial map of a parking lot constructed by a vehicle terminal disclosed in an embodiment of the present invention; FIG.

5 is a schematic structural diagram of a vehicle-mounted terminal disclosed in an embodiment of the present invention;

6 is a schematic structural diagram of another vehicle-mounted terminal disclosed by an embodiment of the present invention;

7 is a schematic structural diagram of another vehicle-mounted terminal disclosed by an embodiment of the present invention;

FIG. 8 is a schematic structural diagram of another vehicle-mounted terminal disclosed in an embodiment of the present invention.

detailed description

In the following, the technical solutions in the embodiments of the present invention will be clearly and completely described with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of them. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

It should be noted that, the terms “including” and “having” and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device containing a series of steps or units is not limited to the listed steps or units, but optionally also includes steps or units that are not listed, or optionally also includes Other steps or units inherent to these processes, methods, products or equipment.

The embodiment of the invention discloses a positioning initialization method and a vehicle-mounted terminal applied to vehicle positioning, which can implement vehicle positioning initialization when a priori information such as GPS signals is missing. Each of them will be described in detail below.

Example one

Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a positioning initialization method applied to vehicle positioning according to an embodiment of the present invention. The method is applied to a vehicle-mounted computer, a vehicle-mounted industrial control computer (Industrial Personal Computer, IPC) and other vehicle-mounted terminals, which are not limited in the embodiment of the present invention. The above-mentioned vehicle-mounted terminal is connected to each sensor of the vehicle, and receives and processes data collected by each sensor. As shown in FIG. 1, the positioning initialization method applied to vehicle positioning may include the following steps:

101. A vehicle-mounted terminal uses a target image captured by an image acquisition device to construct a surrounding environment of a vehicle to obtain a local map.

In the embodiment of the present invention, the image acquisition device may be a camera. For convenience of description, unless otherwise specified, the camera in the following refers to the image acquisition device. The camera is installed on the vehicle and is used to capture the surrounding environment of the vehicle. The vehicle terminal can use local positioning and map construction (Simultaneous Localization and Mapping) technology to build a local map using the target image captured by the camera. The local map is used for Describe the surroundings of the vehicle. Based on SLAM technology, the vehicle-mounted terminal can identify feature points in the target image and use these feature points to build a map. That is to say, for a vehicle-mounted terminal, when the vehicle is in an unknown position in an unknown environment, the vehicle-mounted terminal can use the images captured by the camera to gradually draw a local map of the vehicle's path environment while the vehicle is continuously traveling.

As an optional implementation manner, cameras can be installed in the front, rear, left, and right directions of the vehicle, and the framing range of each camera includes at least the ground below the camera. Optionally, the foregoing camera may be a fish-eye camera, and a field of view (FOV) of the fish-eye camera is relatively large, so that the target image captured by a single fish-eye camera includes as much of the periphery of the vehicle as possible. Environment, improve the integrity of the local map, and increase the amount of information contained in the local map.

102. The vehicle-mounted terminal matches the local map with a pre-built global map to obtain the position of the local map in the global map.

In the embodiment of the present invention, the global map is an electronic map for autonomous driving navigation, and is a digital description of the real geographic environment. Compared with the local map constructed by the vehicle terminal in step 101, the global map has a larger range. Taking the application scenario of the parking lot as an example, the global map may be a map of the entire parking lot, and the local map may be a map including the route of the vehicle in the parking lot and the surrounding environment of the route. It can be seen that for the same geographical environment, the local map is part of the global map. Some features in the local map are the same as those in the global map. By looking for these same features, you can match the local map with the local map. In the same area, the location of the area is the position of the local map in the global map.

It should be noted that the local map may be a map composed of gradually accumulated map fragments. Due to the similarity of features, when the range of the local map is small, there may be multiple areas in the global map that are the same as the local map. At this time, it is difficult to determine the exact location of the local map in the global map. Therefore, the vehicle can continue to drive, and the in-vehicle terminal continuously acquires the target image captured by the vehicle during the driving process, adds the information of the target image to the original local map, and constructs a new local map, which is a process of gradual accumulation. When the vehicle travels far enough and the local map constructed by the in-vehicle terminal is large enough, there is only a high probability that there is only one area in the global map that is the same as the local map, so that the position of the local map in the global map can be accurately determined .

103. The in-vehicle terminal maps the position of the vehicle in the local map to the global map based on the position of the local map in the global map to obtain the initial position of the vehicle in the global map.

In the embodiment of the present invention, when the in-vehicle terminal successfully matches the local map with the global map, it can be understood that the same area as the local map is found from the global map, so that the correspondence between the local map and the global map can be determined. Therefore, based on the above-mentioned correspondence relationship, the position of the vehicle in the local map can be mapped to the global map, and the initial position of the vehicle in the global map can be obtained. Because the global map is a digital description of the real geographic environment, the features in the global map correspond one-to-one with objects in the real geographic environment. When the initial position of the vehicle in the global map is determined, that is, the initial position of the vehicle in the real geographic environment is determined, the vehicle terminal uses only visual information to complete the vehicle's base on the basis of not relying on prior information such as GPS signals. Positioning initialization.

In a possible application scenario, the vehicle is started in an underground garage, and the vehicle-mounted terminal cannot receive the GPS signal at this time, and thus the GPS signal cannot be used to complete the preliminary positioning of the vehicle. In order to complete the positioning initialization, the vehicle travels slowly. The vehicle-mounted terminal controls the camera to capture the surrounding environment of the vehicle to obtain the target image. The target image is used to construct a local map describing the surrounding environment of the vehicle. Increase, when the range of the local map is increased to match the only area that is the same as the local map from the global map, the position of the local map in the global map is determined according to the location of the area in the global map, so as to determine The initial position of the vehicle in the global map completes the initialization of the vehicle's positioning.

It can be seen that in the method described in FIG. 1, the vehicle-mounted terminal uses the target image captured by the camera to construct a local map describing the surrounding environment of the vehicle. After the local map and the pre-built local map are successfully matched, the local map can be used according to the local map. The position in the global map determines the initial position of the vehicle in the global map, so that the vehicle's positioning initialization can be completed using only visual information without relying on prior information such as GPS signals.

Example two

Please refer to FIG. 2, which is a schematic flowchart of another positioning initialization method applied to vehicle positioning disclosed in an embodiment of the present invention. As shown in FIG. 2, the positioning initialization method applied to vehicle positioning may include the following steps:

201. The in-vehicle terminal determines whether the positioning end position at the end of the last positioning calculation is recorded. If yes, step 202 is performed, and if no, step 205 is performed.

202. The vehicle-mounted terminal determines the positioning end position as an initial position of the vehicle in the global map.

In the embodiment of the present invention, the last positioning calculation is a previous positioning calculation that occurred before the vehicle was started. In a possible application scenario, after the vehicle is turned off, the power of the vehicle terminal is cut off, and the vehicle terminal stops the positioning calculation. From the time the vehicle goes out to the next start, the vehicle is likely to have no position change and will not move. When the vehicle starts, positioning initialization is required. At this time, if the positioning end position at the end of the last positioning is recorded, the positioning end position can be directly used as the initial position of the vehicle in the global map, thereby shortening the positioning initialization position. It takes time to improve the user experience.

203. The in-vehicle terminal determines whether a satellite positioning signal is received, and if yes, executes step 204; if not, executes step 205.

204. The vehicle-mounted terminal determines an initial position of the vehicle in the global map based on the satellite positioning signal.

In a possible application scenario, the global map described above may be an underground map used when a vehicle is driving in an underground garage. The above-ground map used by vehicles on the ground and the underground map used by vehicles in underground garages may belong to two different map expression systems. For example, the above-ground map is a three-dimensional map, and the underground map is a two-dimensional map. When a vehicle drives from an above-ground road into an underground garage, it is necessary to complete the switching between the above-ground map and the underground map, and at the same time, it is necessary to complete the positioning initialization based on the underground map. At the entrance of the underground garage, the vehicle terminal may receive the satellite positioning signal. Therefore, the vehicle terminal can directly use the satellite positioning signal to determine the initial position of the vehicle terminal in the global map (that is, the underground map), thereby reducing the time required for positioning initialization To improve user experience.

It should be noted that, in the embodiment of the present invention, there is no logical sequence relationship between step 201 and step 203, and the vehicle-mounted terminal may execute step 201 and step 203 simultaneously. If the in-vehicle terminal receives the satellite positioning signal while determining that the positioning end position is recorded, there may be errors in the satellite positioning result according to the propagation characteristics of the satellite positioning signal. Therefore, the in-vehicle terminal can select the positioning end position as the vehicle on the global map In the initial position.

205. The vehicle-mounted terminal acquires multiple target images captured by multiple cameras at the same moment.

In the embodiment of the present invention, the above-mentioned multiple cameras are cameras respectively installed in the front, rear, left, and right directions of the vehicle, and the framing range of each camera includes at least the ground below the camera. The cameras installed in the above four directions form the camera's surround view solution, so that the vehicle terminal can obtain environmental information in all directions around the vehicle at one time, so that the local map constructed using the target image obtained in a single acquisition contains more The features are beneficial to improve the matching success rate of local maps and global maps. In addition, there is a certain degree of redundancy in the image data collected by the four cameras. If one camera fails, the image data collected by the remaining cameras can be used as a supplement, which has a low impact on the local map construction and positioning of the vehicle terminal.

206. The vehicle-mounted terminal stitches multiple target images to obtain a top-view mosaic image.

In the embodiment of the present invention, the in-vehicle terminal stitches the target images captured by the cameras installed in the front, rear, left, and right directions of the vehicle at the same time, and the resulting top-view mosaic image includes a 360-degree environment centered on the vehicle. information. In addition, if the camera used to capture the target image is the fisheye camera described above, the vehicle terminal needs to perform anti-distortion processing on the target image before performing step 206 to stitch multiple target images, that is, according to a certain mapping rule, the The target image captured by the fisheye camera is projected onto the ground plane, and the images obtained after the projection are stitched together.

207. The in-vehicle terminal recognizes image semantic features in the top-view mosaic image, and constructs a local map based on the identified image semantic features.

In the embodiment of the present invention, the image semantic feature is a semantic feature that can be empirically filtered, has a special meaning, and is helpful for vehicle positioning. In a possible application scenario, the vehicle is located in a parking lot, and the parking lot may be an above-ground parking lot or an underground parking lot, which is not limited in the embodiment of the present invention. In an application scenario of a parking lot, the image semantic features may be lane lines, parking space lines, storage locations (intersection points between the storage space lines), zebra crossings, lane arrows, and the like, which are not limited in the embodiments of the present invention. Please refer to FIG. 3 together. FIG. 3 is an exemplary diagram of a partial map of a parking lot constructed by a vehicle-mounted terminal according to an embodiment of the present invention. As can be seen from FIG. 3, when the vehicle-mounted terminal is driving in the parking lot, The passing lanes, storage lines, storage locations and other semantic features are composed of them. Among them, the dotted line with arrows shows the driving trajectory of the vehicle.

In addition, as an optional implementation manner, in the embodiment of the present invention, the in-vehicle terminal may recognize image semantic features from a top-view mosaic image through an image recognition algorithm such as deep learning or image segmentation. Preferably, a neural network model suitable for deep learning can be used to identify image semantic features, and a large number of top-down mosaic sample images labeled with image semantic features are used to train the neural network model in advance. The neural network model is as follows:

The network structure uses the Encoder-Decoder model, which mainly includes two parts: the Encoder part and the Decoder part.

In the embodiment of the present invention, the stitched image is input into the network, and the coding part of the network mainly extracts features of the image through convolution and pooling layers. The network is trained with labeled large samples, and the network parameters are adjusted to make the coding network accurate semantic and non-semantic features. After the coding network extracts features through two convolutions, it performs downsampling through pooling. By cascading four two-layer convolutions and one pooling structure, the receptive fields of neurons on the top layer of the coding network can cover semantic elements of different scales in the examples of the present invention.

The decoding network is a symmetric structure with the coding network, where the pooling layer of the coding network is changed to the upsampling layer. After upsampling four times in the decoding part, the features extracted from the encoding are enlarged to the size of the original image, thereby achieving pixel semantic classification. Upsampling is achieved by deconvolution. This operation can get most of the information in the input data, but some information is still lost. Therefore, we introduce the underlying features to supplement the details lost during the decoding process. These low-level features are mainly used to encode convolutional layers of different scales in the network. The features extracted by encoding the network convolutional layers on the same scale can be combined with deconvolution to generate more accurate feature maps. Network training mainly uses cross entropy to measure the difference between the predicted value and the actual value of the network. The cross entropy formula is as follows:

Where y is the label value of the image element, that is, whether a pixel of the image is a semantic element or a non-semantic element. Generally, 1 is used for semantic elements and 0 is used for non-semantic elements. N is the total number of pixels in the image, x is the input, and a is the neuron. The output a = σ (z), z = ∑ _j w _j x _j + b, which can overcome the problem that the network weights are updated too slowly. After the training of the network model is completed, when the example of the present invention is actually used, the network predicts each pixel of the input image, and outputs the corresponding attribute value of each pixel as 0 or 1, and the connected block of the image element marked as 1. For meaningful semantic image structure, the image semantic segmentation has been realized so far. The top-view mosaic image obtained by splicing the vehicle terminal is input to the trained neural network model, and based on the recognition result of the neural network model, the image semantic features in the top-view mosaic image can be identified. Compared with the traditional image segmentation technology, the deep learning method can be used to extract the image semantic features from the top-view mosaic, which can improve the recognition accuracy of the image semantic features. The above network structure is specifically designed for the extraction of semantic features of stitched images, and ensures the accuracy of the extraction of semantic features, which belongs to one of the invention points of the present invention. In addition, the target images are spliced first, and then the image semantic features are extracted from the top-view mosaic, instead of extracting the image semantic features in the target image one by one, which can improve the extraction efficiency of the image semantic features, which also belongs to the invention point of the invention One.

208. The vehicle-mounted terminal detects a specific feature in the local map.

In the embodiment of the present invention, the specific feature is an image semantic feature whose appearance probability in the local map is lower than the non-specific feature.

209. The in-vehicle terminal identifies a target feature that matches a specific feature in a pre-built global map.

210. The vehicle-mounted terminal determines the position of the local map in the global map according to the position of the target feature in the global map.

In the embodiment of the present invention, it can be known from the construction process of the local map in step 207 that the local map may include multiple types of semantic features, and different types of semantic features have different probability of appearing in the local map. Taking a parking lot as an example, in the prior art, lane lines and parking space lines are usually selected as semantic features. However, in addition to the above two semantic features, the present invention is concerned that the probability of the zebra crossing and the lane arrow appearing is lower than the lane line and the parking space line, which can be set as specific features for map matching. The use of specific features in the local map for matching between the local map and the global map can increase the probability of successful matching, which is also one of the inventive points of the present invention. In particular, for a site, it is generally considered that its occurrence probability is large, and it cannot be used as a specific feature. However, the present invention is concerned that their distribution densities are not the same in curves and straights. According to this feature, it can also be used as a specific feature to assist in positioning the local map on the global map. This is also one of the invention points of the present invention.

In order to better understand the matching between the local map and the global map through specific features in steps 208 to 210, please refer to FIG. 4 together. FIG. 4 is another exemplary partial map of a parking lot constructed by a vehicle terminal disclosed in an embodiment of the present invention. As shown in FIG. 4, the local map includes three image semantic features of a location line, a location and a lane arrow. If the location line and location are used for matching, it is possible to match multiple areas from the global map with the local map. At this time, the matching accuracy is low. For lane arrows, lane arrows in different positions are also different in shape, size, and location relationship with surrounding location lines and locations. Therefore, using lane arrows to match local and global maps can Increase the probability of a successful match.

211. The vehicle-mounted terminal maps the position of the vehicle in the local map to the global map based on the position of the local map in the global map to obtain the initial position of the vehicle in the global map.

It can be seen that in the method described in FIG. 2, the vehicle-mounted terminal may directly use the positioning end position as a result of positioning initialization when recording the positioning end position, or may determine the positioning initialization based on the satellite positioning signal when receiving the satellite positioning signal. As a result, the time required for positioning initialization can be shortened, and the user experience can be improved. Further, in the method described in FIG. 2, the vehicle-mounted terminal uses four cameras installed around the vehicle to form a camera look-around solution, so that the local map constructed using the target image obtained by a single acquisition contains more features, which is beneficial to The matching between the local map and the global map can also reduce the impact on the local map construction and positioning of the vehicle terminal when some cameras fail. Furthermore, in the method described in FIG. 2, the in-vehicle terminal uses a specific feature with a low probability of occurrence in the local map to perform matching between the local map and the global map, which can increase the probability of successful matching.

Example three

Please refer to FIG. 5, which is a schematic structural diagram of a vehicle-mounted terminal disclosed in an embodiment of the present invention. The vehicle-mounted terminal shown in FIG. 5 is connected to each sensor of the vehicle, and receives and processes data collected by each sensor. As shown in FIG. 5, the vehicle terminal includes:

A constructing unit 501 is configured to construct a surrounding environment of a vehicle by using a target image captured by a camera to obtain a local map.

In the embodiment of the present invention, the camera is installed on the vehicle and used to photograph the surrounding environment of the vehicle. As an optional implementation manner, the construction unit 501 may identify feature points in the target image based on the SLAM technology, and use these feature points to construct a map.

Further optionally, cameras can be installed in front, rear, left, and right directions of the vehicle, and a viewing range of each camera includes at least the ground below the camera. In addition, the foregoing camera may be a fisheye camera, so that the target image captured by a single fisheye camera may include the surrounding environment of the vehicle as much as possible, improve the integrity of the local map, and increase the amount of information contained in the local map.

The matching unit 502 is configured to match the local map constructed by the construction unit 501 with a pre-built global map to obtain the position of the local map in the global map.

In the embodiment of the present invention, the matching unit 502 can find the same feature in the global map as the local map by searching for the same features in the local map and the global map, so that the location of the area is the local map in the global Location in the map.

It should be noted that the local map constructed by the construction unit 501 may be a map composed of gradually accumulated map fragments. Due to the similarity of features, when the range of the local map is small, there may be multiple areas in the global map that are the same as the local map. At this time, it is difficult to determine the exact location of the local map in the global map. Therefore, the vehicle can continue to drive, and the in-vehicle terminal continuously acquires the target image captured by the vehicle during the driving process, adds the information of the target image to the original local map, and constructs a new local map, which is a process of gradual accumulation. When the vehicle travels far enough and the local map constructed by the in-vehicle terminal is large enough, there is only a high probability that there is only one area in the global map that is the same as the local map, so that the position of the local map in the global map can be accurately determined . That is, when the matching unit 502 fails to match the local map and the global map, the construction unit 501 may be triggered to continue to acquire the target image captured by the camera, and continue to use the target image to construct the local map described above.

The first determining unit 503 is configured to map the position of the vehicle in the local map to the global map based on the position of the local map in the global map obtained by the matching unit 502 to obtain the initial position of the vehicle in the global map.

In the embodiment of the present invention, after the first determining unit 503 maps the position of the vehicle in the local map to the global map through the correspondence between the local map and the global map, the first determining unit 503 may determine the initial position of the vehicle in the global map. That is, determine the initial position of the vehicle in the real geographic environment (such as a parking lot).

It can be seen that when the vehicle-mounted terminal shown in FIG. 5 is implemented, the target image captured by the camera can be used to construct a local map describing the surrounding environment of the vehicle. After the local map and the pre-built local map are successfully matched, the local map can be used globally. The position in the map determines the initial position of the vehicle in the global map, so that the vehicle's positioning initialization can be completed using only visual information without relying on prior information such as GPS signals.

Embodiment 4

Please refer to FIG. 6, which is a schematic structural diagram of another vehicle-mounted terminal disclosed by an embodiment of the present invention. The vehicle-mounted terminal shown in FIG. 6 is obtained by optimizing the vehicle-mounted terminal shown in FIG. 5. As shown in FIG. 6, the above-mentioned construction unit 501 may include:

The obtaining subunit 5011 is configured to obtain multiple target images captured by multiple cameras at the same time.

In the embodiment of the present invention, the above-mentioned multiple cameras include at least four cameras respectively installed in front, rear, left, and right directions of the vehicle, and the framing range of each camera includes at least the ground below the camera. The cameras installed in the above-mentioned four directions form the camera's surround view solution, so that the local map constructed using the target image obtained in a single acquisition contains more features, which is beneficial to improving the matching success rate between the local map and the global map. In addition, there is a certain degree of redundancy between the data collected by each camera in the surround view solution, because in the event that a certain camera fails, the collected data of the remaining cameras can be used as a supplement, which can reduce the failure of some cameras to construct the vehicle terminal. Local map and positioning effects.

The stitching sub-unit 5012 is configured to stitch multiple target images acquired by the obtaining sub-unit 5011 to obtain a top-view mosaic image.

In the embodiment of the present invention, if the camera used to capture the target image is a fisheye camera, the stitching subunit 5012 needs to perform anti-distortion processing on the target image before stitching multiple target images, that is, according to a certain mapping rule, The target image captured by the fisheye camera is projected onto the ground plane, and the images obtained after the projection are stitched together.

A construction subunit 5013 is used to identify the image semantic features in the top-view mosaic image obtained by the mosaic subunit 5012, and construct a local map based on the identified image semantic features.

In the embodiment of the present invention, the image semantic feature is a semantic feature that can be empirically filtered, has a special meaning, and is helpful for vehicle positioning. For example, the image semantic feature may be a lane line, a parking space line, a storage location point, a zebra crossing, a lane arrow, and the like, which are not limited in the embodiment of the present invention.

In addition, the construction sub-unit 5013 can recognize image semantic features from a top-view mosaic image through an image recognition algorithm such as deep learning or image segmentation. Preferably, the semantic features of the image can be identified by using a neural network model suitable for deep learning: inputting the top-view mosaic image obtained by splicing the vehicle terminal into the trained neural network model, and the top view can be identified based on the recognition result of the neural network model. Image semantic features in mosaics. Compared with the traditional image segmentation technology, the deep learning method can be used to extract the image semantic features from the top-view mosaic, which can improve the recognition accuracy of the image semantic features.

It can be seen that the above-mentioned construction unit 501 constructs a local map based on the camera's surround view scheme. Compared with the technical solutions of the monocular camera's forward-looking scheme, the construction unit 501 uses a single observation to include more information in the local map, which can shorten positioning. The time required for initialization can also improve the accuracy of positioning.

Optionally, the aforementioned matching unit 502 may include:

The detection sub-unit 5021 is configured to detect specific features in the local map constructed by the construction sub-unit 5013, wherein the specific features mentioned above are image semantic features whose appearance probability in the local map is lower than non-specific features.

In the embodiment of the present invention, the specific feature is an image semantic feature whose appearance probability in the local map is lower than the non-specific feature. It can be known from the specific manner of constructing the local map by the construction unit 501 that the local map may include multiple types of semantic features, and different types of semantic features have different probability of appearing in the local map. For example, in a local map of a parking lot, semantic features such as lane lines, parking space lines, storage locations, zebra crossings, and lane arrows are generally included, while features similar to zebra crossings or lane arrows with lower probability of occurrence can be set Are specific features used for map matching. Using the specific features in the local map to match between the local map and the global map can increase the probability of successful matching.

The identification sub-unit 5022 is used to identify target features in the pre-built global map that match the specific features identified by the detection sub-unit 5021.

The determining subunit 5023 is used for identifying the subunit 5022 to determine the position of the local map in the global map according to the position of the target feature in the global map.

It can be seen that the implementation of the on-board terminal shown in FIG. 6 can use only visual information to complete the positioning initialization of the vehicle, and can also use the four cameras installed around the vehicle to form a camera look-around solution, so that the target image obtained by a single acquisition is used to construct the The local map contains more features, which is beneficial to the matching between the local map and the global map. It can also reduce the impact on the local map construction and positioning of the vehicle terminal when some cameras fail. In addition, the in-vehicle terminal shown in FIG. 6 uses a specific feature with a low probability of occurrence in the local map to perform matching between the local map and the global map, which can increase the probability of successful matching.

Example 5

Please refer to FIG. 7, which is a schematic structural diagram of another vehicle-mounted terminal disclosed by an embodiment of the present invention. The vehicle-mounted terminal shown in FIG. 7 is obtained by optimizing the vehicle-mounted terminal shown in FIG. 6. As shown in FIG. 7, the vehicle-mounted terminal may further include:

A first determining unit 504 is configured to determine whether to record the positioning end position at the end of the last positioning calculation before constructing the surrounding environment of the vehicle using the target image captured by the camera to obtain a local map; One positioning calculation is the previous positioning calculation that occurred before the vehicle was started.

The second determining unit 505 is configured to determine the end position of the positioning as the initial position of the vehicle in the global map when the first determining unit 504 determines that the positioning end position is recorded.

Correspondingly, the above-mentioned construction unit 501 is specifically configured to use the target image captured by the camera to construct the surrounding environment of the vehicle when the first determination unit 504 determines that the positioning end position is not recorded, to obtain a local map.

In a possible application scenario, from the time the vehicle is turned off (that is, the end of the positioning calculation) to the next startup, there is likely to be no change in the position of the vehicle. Therefore, the positioning end position can be directly used as the initial position of the vehicle in the global map. Position, which can shorten the time required for positioning initialization and improve the user experience.

Optionally, the vehicle-mounted terminal shown in FIG. 7 may also include:

A second determining unit 506 is configured to determine whether a satellite positioning signal is received before the constructing unit 501 constructs a surrounding environment of the vehicle using a target image captured by a camera to obtain a local map.

The third determining unit 507 is configured to determine an initial position of the vehicle in the global map based on the satellite positioning signal when the second determining unit 506 determines that the satellite positioning signal is received.

Correspondingly, the above-mentioned constructing unit 501 is specifically configured to use the target image captured by the camera to construct the surrounding environment of the vehicle when the second determining unit 506 determines that no satellite positioning signal is received, to obtain a local map.

In a possible application scenario, when a vehicle drives from an above-ground road into an underground garage, it is necessary to complete the switching between the above-ground map and the underground map, and at the same time complete the positioning initialization based on the underground map. At the entrance position of the underground garage, the second determination unit 506 may determine that a satellite positioning signal is received. Therefore, the vehicle-mounted terminal may directly trigger the third determination unit 507 to determine that the vehicle-mounted terminal is in the global map (ie, the underground map) using the satellite positioning signal. Initial position, which shortens the time required for positioning initialization and improves the user experience.

It should be noted that, in the embodiment of the present invention, if the in-vehicle terminal set includes a first determination unit 504 and a second determination unit 506, and when the first determination unit 504 determines that it is recorded to the positioning end position, the second The judging unit 505 also judges that the satellite positioning signal is received. According to the propagation characteristics of the satellite positioning signal, there may be errors in the satellite positioning results. Therefore, the second determining unit 505 is triggered to execute the positioning end position as the initial position of the vehicle in the global map Position operation determines the positioning end position at the end of the last positioning calculation as the vehicle's positioning initialization result.

It can be seen that when the vehicle-mounted terminal shown in FIG. 7 is implemented, the positioning of the vehicle can be completed using only the visual information. At the same time, when the positioning end position is recorded, the positioning end position can be directly used as a result of positioning initialization. When a satellite positioning signal is received, the result of positioning initialization can be determined based on the satellite positioning signal, thereby reducing the time required for positioning initialization. Time to improve user experience. In addition, the three positioning initialization methods mentioned above are complementary to each other, which can improve the stability of positioning initialization.

Example Six

Please refer to FIG. 8, which is a schematic structural diagram of another vehicle-mounted terminal disclosed in an embodiment of the present invention. As shown in FIG. 8, the vehicle-mounted terminal may include:

At least one processor 801, such as a CPU, at least one network interface 804, user interface 803, memory 805, at least one communication bus 802, and a display screen 806. The communication bus 802 is used to implement connection and communication between these components. The user interface 803 may include a display screen, and the optional user interface 803 may further include a standard wired interface and a wireless interface. The network interface 804 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 805 may be a high-speed RAM memory, or may be a non-volatile memory (non-volatile memory), for example, at least one magnetic disk memory. The memory 805 may optionally be at least one storage device located far from the foregoing processor 801. As shown in FIG. 8, a memory 805 as a computer storage medium stores executable program code, which may include at least an operating system, a network communication module, a user interface module, and a positioning initialization module.

In the vehicle-mounted terminal shown in FIG. 8, the network interface 804 is mainly used to connect to the server and perform data communication with the server (such as downloading a global map); and the processor 801 may be coupled to the memory 805 and used to call the memory stored in the memory 805. The executable program code corresponding to the positioning initialization module executes any positioning initialization method applied to vehicle positioning shown in FIG. 1 or FIG. 2.

It should be noted that the vehicle-mounted terminal shown in FIG. 8 may further include components not shown, such as a power source, input buttons, speakers, and a Bluetooth module, which are not described in this embodiment.

An embodiment of the present invention discloses a computer-readable storage medium that stores a computer program, wherein the computer program causes a computer to execute any one of the positioning initialization methods shown in FIG. 1 or FIG. 2 that is applied to vehicle positioning.

An embodiment of the present invention discloses a computer program product. The computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute any one of FIG. 1 or FIG. 2. Positioning initialization method applied to vehicle positioning

It should be understood that "an embodiment" or "an embodiment" mentioned throughout the specification means that a particular feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of "in one embodiment" or "in an embodiment" appearing throughout the specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. Those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present invention.

In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above processes does not mean the necessary sequence of execution. The execution order of each process should be determined by its function and internal logic, and should not be implemented in the present invention. The implementation process of the example constitutes any limitation.

The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objective of the solution of this embodiment.

In addition, the functional units in the embodiments of the present invention may be integrated into one processing unit, or each of the units may exist separately physically, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional unit.

When the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention essentially or part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory , Including a number of requests to cause a computer device (which may be a personal computer, a server, or a network device, specifically a processor in a computer device) to perform some or all of the steps of the foregoing methods of various embodiments of the present invention.

A person of ordinary skill in the art may understand that all or part of the steps in the various methods of the foregoing embodiments may be implemented by a program instructing related hardware. The program may be stored in a computer-readable storage medium, and the storage medium includes a read-only Read-Only Memory (ROM), Random Access Memory (RAM), Programmable Read-only Memory (PROM), Erasable Programmable Read Only Memory, EPROM), One-time Programmable Read-Only Memory (OTPROM), Electronically-Erasable Programmable Read-Only Memory (EEPROM), Compact Disc (Compact Disc) Read-Only Memory (CD-ROM) or other optical disk storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

The positioning initialization method and vehicle terminal applied to vehicle positioning disclosed in the embodiments of the present invention have been described in detail above. Specific examples are used in this document to explain the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of To help understand the method of the present invention and its core ideas. At the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation and the scope of application. In summary, the content of this description should not be construed as a limitation on the present invention.

Claims

A positioning initialization method applied to vehicle positioning is characterized in that the method includes:

Use the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map;

Matching the local map with a pre-built global map to obtain the position of the local map in the global map; wherein, as the vehicle travels, the range of the local map gradually increases. When it is increased to be able to match the only area that is the same as the local map from the global map, the position of the local map in the global map is determined according to the location of the area in the global map;

Based on the position of the local map in the global map, map the position of the vehicle in the local map to the global map to obtain the initial position of the vehicle in the global map.
The positioning initialization method applied to vehicle positioning according to claim 1, wherein the use of a target image captured by an image acquisition device to construct a surrounding environment of the vehicle to obtain a local map includes:

Acquire multiple target images captured by multiple image acquisition devices at the same time; the multiple image acquisition devices include image acquisition devices installed in the front, rear, left, and right directions of the vehicle, each The framing range of the image acquisition device includes at least the ground below the image acquisition device;

Stitching a plurality of said target images to obtain a top-view mosaic image;

Identify image semantic features in the top-view mosaic image, and build a local map based on the image semantic features.
The positioning initialization method applied to vehicle positioning according to claim 2, wherein the local map is matched with a pre-built global map to obtain a position of the local map in the global map include:

Detecting a specific feature in the local map, where the specific feature is a semantic feature of the image whose appearance probability in the local map is lower than the non-specific feature;

Identifying target features in the pre-built global map that match the specific features;

Determining a position of the local map in the global map according to a position of the target feature in the global map.
The positioning initialization method applied to vehicle positioning according to any one of claims 1 to 3, wherein the surrounding environment of the vehicle is constructed in the target image captured by the image acquisition device to obtain a local Before the map, the method further includes:

Determining whether to record the end position of the positioning at the end of the last positioning calculation, where the last positioning calculation is a previous positioning calculation that occurred before the vehicle was started;

If the positioning end position is recorded, determining the positioning end position as an initial position of the vehicle in the global map;

If the positioning end position is not recorded, the target image captured by the image acquisition device is used to construct the surrounding environment of the vehicle to obtain a local map.
The positioning initialization method applied to vehicle positioning according to any one of claims 1 to 3, wherein the surrounding environment of the vehicle is constructed in the target image captured by the image acquisition device to obtain a local Before the map, the method further includes:

Determine whether a satellite positioning signal is received, and if the satellite positioning signal is received, determine an initial position of the vehicle in the global map based on the satellite positioning signal;

If the satellite positioning signal is not received, executing the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map.
A vehicle-mounted terminal, comprising:

A construction unit, configured to construct a surrounding environment of the vehicle by using a target image captured by an image acquisition device to obtain a local map;

A matching unit configured to match the local map with a pre-built global map to obtain a position of the local map in the global map; wherein, as the vehicle travels, the range of the local map gradually increases Large, when the range of the local map is increased to be able to match from the global map to the only area that is the same as the local map, determining the location of the local map based on the location of the area in the global map A position in the global map;

A first determining unit, configured to map a position of the vehicle in the local map to the global map based on a position of the local map in the global map to obtain the vehicle in the global map The initial position in the map.
The vehicle-mounted terminal according to claim 6, wherein the construction unit comprises:

An acquisition subunit, configured to acquire multiple target images captured by multiple image acquisition devices at the same moment; the multiple image acquisition devices include image acquisitions respectively installed in front, rear, left, and right directions of the vehicle Device, the framing range of each said image acquisition device includes at least the ground below the image acquisition device;

A stitching sub-unit for stitching a plurality of said target images to obtain a top-view stitching image;

A construction subunit is used for identifying the image semantic feature in the top-view mosaic image, and constructing a local map based on the image semantic feature.
The vehicle-mounted terminal according to claim 7, wherein the matching unit comprises:

A detection subunit, configured to detect a specific feature in the local map, where the specific feature is a semantic feature of the image whose appearance probability in the local map is lower than the non-specific feature;

A recognition subunit for identifying a target feature in a pre-built global map that matches the specific feature;

A determining subunit, configured to determine a position of the local map in the global map according to a position of the target feature in the global map.
The vehicle-mounted terminal according to any one of claims 6 to 8, wherein the vehicle-mounted terminal further comprises:

A first determining unit is configured to determine whether the positioning end position at the end of the last positioning calculation is recorded before the construction unit uses the target image captured by the image acquisition device to construct the surrounding environment of the vehicle to obtain a local map. The last positioning calculation is a previous positioning calculation that occurred before the vehicle was started;

A second determining unit, configured to determine the positioning end position as the initial position of the vehicle in the global map when the first determination unit determines that the positioning end position is recorded;

The construction unit is specifically configured to use the target image captured by the image acquisition device to construct the surrounding environment of the vehicle when the first determination unit determines that the positioning end position is not recorded, to obtain a local map .
The vehicle-mounted terminal according to claim 3 or 8, wherein the specific characteristics include a zebra crossing, a lane arrow, and a storage location.