CN114266876B

CN114266876B - Positioning method, visual map generation method and device

Info

Publication number: CN114266876B
Application number: CN202111450133.5A
Authority: CN
Inventors: 赵晨旭; 王志昊
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-11-30
Filing date: 2021-11-30
Publication date: 2023-03-28
Anticipated expiration: 2041-11-30
Also published as: JP2023021469A; KR20220155245A; CN116129087A; CN114266876A; US20230169680A1; JP7375149B2

Abstract

The disclosure provides a positioning method, a generation method of a visual map and a device thereof, and relates to the technical fields of computer vision, optical character recognition, intelligent transportation and augmented reality in the technical field of artificial intelligence. The specific implementation scheme is as follows: the method comprises the steps of obtaining a parking space number corresponding to a parking space image, obtaining a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, obtaining a first conversion matrix from a camera coordinate system to a parking space number coordinate system according to the parking space number, obtaining the parking space number coordinate system according to the parking space number, building the parking space number coordinate system according to the parking space number, and determining the three-dimensional coordinate and the three-dimensional pose of a camera in the world coordinate system according to the first conversion matrix, the three-dimensional coordinate and the three-dimensional pose of the parking space number. The method has the advantages that the positioning is realized without depending on visual features, the method is not influenced by factors such as high environment and texture repeatability, the positioning effect is enhanced, the method is easy to deploy and land quickly, and the later maintenance cost is low.

Description

Positioning method, visual map generation method and device

Technical Field

The present disclosure relates to the technical fields of computer vision, optical character recognition, intelligent transportation and augmented reality in the technical field of artificial intelligence, and in particular, to a positioning method, a method for generating a visual map, and an apparatus thereof.

Background

At present, positioning plays an increasingly important role in daily life of people, and functions such as driving navigation, shop searching and the like are realized by means of positioning technology. Because GPS signals, bluetooth signals or WIFI signals and the like are easily influenced by the environment, stable positioning is difficult to realize under the condition of weak signals, visual positioning is increasingly favored by people as a new positioning scheme and is widely applied to the fields of scientific research, industry and commerce and the like, for example, sweeping robots capable of realizing efficient panoramic navigation, VR house-keeping systems and the like, and the visual positioning technology can be divided into positioning based on a visual map and positioning without a prior map. However, in the related art, when a scene with high texture repeatability is positioned, the positioning effect is poor, the construction of the visual map is easily affected by various environmental factors, the positioning operation cost is high, and the positioning is not easy to deploy and land.

Disclosure of Invention

A positioning method, a method for generating a visual map and an apparatus thereof are provided.

According to a first aspect, there is provided a positioning method comprising: acquiring a parking space number corresponding to the parking space image; acquiring a three-dimensional coordinate and a three-dimensional pose of the parking space number under a world coordinate system according to the parking space number and a visual map, wherein the visual map is generated according to a parking space number plane graph; acquiring a first conversion matrix from a camera coordinate system to a parking space number coordinate system, wherein the parking space number coordinate system is constructed according to the parking space number plane graph; and determining the shooting position and the shooting pose of the parking space image of the camera under the world coordinate system according to the first conversion matrix, the three-dimensional coordinate of the parking space number and the three-dimensional pose.

According to a second aspect, there is provided a method of generating a visual map, comprising: determining three-dimensional coordinates of a plurality of parking space numbers under a world coordinate system according to the parking space number plan; determining the three-dimensional poses of the parking space numbers under the world coordinate system according to the parking space number plan; and generating the visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

According to a third aspect, there is provided a positioning device comprising: the first acquisition module is used for acquiring a parking space number corresponding to the parking space image; the second acquisition module is used for acquiring a three-dimensional coordinate and a three-dimensional pose of the parking space number under a world coordinate system according to the parking space number and a visual map, and the visual map is generated according to the parking space number; the third acquisition module is used for acquiring a first conversion matrix from a camera coordinate system to a parking space number coordinate system, wherein the parking space number coordinate system is constructed according to the parking space number; and the first determining module is used for determining the three-dimensional coordinate and the three-dimensional pose of the camera under the world coordinate system according to the first conversion matrix, the three-dimensional coordinate of the parking space number and the three-dimensional pose.

According to a fourth aspect, there is provided an apparatus for generating a visual map, comprising: the second determining module is used for determining three-dimensional coordinates of a plurality of parking space numbers under a world coordinate system according to the parking space number plan; the third determining module is used for determining the three-dimensional poses of the parking spaces under the world coordinate system according to the parking space number plane graph; and the first generation module is used for generating the visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

According to a fifth aspect, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the positioning method of the first aspect of the disclosure or the generation method of the visual map of the second aspect of the disclosure.

According to a sixth aspect, there is provided a non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the positioning method according to the first aspect of the present disclosure or the generation method of the visual map according to the second aspect of the present disclosure.

According to a seventh aspect, a computer program product is provided, comprising a computer program which, when being executed by a processor, carries out the steps of the positioning method according to the first aspect of the disclosure or the steps of the method for generating a visual map according to the second aspect of the disclosure.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic flow chart diagram of a positioning method according to a first embodiment of the present disclosure;

FIG. 2 is a schematic illustration of a parking space image in an embodiment of the present disclosure;

fig. 3 is a schematic flow chart of a positioning method according to a second embodiment of the present disclosure;

fig. 4 is a scene schematic diagram of setting a camera to a preset position of a parking space number in an embodiment of the present disclosure;

fig. 5 is a schematic flow chart of a positioning method according to a third embodiment of the present disclosure;

fig. 6 is a schematic flow chart of a positioning method according to a fourth embodiment of the disclosure

FIG. 7 is a schematic illustration of a parking space numbering plan in an embodiment of the present disclosure;

FIG. 8 is a schematic diagram of a parking space number coordinate system in an embodiment of the present disclosure;

fig. 9 is a flowchart schematic diagram of a method of generating a visual map according to a first embodiment of the present disclosure;

fig. 10 is a flow chart schematic diagram of a method of generating a visual map according to a second embodiment of the present disclosure;

fig. 11 is a flowchart schematic diagram of a method of generating a visual map according to a third embodiment of the present disclosure;

fig. 12 is a flow chart schematic diagram of a method of generating a visual map according to a fourth embodiment of the present disclosure;

FIG. 13 is a block diagram of a positioning device according to a first embodiment of the present disclosure;

FIG. 14 is a block diagram of a positioning device according to a second embodiment of the present disclosure;

fig. 15 is a block diagram of a generation apparatus of a visual map according to a first embodiment of the present disclosure;

fig. 16 is a block diagram of a generation apparatus of a visual map according to a second embodiment of the present disclosure;

FIG. 17 is a block diagram of an electronic device used to implement methods of embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Artificial Intelligence (AI) is a technical science that studies and develops theories, methods, techniques and application systems for simulating, extending and expanding human intelligence. At present, the AI technology has the advantages of high automation degree, high accuracy and low cost, and is widely applied.

Computer Vision (also known as Machine Vision) is a simulation of biological Vision using a Computer and related equipment, and further refers to a method of using a camera and a Computer to replace human eyes to perform Machine Vision such as identification, tracking and measurement on a target, and further performing image processing, so that the Computer processing becomes an image more suitable for human eyes to observe or transmit to an instrument to detect.

Optical Character Recognition (OCR) refers to a process of scanning text data, analyzing an image file, and acquiring text and layout information. The main indicators for measuring the performance of the OCR system are: rejection rate, false recognition rate, recognition speed, user interface friendliness, product stability, usability, feasibility and the like.

An Intelligent Transportation System (ITS) is a comprehensive Transportation System which effectively and comprehensively applies advanced scientific technologies (information technology, computer technology, data communication technology, sensor technology, electronic control technology, automatic control theory, operational research, artificial intelligence and the like) to Transportation, service control and vehicle manufacturing, and strengthens the relation among vehicles, roads and users, thereby forming a comprehensive Transportation System which ensures safety, improves efficiency, improves environment and saves energy.

Augmented Reality (AR) is a technology for calculating the position and angle of a camera image in real time and adding a corresponding image, and is implemented by superimposing a virtual object generated by a computer or non-geometric information about the real object onto a scene of the real world by using various technical means, thereby enhancing the real world.

The following describes a positioning method, a generation method of a visual map, and apparatuses thereof according to an embodiment of the present disclosure with reference to the drawings.

Fig. 1 is a schematic flow chart of a positioning method according to a first embodiment of the present disclosure.

As shown in fig. 1, the positioning method according to the embodiment of the present disclosure may specifically include the following steps:

and S101, acquiring a parking space number corresponding to the parking space image.

Specifically, the main body of the positioning method according to the embodiment of the present disclosure may be the positioning apparatus provided in the embodiment of the present disclosure, and the positioning apparatus may be a hardware device having a data information processing capability and/or necessary software for driving the hardware device to operate. Alternatively, the execution body may include a workstation, a server, a computer, a user terminal, and other devices. The user terminal includes, but is not limited to, a mobile phone, a computer, an intelligent voice interaction device, an intelligent household appliance, a vehicle-mounted terminal, and the like. The embodiment of the disclosure takes a parking lot as an example, and explains an implementation manner of the positioning method of the embodiment of the disclosure in the scene.

In the embodiment of the present disclosure, a parking space number corresponding to a parking space in a parking space image captured by a camera device such as a camera of a mobile terminal by a user is obtained, as shown in fig. 2, a vehicle is parked in a certain parking space of the parking lot, a parking space number is marked at one end of the parking space close to a road, and the parking space number corresponding to the parking space can be obtained from the parking space image with the parking space number by an optical character recognition technology.

And S102, acquiring a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, wherein the visual map is generated according to the parking space number.

For a parking lot including a plurality of parking spaces, the position and other related information of the parking spaces can be displayed through a plane graph (such as a CAD graph) of the parking lot.

In some implementations, the three-dimensional coordinate and the three-dimensional pose of the parking space number in the world coordinate system are obtained according to the parking space number and the visual map obtained from the parking space image. The world coordinate system can be understood as a coordinate system in which a Z axis is added to a plane diagram origin coordinate system.

S103, a first conversion matrix from the camera coordinate system to the parking space number coordinate system is obtained, and the parking space number coordinate system is constructed according to the parking space number.

In some embodiments, a parking space number coordinate system may be constructed with coordinates of a parking space number as an origin of the coordinate system, a camera coordinate system may be established with a light spot of a camera, which is used by a user to shoot an image of the parking space, as the origin of the coordinate system, and a first transformation matrix from the camera coordinate system to the parking space coordinate system may be obtained based on the established camera coordinate system and the parking space number coordinate system, so that coordinates of a position point in the camera coordinate system in the parking space number coordinate system may be determined according to the first transformation matrix. For example, the coordinate of a position point P in the camera coordinate system is (X) _C ，Y _C ，Z _C ) Can be according to (X) _C ，Y _C ，Z _C ) And a first conversion matrix for calculating the coordinate (X) of the space point P in the parking space number coordinate system _O ，Y _O ，Z _O )。

And S104, determining the three-dimensional coordinate and the three-dimensional pose of the camera under a world coordinate system according to the first conversion matrix and the three-dimensional coordinate and the three-dimensional pose of the parking space number.

In the embodiment of the disclosure, the three-dimensional coordinate and the three-dimensional pose of the camera in the world coordinate system can be calculated according to the first conversion matrix from the camera coordinate system to the parking space number coordinate system, and the three-dimensional coordinate and the three-dimensional pose of the parking space number in the world coordinate system.

For example, the following steps are carried out: according To the parking space number and the visual map obtained from the parking space image, determining the three-dimensional coordinate and the three-dimensional pose of the parking space number in the world coordinate system (namely the position and the orientation of the parking space number in the world coordinate system, which are represented by a matrix To2 w) according To the To2w and the first conversion matrix Tc2o, obtaining the three-dimensional coordinate and the three-dimensional pose of the camera in the world coordinate system, namely realizing the positioning of six degrees of freedom: tc2w = To2w Tc2o.

In summary, according to the positioning method disclosed by the embodiment of the disclosure, the parking space number corresponding to the parking space image is obtained, the three-dimensional coordinate and the three-dimensional pose of the parking space number in the world coordinate system are obtained according to the parking space number and the visual map, the visual map is generated according to the parking space number, the first conversion matrix from the camera coordinate system to the parking space number coordinate system is obtained, the parking space number coordinate system is constructed according to the parking space number, and the three-dimensional coordinate and the three-dimensional pose of the camera in the world coordinate system are determined according to the first conversion matrix, the three-dimensional coordinate and the three-dimensional pose of the parking space number. The method and the device have the advantages that the positioning is realized without depending on visual features, the influence of factors such as high environment and texture repeatability is avoided, the positioning effect is enhanced, the camera (namely the position of a user) can be positioned according to the first conversion matrix from the camera coordinate system to the parking space number coordinate system, the visual map of the parking scene and the parking space number obtained from the parking space image, the rapid deployment and the landing are easy, the later maintenance cost is low, and the method and the device are favorable for the commercial batch landing.

Fig. 3 is a schematic flow chart diagram of a positioning method according to a second embodiment of the present disclosure.

As shown in fig. 3, on the basis of the embodiment shown in fig. 1, the positioning method of the embodiment of the present disclosure may specifically include the following steps:

s301, acquiring a parking space number corresponding to the parking space image.

And S302, acquiring a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, wherein the visual map is generated according to the parking space number.

Specifically, step S103 in the above embodiment may specifically include the following steps S303 to S304:

and S303, acquiring a rotation matrix from the camera coordinate system to the parking space number coordinate system.

In the embodiment of the present disclosure, according to the rotation matrix from the camera coordinate system to the parking space number coordinate system, the main axis of the camera coordinate system may be rotated to the direction of the main axis of the corresponding parking space number coordinate system, so that the rotation matrix may be constructed based on a plurality of angles of rotation by recording the angles at which the main axes of the camera coordinate system need to be rotated when the camera coordinate system is converted to the parking space number coordinate system. Wherein the rotation matrix may be used as a rotation component from the camera coordinate system to the parking space number coordinate system.

In some embodiments, for a vector in the camera coordinate system, the direction of the vector when represented in the parking space number coordinate system can be obtained according to the rotation matrix (represented by R).

S304, determining a first conversion matrix according to a position vector and a rotation matrix of a preset camera to parking space number.

In this disclosure, the position vector from the camera to the parking space number may be preset as needed, and the position vector may be used as the translation component from the camera coordinate system to the parking space number coordinate system, for example, the origin of the two coordinate systems may coincide with each other by moving the camera coordinate system and the parking space number coordinate system according to the position vector.

For example, as shown in fig. 4, the camera is set to the preset position of the parking space number: the height of the camera (point A) from the parking space number (point B) is 1.5m, the yaw angle relative to the parking space number is 0, and the projection of the camera on the ground is positioned at a position of-3 m along the y-axis direction of the parking space number coordinate system. ) According to

A position vector T from the camera to the parking space number is obtained, which can also be understood as an offset vector.

In some embodiments, a first conversion matrix of the camera coordinate system to the parking space number coordinate system is determined from the position vector T and the rotation matrix R

S305, determining the three-dimensional coordinate and the three-dimensional pose of the camera in the world coordinate system according to the first conversion matrix and the three-dimensional coordinate and the three-dimensional pose of the parking space number.

Specifically, steps S301 to S302 are the same as steps S101 to S102 in the above embodiment, and step S305 is the same as step S104 in the above embodiment, which is not described again here.

Further, on the basis of any one of the above embodiments, as shown in fig. 5, the step S303 of "acquiring a rotation matrix from the camera coordinate system to the parking space number coordinate system" may specifically include the following steps:

s501, acquiring a first direction of gravity in a camera coordinate system.

In the disclosed embodiment, a direction of gravity in a camera coordinate system, i.e. a first direction, may be acquired by an accelerometer in the mobile device, where the first direction is represented by a vector V1.

And S502, acquiring a second direction of the gravity under the parking space number coordinate system.

In the embodiment of the present disclosure, based on the parking space number coordinate system, a direction of gravity in the coordinate system, that is, a second direction is calculated, and the second direction is represented by a vector V2 in the parking space number coordinate system.

S503, determining a rotation matrix according to the first direction and the second direction.

In the embodiment of the present disclosure, a rotation matrix from the camera coordinate system to the parking space number coordinate system, that is, a rotation matrix from the vector V1 in the camera coordinate system to the vector V2 in the parking space number coordinate system, is calculated according to the vector V1 corresponding to the first direction and the vector V2 corresponding to the second direction.

Therefore, a first conversion matrix between the two coordinate systems can be obtained based on the rotation matrix from the camera coordinate system to the parking space number coordinate system and the preset position vector, and the coordinates of the camera in the parking space coordinate system can be determined based on the first conversion matrix.

As a possible implementation manner, the visual map in the above embodiment may be generated according to the plan view information of the parking space number corresponding to the parking space number in the parking space number plan view.

On the basis of the foregoing embodiment, as shown in fig. 6, the positioning method in the embodiment of the present disclosure may further include a process of generating a visual map, which specifically includes the following steps:

s601, determining three-dimensional coordinates of a plurality of parking space numbers in a world coordinate system according to the parking space number plane graph.

In the embodiment of the present disclosure, a world coordinate system is established with the origin of the plane diagram as the origin of the world coordinate system, and three-dimensional coordinates of a plurality of parking space numbers in the parking space number plane diagram are determined based on the world coordinate system.

As a possible implementation manner, when constructing a visual map of a parking lot on the ground, only two-dimensional coordinates of a parking space number in a plane graph origin coordinate system, that is, values of (x, y) need to be determined, and on the basis, a three-dimensional coordinate of the parking space number in a world coordinate system can be obtained by adding 0 to the Z axis. If the coordinate value of the z axis is the underground parking lot, the coordinate value can be determined according to the actual numerical value of the parking lot from the ground.

In some embodiments, the two-dimensional coordinates of the parking space number in the plane view origin coordinate system may be determined by: as shown in fig. 7, each parking space number corresponds to a parking space frame in the parking space number plane diagram, for each parking space number, two-dimensional coordinates of two corner points of the corresponding parking space frame (e.g., two corner points: corner point C and corner point D of the road near the parking space frame corresponding to the parking space 037 in fig. 7) are obtained, two-dimensional coordinates of a midpoint of a line segment between the two corner points are obtained according to the two-dimensional coordinates of the two corner points, and the two-dimensional coordinates are determined as the two-dimensional coordinates of the parking space number corresponding to the parking space frame.

And S602, determining the three-dimensional poses of the parking spaces under the world coordinate system according to the parking space number plane graph.

In some embodiments, a parking space number coordinate system corresponding to the parking space number is constructed according to the parking space number plane graph, and the three-dimensional pose of the parking space number in the world coordinate system is determined according to the parking space number coordinate system and the world coordinate system.

For example, as shown in fig. 8, a parking space number coordinate system is established, with a coordinate point of a parking space number as an origin, a direction parallel to a road, a direction from left to right facing the parking space number as an x-axis, a direction from a side of the parking space close to the road to a side far from the road as a y-axis, and a direction from the ground to the sky as a z-axis, such as a parking space number coordinate system corresponding to the parking space No. 007 and a parking space number coordinate system corresponding to the parking space No. 037 in fig. 8. And determining a translation component and a rotation component from a corresponding parking space number coordinate system to a world coordinate system according to the plurality of parking space numbers, and determining the three-dimensional poses of the plurality of parking space numbers in the world coordinate system according to the translation component and the rotation component, namely the orientations of the parking space numbers in the world coordinate system.

And S603, generating a visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

In the embodiment of the disclosure, according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers, a mapping relation between the parking space numbers and the corresponding three-dimensional coordinates and three-dimensional poses is formed, and the position and orientation information corresponding to the parking space numbers is summarized to form the visual map.

Therefore, according to the parking space information corresponding to each parking space number in the parking space number plane graph, such as the positions of two corner points of a parking space frame and the arrangement distribution information of each parking space, a parking space number coordinate system is constructed, and the parking space number coordinate system and the world coordinate system are determined to determine the three-dimensional position and pose of the parking space number under the world coordinate system, so that the degree of freedom of positioning is enhanced, and more accurate positioning information is provided.

In summary, according to the positioning method disclosed by the embodiment of the disclosure, the parking space number corresponding to the parking space image is obtained, the three-dimensional coordinate and the three-dimensional pose of the parking space number in the world coordinate system are obtained according to the parking space number and the visual map, the visual map is generated according to the parking space number, the first conversion matrix from the camera coordinate system to the parking space number coordinate system is obtained, the parking space number coordinate system is constructed according to the parking space number, and the three-dimensional coordinate and the three-dimensional pose of the camera in the world coordinate system are determined according to the first conversion matrix, the three-dimensional coordinate and the three-dimensional pose of the parking space number. The method can realize positioning without depending on visual features, is not influenced by factors such as high environment and texture repeatability, enhances the positioning effect, determines a first conversion matrix from a camera coordinate system to a parking space number coordinate system according to a preset position vector from a camera to a parking space number and a rotation matrix from the camera coordinate system to the parking space number coordinate system, and can position the camera (namely the position of a user) based on the first conversion matrix, a visual map of a parking lot and the parking space number obtained from a parking space image.

To implement the foregoing embodiments, the present disclosure further provides a method for generating a visual map, and fig. 9 is a flowchart illustrating the method for generating a visual map according to the first embodiment of the present disclosure.

As shown in fig. 9, the method for generating a visual map according to an embodiment of the present disclosure may specifically include the following steps:

and S901, determining three-dimensional coordinates of a plurality of parking space numbers in a world coordinate system according to the parking space number plane graph.

Specifically, the executing body of the method for generating a visual map according to the embodiment of the present disclosure may be a device for generating a visual map according to the embodiment of the present disclosure, and the generating device may be a hardware device having a data information processing capability and/or software necessary for driving the hardware device to operate. Alternatively, the execution body may include a workstation, a server, a computer, a user terminal, and other devices. The user terminal includes, but is not limited to, a mobile phone, a computer, an intelligent voice interaction device, an intelligent household appliance, a vehicle-mounted terminal, and the like.

And S902, determining the three-dimensional poses of the parking spaces under the world coordinate system according to the parking space number plane graph.

And S903, generating a visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

It should be noted that, the method for generating the visual map according to the embodiment of the present disclosure is the same as that of the above embodiment, and is not described herein again.

In summary, according to the method for generating the visual map in the embodiment of the present disclosure, the three-dimensional coordinates of the plurality of parking space numbers in the world coordinate system are determined according to the parking space number plan, the three-dimensional positions of the plurality of parking space numbers in the world coordinate system are determined according to the parking space number plan, and the visual map is generated according to the three-dimensional coordinates and the three-dimensional positions of the plurality of parking space numbers. According to the method and the device for generating the visual map based on the parking space number plan, the visual map is generated based on the parking space number plan, a large amount of data and images of a parking scene do not need to be collected, the cost is saved, and meanwhile the influence of environmental factors such as illumination on the construction of the visual map is avoided.

Fig. 10 is a flowchart illustrating a method of generating a visual map according to a second embodiment of the present disclosure.

As shown in fig. 10, on the basis of the above embodiments, the method for generating a visual map according to an embodiment of the present disclosure may specifically include the following steps:

the step S901 may specifically include steps S1001 to S1002:

s1001, acquiring two-dimensional coordinates of the parking space number in a plane diagram origin coordinate system according to the parking space number plane diagram.

And S1002, determining the three-dimensional coordinate of the parking space number in the world coordinate system according to the two-dimensional coordinate of the parking space number.

And S1003, determining the three-dimensional poses of the parking space numbers in the world coordinate system according to the parking space number plane graph.

And S1004, generating a visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

Specifically, steps S1003 to S1004 are the same as steps S902 to S903 in the above embodiment, and the method for generating the visual map in the embodiment of the present disclosure is the same as that in the above embodiment, and will not be described again here.

Further, on the basis of the above embodiment, as shown in fig. 11, the step S1001 of "obtaining the two-dimensional coordinates of the parking space number under the original coordinate system of the plane view according to the parking space number plane view" may specifically include the following steps:

and S1101, acquiring two-dimensional coordinates of two corner points of a parking space frame according to the parking space number plan.

And S1102, determining the two-dimensional coordinates of the parking space number corresponding to the parking space frame according to the two-dimensional coordinates of the two corner points.

Further, on the basis of the above embodiment, as shown in fig. 12, the step S1003 of "determining three-dimensional poses of a plurality of parking space numbers in the world coordinate system according to the parking space number plan" may specifically include the following steps:

and S1201, constructing a parking space number coordinate system corresponding to the parking space number according to the parking space number plane graph.

And S1202, determining the three-dimensional pose of the parking space number in the world coordinate system according to the parking space number coordinate system and the world coordinate system.

It should be noted that, the method for generating the visual map of the embodiment of the present disclosure is the same as that of the above embodiment, and is not described herein again.

Fig. 13 is a block diagram of a positioning device according to a first embodiment of the present disclosure.

As shown in fig. 13, a positioning apparatus 1300 according to an embodiment of the present disclosure includes: a first obtaining module 1301, a second obtaining module 1302, a third obtaining module 1303, and a first determining module 1304.

The first obtaining module 1301 is configured to obtain a parking space number corresponding to the parking space image.

A second obtaining module 1302, configured to obtain a three-dimensional coordinate and a three-dimensional pose of the parking space number in the world coordinate system according to the parking space number and a visual map, where the visual map is generated according to the parking space number.

And a third obtaining module 1303, configured to obtain a first conversion matrix from the camera coordinate system to a parking space number coordinate system, where the parking space number coordinate system is constructed according to the parking space number.

And a first determining module 1304, configured to determine a three-dimensional coordinate and a three-dimensional pose of the camera in the world coordinate system according to the first transformation matrix, the three-dimensional coordinate and the three-dimensional pose of the parking space number.

It should be noted that the above explanation of the embodiment of the positioning method is also applicable to the positioning apparatus in the embodiment of the present disclosure, and the specific process is not described herein again.

In summary, the positioning device according to the embodiment of the present disclosure obtains a parking space number corresponding to a parking space image, and obtains a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, where the visual map is generated according to the parking space number and obtains a first conversion matrix from a camera coordinate system to a parking space number coordinate system, the parking space number coordinate system is constructed according to the parking space number, and determines a three-dimensional coordinate and a three-dimensional pose of the camera in the world coordinate system according to the first conversion matrix, the three-dimensional coordinate of the parking space number, and the three-dimensional pose. The method and the device have the advantages that the positioning is realized without depending on visual features, the influence of factors such as high environment and texture repeatability is avoided, the positioning effect is enhanced, the camera (namely the position of a user) can be positioned according to the first conversion matrix from the camera coordinate system to the parking space number coordinate system, the visual map of the parking scene and the parking space number obtained from the parking space image, the method and the device are easy to deploy and land, the later maintenance cost is low, and the method and the device are beneficial to commercial mass land landing.

Fig. 14 is a block diagram of a positioning device according to a second embodiment of the present disclosure.

As shown in fig. 14, a positioning apparatus 1400 of the present disclosure includes: a first obtaining module 1401, a second obtaining module 1402, a third obtaining module 1403, and a first determining module 1404.

The first obtaining module 1401 has the same structure and function as the first obtaining module 1301 in the previous embodiment, the second obtaining module 1402 has the same structure and function as the second obtaining module 1302 in the previous embodiment, the third obtaining module 1403 has the same structure and function as the third obtaining module 1303 in the previous embodiment, and the first determining module 1404 has the same structure and function as the first determining module 1304 in the previous embodiment.

Further, the third obtaining module 1403 may include: a first obtaining unit 14031, configured to obtain a rotation matrix from a camera coordinate system to a parking space number coordinate system; and a first determining unit 14032 configured to determine a first transformation matrix according to a preset position vector and a rotation matrix of a camera to parking space number.

Further, the first acquisition unit includes: the first acquiring subunit is used for acquiring a first direction of gravity in a camera coordinate system; the second acquiring subunit is used for acquiring a second direction of the gravity under the parking space number coordinate system; and a first determining subunit for determining a rotation matrix according to the first direction and the second direction.

Further, the first obtaining module 1401 may include a detecting unit, configured to perform optical character recognition detection on the parking space image to obtain the parking space number.

Further, the visual map is generated according to the plan view information corresponding to the parking space number in the parking space number plan view.

Further, the positioning apparatus 1400 may further include: the fourth determining module is used for determining three-dimensional coordinates of the parking space numbers under the world coordinate system according to the parking space number plan; the fifth determining module is used for determining the three-dimensional poses of the parking space numbers under the world coordinate system according to the parking space number plan; and the second generation module is used for generating a visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

Further, the fourth determining module includes: the second acquisition unit is used for acquiring the two-dimensional coordinates of the parking space number in the original point coordinate system of the plane diagram according to the parking space number plane diagram; and the second determining unit is used for determining the three-dimensional coordinate of the parking space number under the world coordinate system according to the two-dimensional coordinate of the parking space number.

Further, the second acquisition unit includes: the third acquisition subunit is used for acquiring two-dimensional coordinates of two angular points of the parking space frame according to the parking space number plan; and the second determining subunit is used for determining the two-dimensional coordinates of the parking space number corresponding to the parking space frame according to the two-dimensional coordinates of the two angular points.

Further, the fifth determining module includes: the first construction unit is used for constructing a parking space number coordinate system corresponding to the parking space number according to the parking space number plane graph; and the third determining unit is used for determining the three-dimensional pose of the parking space number in the world coordinate system according to the parking space number coordinate system and the world coordinate system.

In summary, the positioning device according to the embodiment of the present disclosure obtains a parking space number corresponding to a parking space image, and obtains a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, where the visual map is generated according to the parking space number and obtains a first conversion matrix from a camera coordinate system to a parking space number coordinate system, the parking space number coordinate system is constructed according to the parking space number, and determines a three-dimensional coordinate and a three-dimensional pose of the camera in the world coordinate system according to the first conversion matrix, the three-dimensional coordinate of the parking space number, and the three-dimensional pose. The method has the advantages that the positioning is realized without depending on visual features, the influence of factors such as high environment repeatability and texture repeatability is avoided, the positioning effect is enhanced, the first conversion matrix from the camera coordinate system to the parking space number coordinate system is determined according to the preset position vector from the camera to the parking space number and the rotation matrix from the camera coordinate system to the parking space number coordinate system, and the camera (namely the position of a user) can be positioned based on the first conversion matrix, the visual map of the parking lot and the parking space number obtained from the parking space image.

Fig. 15 is a block diagram of a generation apparatus of a visual map according to a first embodiment of the present disclosure.

As shown in fig. 15, the apparatus 1500 for generating a visual map according to an embodiment of the present disclosure includes: a second determination module 1501, a third determination module 1502, and a first generation module 1503.

And a second determining module 1501, configured to determine three-dimensional coordinates of the parking spaces in the world coordinate system according to the parking space number plan.

And a third determining module 1502, configured to determine three-dimensional poses of the parking spaces in the world coordinate system according to the parking space number plan.

The first generation module 1503 is configured to generate a visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

It should be noted that the explanation of the embodiment of the method for generating a visual map is also applicable to the apparatus for generating a visual map in the embodiment of the present disclosure, and the specific process is not described herein again.

In summary, the apparatus for generating a visual map according to the embodiments of the present disclosure determines, according to the parking space number plan, three-dimensional coordinates of a plurality of parking space numbers in the world coordinate system, determines, according to the parking space number plan, three-dimensional poses of the plurality of parking space numbers in the world coordinate system, and generates the visual map according to the three-dimensional coordinates and the three-dimensional poses of the plurality of parking space numbers. According to the method and the device for generating the visual map based on the parking space number plan, the visual map is generated based on the parking space number plan, a large amount of data and images of a parking scene do not need to be collected, the cost is saved, and meanwhile the influence of environmental factors such as illumination on the construction of the visual map is avoided.

Fig. 16 is a block diagram of a generation apparatus of a visual map according to a second embodiment of the present disclosure.

As shown in fig. 16, the generation apparatus 1600 of the visual map of the embodiment of the present disclosure includes: a second determining module 1601, a third determining module 1602, and a first generating module 1603.

The second determining module 1601 has the same structure and function as the second determining module 1501 in the previous embodiment, the third determining module 1602 has the same structure and function as the third determining module 1502 in the previous embodiment, and the first generating module 1603 has the same structure and function as the first generating module 1503 in the previous embodiment.

Further, the second determining module 1601 includes: a third obtaining unit 16011, configured to obtain a two-dimensional coordinate of the parking space number in the plane view origin coordinate system according to the parking space number plane view; and a fourth determining unit 16012, configured to determine a three-dimensional coordinate of the parking space number in the world coordinate system according to the two-dimensional coordinate of the parking space number.

Further, the third acquiring unit 16011 includes: the fourth acquiring subunit is used for acquiring two-dimensional coordinates of two angular points of the parking space frame according to the parking space number plan; and the third determining subunit is used for determining the two-dimensional coordinates of the parking space number corresponding to the parking space frame according to the two-dimensional coordinates of the two corner points.

Further, the third determining module 1602 includes: a second constructing unit 16021, configured to construct a parking space number coordinate system corresponding to the parking space number according to the parking space number plan; and a fifth determining unit 16022, configured to determine a three-dimensional pose of the parking space number in the world coordinate system according to the parking space number coordinate system and the world coordinate system.

In summary, the apparatus for generating a visual map according to the embodiments of the present disclosure determines, according to the parking space number plan, three-dimensional coordinates of a plurality of parking space numbers in the world coordinate system, determines, according to the parking space number plan, three-dimensional poses of the plurality of parking space numbers in the world coordinate system, and generates the visual map according to the three-dimensional coordinates and the three-dimensional poses of the plurality of parking space numbers. The visual map is generated based on the parking space number plan, a large amount of data and images of a parking scene do not need to be collected, cost is saved, and meanwhile, the influence of environmental factors such as illumination on the construction of the visual map is avoided.

In the technical scheme of the disclosure, the processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the common customs of public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

Fig. 17 illustrates a schematic block diagram of an example electronic device 1700 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 17, the electronic apparatus 1700 includes a computing unit 1701 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1702 or a computer program loaded from a storage unit 1708 into a Random Access Memory (RAM) 1703. In the RAM1703, various programs and data required for the operation of the electronic device 1700 can also be stored. The computing unit 1701, the ROM 1702, and the RAM1703 are connected to each other through a bus 1704. An input/output (I/O) interface 1705 is also connected to bus 1704.

Various components in the electronic device 1700 are connected to the I/O interface 1705, including: an input unit 1706 such as a keyboard, a mouse, or the like; an output unit 1707 such as various types of displays, speakers, and the like; a storage unit 1708 such as a magnetic disk, optical disk, or the like; and a communication unit 1709 such as a network card, modem, wireless communication transceiver, etc. The communication unit 1709 allows the electronic device 1700 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 1701 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 1701 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various dedicated Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 1701 executes various methods and processes described above, such as the positioning method shown in fig. 1 to 8 or the generation method of the visual map shown in fig. 9 to 12. For example, in some embodiments, the positioning method or the visual map generation method may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1708. In some embodiments, part or all of the computer program may be loaded and/or installed onto electronic device 1700 via ROM 1702 and/or communications unit 1709. When the computer program is loaded into the RAM1703 and executed by the computing unit 1701, one or more steps of the positioning method or the generation method of the visual map described above may be performed. Alternatively, in other embodiments, the computing unit 1701 may be configured in any other suitable manner (e.g., by means of firmware) to perform the positioning method or the visual map generation method.

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the Internet, and blockchain networks.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The Server can be a cloud Server, also called a cloud computing Server or a cloud host, and is a host product in a cloud computing service system, so as to solve the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS service ("Virtual Private Server", or simply "VPS"). The server may also be a server of a distributed system, or a server incorporating a blockchain.

According to an embodiment of the present disclosure, there is also provided a computer program product comprising a computer program, wherein the computer program, when being executed by a processor, realizes the steps of the positioning method or the visual map generation method shown in the above-mentioned embodiment of the present disclosure.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel or sequentially or in different orders, and are not limited herein as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made, depending on design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method of positioning, comprising:

acquiring a parking space number corresponding to the parking space image;

acquiring a three-dimensional coordinate and a three-dimensional pose of the parking space number in a world coordinate system according to the parking space number and a visual map, wherein the visual map is generated according to corresponding plan view information of the parking space number in a parking space number plan view;

acquiring a first conversion matrix from a camera coordinate system to a parking space number coordinate system, wherein the parking space number coordinate system is constructed according to the parking space number; and

and determining the three-dimensional coordinate and the three-dimensional pose of the camera under the world coordinate system according to the first conversion matrix, the three-dimensional coordinate of the parking space number and the three-dimensional pose.

2. The positioning method according to claim 1, wherein the obtaining a first conversion matrix from a camera coordinate system to a parking space number coordinate system comprises:

acquiring a rotation matrix from the camera coordinate system to the parking space number coordinate system; and

and determining the first conversion matrix according to a preset position vector from the camera to the parking space number and the rotation matrix.

3. The positioning method according to claim 2, wherein the acquiring of the rotation matrix of the camera coordinate system to the parking space number coordinate system includes:

acquiring a first direction of gravity under the camera coordinate system;

acquiring a second direction of the gravity under the parking space number coordinate system; and

determining the rotation matrix according to the first direction and the second direction.

4. The positioning method according to claim 1, wherein the acquiring of the parking space number corresponding to the parking space image comprises:

and carrying out optical character recognition detection on the parking space image to obtain the parking space number.

5. The positioning method according to claim 1, further comprising:

determining three-dimensional coordinates of a plurality of parking space numbers in the world coordinate system according to the parking space number plan;

determining the three-dimensional poses of the parking space numbers under the world coordinate system according to the parking space number plan; and

and generating the visual map according to the three-dimensional coordinates and the three-dimensional poses of the parking space numbers.

6. The positioning method according to claim 5, wherein the determining three-dimensional coordinates of a plurality of parking spaces in the world coordinate system according to the parking space number plan comprises:

acquiring a two-dimensional coordinate of the parking space number under a plane graph origin coordinate system according to the parking space number plane graph; and

and determining the three-dimensional coordinate of the parking space number under the world coordinate system according to the two-dimensional coordinate of the parking space number.

7. The positioning method according to claim 6, wherein the obtaining of the two-dimensional coordinates of the parking space number in a plane view origin coordinate system according to the parking space number plane view comprises:

acquiring two-dimensional coordinates of two angular points of a parking space frame according to the parking space serial number plane graph; and

and determining the two-dimensional coordinates of the parking space number corresponding to the parking space frame according to the two-dimensional coordinates of the two corner points.

8. The positioning method according to claim 5, wherein the determining the three-dimensional poses of the plurality of parking space numbers in the world coordinate system according to the parking space number plan comprises:

constructing the parking space number coordinate system corresponding to the parking space number according to the parking space number plane graph; and

and determining the three-dimensional pose of the parking space number under the world coordinate system according to the parking space number coordinate system and the world coordinate system.

9. A positioning device, comprising:

the first acquisition module is used for acquiring a parking space number corresponding to the parking space image;

the second acquisition module is used for acquiring a three-dimensional coordinate and a three-dimensional pose of the parking space number under a world coordinate system according to the parking space number and a visual map, and the visual map is generated according to plane map information corresponding to the parking space number in a parking space number plane map;

the third acquisition module is used for acquiring a first conversion matrix from a camera coordinate system to a parking space number coordinate system, wherein the parking space number coordinate system is constructed according to the parking space number; and

and the first determining module is used for determining the three-dimensional coordinate and the three-dimensional pose of the camera under the world coordinate system according to the first conversion matrix, the three-dimensional coordinate of the parking space number and the three-dimensional pose.

10. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein, the first and the second end of the pipe are connected with each other,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-8.