CN113592980B

CN113592980B - Signboard topological relation construction method and device, electronic equipment and storage medium

Info

Publication number: CN113592980B
Application number: CN202110729812.XA
Authority: CN
Inventors: 甘露; 吴云鹏; 王洪志
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2023-07-25
Anticipated expiration: 2041-06-29
Also published as: CN113592980A

Abstract

The disclosure provides a method, a device, electronic equipment and a storage medium for constructing a signboard topological relation, relates to the technical field of artificial intelligence, and particularly relates to the field of image processing and deep learning. The specific implementation scheme is as follows: acquiring a plurality of street view images; generating a corresponding first signboard topological relation according to the signboard information in the street view image; and generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images. According to the construction method of the signboard topological relation, the linear signboard topological relation is constructed by acquiring the plurality of street view images, generating the topological structure diagram of the single diagram corresponding to the single street view image, fusing the topological structure diagram of the plurality of single diagrams corresponding to the plurality of street view images to expand the topological relation, and the topological relation among the signboards can be accurately and detailed described.

Description

Signboard topological relation construction method and device, electronic equipment and storage medium

Technical Field

The disclosure relates to the field of image processing and deep learning in the technical field of artificial intelligence, and in particular relates to a method, a device, electronic equipment and a storage medium for constructing a signboard topological relation.

Background

In reality, each sign has a relative position relationship, and the relative position relationship plays an important role in services such as navigation, point of interest (Point of Interest, POI for short) state update and the like.

In the related art, the relative positional relationship with respect to the sign is described only by coordinates, but the coordinates have a certain deviation, and the relative positional relationship between the signs cannot be accurately represented.

Disclosure of Invention

The disclosure provides a method, a device, electronic equipment and a storage medium for constructing a signboard topological relation.

According to an aspect of the present disclosure, there is provided a method for constructing a signboard topological relation, including: acquiring a plurality of street view images; generating a corresponding first signboard topological relation according to the signboard information in the street view image; and generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

According to another aspect of the present disclosure, there is provided a construction apparatus for a signboard topological relation, including: the first acquisition module is used for acquiring a plurality of street view images; the first generation module is used for generating a corresponding first signboard topological relation according to the signboard information in the street view image; and the second generation module is used for generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of constructing a sign topology as described in one aspect of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform a method of constructing a sign topology according to an aspect of the present disclosure.

According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements a method of construction of a sign topology according to an aspect of the present disclosure.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is a flow diagram of a method of constructing a sign topology according to a first embodiment of the present disclosure;

FIG. 2 is a street view image artwork of a corresponding sign "ABCD" of a method of construction of a sign topology according to a first embodiment of the present disclosure;

FIG. 3 is a flow diagram of a method of constructing a sign topology according to a second embodiment of the present disclosure;

FIG. 4 is a street view image correction chart of a corresponding sign "ABCD" of a method of construction of a sign topology according to a second embodiment of the present disclosure;

FIG. 5 is a street view image rectification of a corresponding sign "EA" of a method of construction of a sign topology according to a second embodiment of the present disclosure;

FIG. 6 is a continuous acquisition trajectory diagram of a method of construction of a sign topology according to a second embodiment of the present disclosure;

FIG. 7 is a street view image acquired by the next track point A ', B' of a method of construction of a sign topology according to a second embodiment of the present disclosure;

FIG. 8 is a person image corresponding to an end point sign "E" of a method of construction of a sign topology according to a second embodiment of the present disclosure;

FIG. 9 is a flow diagram of a method of constructing a sign topology according to a third embodiment of the present disclosure;

FIG. 10 is a flow diagram of a method of constructing a sign topology according to a fourth embodiment of the present disclosure;

FIG. 11 is an overall flow diagram of a method of constructing a sign topology according to a fifth embodiment of the present disclosure;

FIG. 12 is a block diagram of a sign topology construction apparatus according to a first embodiment of the present disclosure;

FIG. 13 is a block diagram of a sign topology construction apparatus according to a second embodiment of the present disclosure;

fig. 14 is a block diagram of an electronic device used to implement construction of a sign topology of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Artificial Intelligence (AI) is a technical science that studies, develops, and simulates the theory, method, technology, and application of human intelligence. At present, the AI technology has the advantages of high automation degree, high accuracy and low cost, and is widely applied.

Image Processing (Image Processing) is a technique in which an Image is analyzed with a computer to achieve a desired result. The image processing is to process the image information by using a computer so as to meet the behaviors of visual psychology or application requirements of people, and has wide application, and is mostly used for mapping, atmospheric science, astronomy, image beautifying, image improvement identification and the like.

Deep Learning (DL) is a new research direction in the field of Machine Learning (ML), and learns the internal rules and presentation layers of sample data, and the information obtained in the Learning process is greatly helpful to the interpretation of data such as text, images and sounds. Its final goal is to have the machine have analytical learning capabilities like a person, and to recognize text, image, and sound data. For the specific research content, the method mainly comprises a neural network system based on convolution operation, namely a convolution neural network; a self-encoding neural network based on a plurality of layers of neurons; and (3) pre-training in a multi-layer self-coding neural network mode, and further optimizing a deep confidence network of the neural network weight by combining the identification information. Deep learning has achieved many results in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization techniques, and other related fields.

The following describes a method, an apparatus, an electronic device, and a storage medium for constructing a signboard topological relation according to an embodiment of the present disclosure with reference to the accompanying drawings.

Fig. 1 is a flow diagram of a method of constructing a sign topology according to a first embodiment of the present disclosure.

As shown in fig. 1, the method for constructing a signboard topological relation according to an embodiment of the present disclosure specifically includes the following steps:

s101, acquiring a plurality of street view images.

In particular, the execution subject of the method for constructing a signboard topological relation according to the embodiment of the present disclosure may be a device for constructing a signboard topological relation provided by the embodiment of the present disclosure, where the device for constructing a signboard topological relation may be a hardware device having a data information processing capability and/or software necessary for driving the hardware device to operate. Alternatively, the execution body may include a workstation, a server, a computer, a user terminal, and other devices. The user terminal comprises, but is not limited to, a mobile phone, a computer, intelligent voice interaction equipment, intelligent household appliances, vehicle-mounted terminals and the like.

In the disclosed embodiments, the acquired street view image may be a street view image captured by an image capture device that includes one or more signs. The image capturing device may include, but is not limited to, a special camera, a smart phone, glasses with a camera function, a helmet, and the like.

S102, generating a corresponding first signboard topological relation according to the signboard information in the street view image.

Specifically, a corresponding single-image signboard topological relation, namely a first signboard topological relation, is generated according to the signboard information of the single-street view image acquired in the step S101. The sign information may include, among other things, the center point of the sign frame and the relative position between the sides. For example, from the relative positional relationship between the center point and the sides of each signboard in the street view image shown in fig. 2, a single image signboard topology as shown in table 1 can be obtained, where the signboard topology includes the relative positional relationship of each signboard in fig. 2.

Table 1 first sign topology corresponding to street view image of fig. 2



A	B	C	D

Here, the blank boxes in table 1 represent the sign topological relation that is not currently involved, for example, the sign topological relation obtained based on the history image, or the sign topological relation that is not yet obtained based on the current image and the history image, and the blank boxes in the subsequent tables have the same meaning as the blank boxes in table 1, and are not described again.

S103, generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

Specifically, according to the first sign topology information corresponding to the street view images generated in step S102, a second sign topology relationship which is a plurality of image fusion signs topology relationships corresponding to the street view images is generated.

In summary, the method for constructing a signboard topological relation according to the embodiment of the present disclosure acquires a plurality of street view images, generates a corresponding first signboard topological relation according to signboard information in the street view images, and generates a second signboard topological relation corresponding to the plurality of street view images according to the plurality of first signboard topological relations corresponding to the plurality of street view images. According to the method for constructing the signboard topological relation, the linear signboard topological relation is constructed by acquiring the plurality of street view images, generating the topological structure diagram of the single diagram corresponding to the single street view image, fusing the topological structure diagrams of the plurality of single diagrams corresponding to the plurality of street view images to expand the topological relation, and the topological relation among the signboards can be accurately and detailed described.

Fig. 3 is a flow diagram of construction of a sign topology according to a second embodiment of the present disclosure. As shown in fig. 3, on the basis of the embodiment shown in fig. 1, the construction of the signboard topological relation according to the embodiment of the present disclosure may specifically include the following steps:

s301, acquiring a plurality of street view images.

Specifically, step S301 in the embodiment of the present disclosure is the same as step S101 in the above embodiment, and will not be described here again.

S302, performing image correction on the street view image to correct the street view image into a street view image with a forward viewing angle.

Specifically, the single street view image obtained in step S301 is subjected to image correction, so that the street view image is corrected to be a street view image with a front view. Because the acquired street view images are different in sources and are influenced by factors such as acquisition positions, angles and directions, if the original image is directly used for constructing the topological relation, larger deviation exists in the result, and the images at different angles and different positions can be converted into a unified front view angle through an image correction algorithm. For example, the street view image shown in fig. 2 is corrected to obtain the street view image with the front view angle shown in fig. 4.

S303, generating a corresponding first signboard topological relation according to the signboard information in the street view image.

Specifically, step S303 in the embodiment of the present disclosure is the same as step S102 in the above embodiment, and will not be described herein.

Step S103 "generating the second signboard topological relation corresponding to the plurality of street view images according to the plurality of first signboard topological relations corresponding to the plurality of street view images" in the above embodiment may specifically include the following steps S304 and S305.

S304, fusing the two first signboard topological relations corresponding to the two street view images with the same signboard to obtain a continuous signboard topological relation.

Specifically, each signboard represents a unique shop in reality, the same shop in different images can be used as a reference point, and two first signboard topological relations corresponding to any two street view images with the same signboard are fused to obtain a continuous signboard topological relation.

For example, the first sign topology corresponding to the street view image shown in fig. 5 is shown in table 2, the same shop sign "a" is shown in fig. 5 and 2, the topology of the sign "EA" is known from the first sign topology shown in table 2, and the topology of the sign "ABCD" is also known from table 1, so that the continuous topology of the sign "EABCD" shown in table 3 can be obtained by fusing table 2 and table 1, and in this way, the topology of the sign in a plurality of continuous images can be constructed into a complete linear topology of the sign.

Table 2 first sign topology corresponding to street view image of fig. 5



	E	A

TABLE 3 continuous topology of the sign "EABCD



E	A	B	C	D

S305, for the street view images which do not have the same signboard with other street view images in the plurality of street view images, determining that the acquisition track acquires the images on the same side of the same basic ground object according to the historical acquisition position, the road network, the shooting direction and the basic ground object of the acquisition track corresponding to the acquisition position of the street view image, and fusing the continuous signboard topological relation after the street view images corresponding to the historical acquisition position are fused with the first signboard topological relation of the street view image to obtain the discontinuous signboard topological relation.

Specifically, for a street view image which does not have the same sign as other street view images in the plurality of street view images, the topology of the sign relationship cannot be performed by adopting the method shown in the step S304, the topology of the sign relationship can be performed by adopting the method shown in the step, that is, whether the images of the same side of the same basic ground object are acquired by the acquisition track is determined according to the historical acquisition position, the road network, the shooting direction and the basic ground object of the acquisition track corresponding to the acquisition position of the street view image, if yes, the continuous sign topology relationship after the street view images corresponding to the historical acquisition position are fused with the first sign topology relationship of the street view image, so that the discontinuous sign topology relationship is obtained. For example, in the continuously collected track diagram shown in fig. 6, a ', B', C 'are track points in the continuously collected track, track point "a'" is the collection position of the street view image in fig. 2, track point "B '" is the collection position of the street view image in fig. 5, track point "C'" is the collection position of the street view image in fig. 7, table 2 corresponding to table 1 and table 5 in fig. 2 can obtain the topological relation "EABCD" shown in table 3, it is known from the continuous information of track points, fig. 7 is the street view image collected by the next track point of a ', and the road network, the shooting direction and the ground object are combined, it is known that points "a'", "B '" and "C'" are images on the same side of the same ground object, and further it is known that sign "F" is on the left side of sign "E", and further, the direct adjacent relation between sign "F" and sign "E" is obtained from fig. 5 and fig. 7, therefore, a certain space is left in the topological relation is left, and the topological relation is only indicated by the topological relation between sign "F" and sign "E" is not shown as the continuous relation of the sign "4.

Table 4 discontinuous sign topology from fig. 5 and 7

F

E

A

B

C

D

S306, acquiring a person acquisition image or a vehicle recorder image corresponding to the end point sign in the discontinuous sign topological relation.

Specifically, when the second sign topological relation corresponding to the plurality of street view images generated by the obtained street view images is a discontinuous sign topological relation, continuing to obtain the people acquisition image or the automobile data recorder image corresponding to the endpoint sign in the discontinuous sign topological relation, wherein the endpoint sign can be a sign on two sides of a vacancy, for example, the people acquisition image corresponding to the endpoint sign 'E' shown in fig. 8 is acquired according to the discontinuous sign topological relation mined through continuous tracks shown in the table 4.

S307, generating a corresponding first signboard topological relation according to the signboard information in the person acquisition image or the automobile data recorder image.

Specifically, a corresponding first sign topological relation is generated according to the sign information in the person acquisition image or the automobile data recorder image acquired in the step S306. The first sign topology of sign "GE" shown in table 5 may be generated, for example, from the person-acquired image shown in fig. 8.

And S308, fusing the person acquisition image or the automobile data recorder image with the street view image corresponding to the end point signboard, and obtaining the continuous signboard topological relation by the corresponding two first signboard topological relations.

Specifically, the person-picked image or the vehicle event data recorder image and the street view image corresponding to the end point signboard are fused, for example, the first signboard topological relation of the signboard "GE" corresponding to fig. 8 and the first signboard topological relation of the signboard "EA" corresponding to fig. 5 are fused, so that the continuous signboard topological relation "GEA" can be obtained, and the topological relation of the signboard "F EABCD" obtained before is combined to further obtain the topological relation of the signboard "FGEABCD" shown in table 5.

Table 5 topological relation of the signboard "FGEABCD

F

G

E

A

B

C

D

As a first possible implementation manner, as shown in fig. 9, based on the example shown in fig. 3, step S302 "image correction on a street view image" may specifically include:

s901, determining vanishing points in street view images.

Specifically, vanishing points in the street view image are determined through straight line detection and line segment clustering. Wherein the vanishing point is the point at which two or more representative parallel lines extend toward the far horizon until convergence.

S902, calculating a perspective transformation matrix of the street view image according to vanishing points.

Specifically, according to vanishing points in the street view image determined in step S901, pixels for calculating the perspective transformation matrix are constructed by using vanishing points in the vertical direction and vanishing points in the horizontal direction, and parallel points found by the vanishing points are substituted into the perspective transformation matrix calculation formula to calculate the perspective transformation matrix of the street view image.

S903, carrying out image correction on the street view image according to the perspective transformation matrix.

Specifically, the perspective transformation matrix of the street view image calculated in step S902 is used to perform image correction on the street view image, and all the signboard frames in the original image of the single street view image are transformed to the front view angle through the perspective transformation matrix.

As a second possible implementation manner, as shown in fig. 10, based on the example shown in fig. 3, step S302 "image correction on a street view image" may specifically further include:

s1001, inputting the street view image into the image correction model to obtain a perspective transformation matrix of the street view image.

Specifically, a data set is constructed according to the vanishing point method, and the data set comprises a plurality of street view images and a plurality of corresponding perspective transformation matrixes. The image correction model can be specifically obtained by training through a spatial transformation network (Spacial Transformer Network, STN) and the like. And (3) inputting the street view image obtained in the step (S301) into an image correction model, and obtaining a perspective transformation matrix of the street view image through model prediction.

S1002, performing image correction on the street view image according to the perspective transformation matrix.

Specifically, step S1002 in the embodiment of the present disclosure is the same as step S903 in the above embodiment, and will not be described here again.

In summary, the method for constructing a signboard topological relation according to the embodiment of the present disclosure acquires a plurality of street view images, generates a corresponding first signboard topological relation according to signboard information in the street view images, and generates a second signboard topological relation corresponding to the plurality of street view images according to the plurality of first signboard topological relations corresponding to the plurality of street view images. According to the method for constructing the signboard topological relation, the linear signboard topological relation is constructed by acquiring the plurality of street view images, generating the topological structure diagram of the single diagram corresponding to the single street view image, fusing the topological structure diagrams of the plurality of single diagrams corresponding to the plurality of street view images to expand the topological relation, and the topological relation among the signboards can be accurately and detailed described. The images are uniformly converted into uniform front view angles through an image correction algorithm, and the topological relation lacking the direct adjacent relation is supplemented by combining the personal image information, so that the accuracy of describing the topological relation between the signboards is improved.

Fig. 11 is an overall flowchart of construction of a sign topology according to an embodiment of the fifth aspect of the present disclosure. As shown in fig. 11, the construction of the signboard topological relation according to the embodiment of the present disclosure specifically includes the following steps:

s1101, a plurality of street view images are acquired.

S1102, selecting an image correction algorithm to correct the image.

If the vanishing point detection method is selected, executing steps S1103-S1104; if the deep learning algorithm is selected, steps S1105 to S1106 are executed.

S1103, determining vanishing points in the street view image.

S1104, calculating a perspective transformation matrix of the street view image according to vanishing points. Execution continues with S1106.

S1105, inputting the street view image into the image correction model to obtain a perspective transformation matrix of the street view image.

S1106, performing image correction on the street view image according to the perspective transformation matrix.

S1107, generating a corresponding first signboard topological relation according to the signboard information in the street view image.

S1108, fusing the two first signboard topological relations corresponding to the two street view images with the same signboard to obtain a continuous signboard topological relation.

S1109, for the street view images which do not have the same signboard with other street view images in the plurality of street view images, determining that the acquisition track acquires the images on the same side of the same basic ground object according to the historical acquisition position, the road network, the shooting direction and the basic ground object of the acquisition track corresponding to the acquisition position of the street view image, and fusing the continuous signboard topological relation after the street view images corresponding to the historical acquisition position and the first signboard topological relation of the street view image to obtain the discontinuous signboard topological relation.

S1110, acquiring a person acquisition image or a vehicle recorder image corresponding to the end point sign in the discontinuous sign topological relation.

S1111, generating a corresponding first signboard topological relation according to the signboard information in the person acquisition image or the automobile data recorder image.

S1112, fusing the person acquisition image or the automobile data recorder image with the street view image corresponding to the end point signboard, and obtaining the continuous signboard topological relation by the corresponding two first signboard topological relations.

Fig. 12 is a block diagram of a construction apparatus of a signboard topological relation according to the first embodiment of the present disclosure.

As shown in fig. 12, a sign topology construction apparatus 1200 according to an embodiment of the present disclosure includes: a first acquisition module 1201, a first generation module 1202, a second generation module 1203.

A first obtaining module 1201 is configured to obtain a plurality of street view images.

A first generation module 1202 is configured to generate a corresponding first sign topology relationship according to sign information in the street view image.

The second generating module 1203 is configured to generate a second sign topological relation corresponding to the plurality of street view images according to the plurality of first sign topological relations corresponding to the plurality of street view images.

The explanation of the embodiment of the method for constructing the topological relation of the signboard is also applicable to the device for constructing the topological relation of the signboard in the embodiment of the disclosure, and the specific process is not repeated here.

In summary, the device for constructing a signboard topological relation according to the embodiment of the present disclosure acquires a plurality of street view images, generates a corresponding first signboard topological relation according to signboard information in the street view images, and generates a second signboard topological relation corresponding to the plurality of street view images according to the plurality of first signboard topological relations corresponding to the plurality of street view images. According to the signboard topological relation constructing device, the topological structure diagrams of the single charts corresponding to the single street view images are obtained and generated, and the topological structure diagrams of the single charts corresponding to the street view images are fused to expand the topological relation, so that the linear signboard topological relation is constructed, and the topological relation among the signboards can be accurately and detailed.

Fig. 13 is a block diagram of a construction apparatus of a signboard topological relation according to the second embodiment of the present disclosure.

As shown in fig. 13, a sign topology construction apparatus 1300 of an embodiment of the present disclosure includes: a first acquisition module 1301, a first generation module 1302, a second generation module 1303.

The first acquiring module 1301 has the same structure and function as the first acquiring module 1201 in the previous embodiment, the first generating module 1302 has the same structure and function as the first generating module 1202 in the previous embodiment, and the second generating module 1303 has the same structure and function as the second generating module 1203 in the previous embodiment.

Further, the construction apparatus 1300 of the sign topology according to the embodiment of the present disclosure further includes: the correction module 1304 is configured to perform image correction on the street view image, so as to correct the street view image into a street view image with a front viewing angle.

Further, the correction module 1304 may specifically include: a determining unit 13041 configured to determine vanishing points in the street view image; a calculating unit 13042 for calculating a perspective transformation matrix of the street view image according to the vanishing points; the first rectification unit 13043 is used for conducting image rectification on the street view image according to the perspective transformation matrix.

Further, the correction module 1304 may specifically include: an input unit 13044 for inputting the street view image into the image correction model to obtain a perspective transformation matrix of the street view image; the second correction unit 13045 performs image correction on the street view image based on the perspective transformation matrix.

Further, the second generating module 1303 may specifically include: and the first fusing unit 13031 is used for fusing two first signboard topological relations corresponding to two street view images with the same signboard to obtain a continuous signboard topological relation.

Further, the second generating module 1303 may specifically further include: and the second fusing unit 13032 is configured to, for the street view images that do not have the same sign as other street view images in the plurality of street view images, determine that the collection track collects the images on the same side of the same basic ground object according to the historical collection position, the road network, the shooting direction and the basic ground object of the collection track corresponding to the collection position of the street view image, fuse the continuous sign topological relation after the street view images corresponding to the historical collection position with the first sign topological relation of the street view image, and obtain the discontinuous sign topological relation.

Further, the construction apparatus 1300 of the sign topology according to the embodiment of the present disclosure further includes: a second acquiring module 1305, configured to acquire a person acquisition image or a vehicle recorder image corresponding to an endpoint sign in the discontinuous sign topological relation; a third generating module 1306, configured to generate a corresponding first sign topological relation according to sign information in the person-in-person image or the vehicle event data recorder image; and the fusion module 1307 is used for fusing the person acquisition image or the automobile data recorder image with the street view image corresponding to the end point signboard, and the two corresponding first signboard topological relations to obtain a continuous signboard topological relation.

In summary, the device for constructing a signboard topological relation according to the embodiment of the present disclosure acquires a plurality of street view images, generates a corresponding first signboard topological relation according to signboard information in the street view images, and generates a second signboard topological relation corresponding to the plurality of street view images according to the plurality of first signboard topological relations corresponding to the plurality of street view images. According to the signboard topological relation constructing device, the topological structure diagrams of the single charts corresponding to the single street view images are obtained and generated, and the topological structure diagrams of the single charts corresponding to the street view images are fused to expand the topological relation, so that the linear signboard topological relation is constructed, and the topological relation among the signboards can be accurately and detailed. The images are uniformly converted into uniform front view angles through an image correction algorithm, and the topological relation lacking the direct adjacent relation is supplemented by combining the personal image information, so that the accuracy of describing the topological relation between the signboards is improved.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information all conform to the regulations of related laws and regulations, and the public sequence is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

Fig. 14 shows a schematic block diagram of an example electronic device 1400 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 14, the apparatus 1400 includes a computing unit 1401 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 1402 or a computer program loaded from a storage unit 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the device 1400 can also be stored. The computing unit 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input/output (I/O) interface 1405 is also connected to the bus 1404.

Various components in device 1400 are connected to I/O interface 1405, including: an input unit 1406 such as a keyboard, a mouse, or the like; an output unit 1407 such as various types of displays, speakers, and the like; a storage unit 1408 such as a magnetic disk, an optical disk, or the like; and a communication unit 1409 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1409 allows the device 1400 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunications networks.

The computing unit 1401 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 1401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1401 performs the respective methods and processes described above, for example, the construction method of the signboard topological relation shown in fig. 1 to 11. For example, in some embodiments, the method of construction of a sign topology may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 1408. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 1400 via the ROM 1402 and/or the communication unit 1409. When the computer program is loaded into the RAM 1403 and executed by the computing unit 1401, one or more steps of the above-described construction method of a signboard topological relation can be performed. Alternatively, in other embodiments, the computing unit 1401 may be configured to perform the construction method of the sign topology by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

According to an embodiment of the present disclosure, the present disclosure further provides a computer program product comprising a computer program, wherein the computer program, when executed by a processor, implements a method of constructing a sign topology as shown in the above embodiments of the present disclosure.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of constructing a sign topology, comprising:

acquiring a plurality of street view images;

generating a corresponding first signboard topological relation according to the signboard information in the street view image, wherein the signboard information comprises a signboard frame center point and relative positions among all sides; and

generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images;

wherein the generating a second sign topological relation corresponding to the street view images according to the first sign topological relations corresponding to the street view images comprises:

fusing two first signboard topological relations corresponding to two street view images with the same signboard to obtain a continuous signboard topological relation;

and for the street view images which do not have the same signboard with other street view images in the plurality of street view images, determining that the acquisition track acquires the images on the same side of the same basic ground object according to the historical acquisition position, the road network, the shooting direction and the basic ground object of the acquisition track corresponding to the acquisition position of the street view image, and fusing the continuous signboard topological relation after the street view images corresponding to the historical acquisition position and the first signboard topological relation of the street view image to obtain the discontinuous signboard topological relation.

2. The construction method according to claim 1, wherein before the generating the corresponding first sign topological relation according to the sign information in the street view image, further comprises:

and carrying out image correction on the street view image so as to correct the street view image into a street view image with a forward viewing angle.

3. The construction method according to claim 2, wherein the performing image correction on the street view image includes:

determining vanishing points in the street view image;

calculating a perspective transformation matrix of the street view image according to the vanishing points;

and carrying out image correction on the street view image according to the perspective transformation matrix.

4. The construction method according to claim 2, wherein the performing image correction on the street view image includes:

inputting the street view image into an image correction model to obtain a perspective transformation matrix of the street view image;

5. The build method of claim 1, further comprising:

acquiring a person acquisition image or a vehicle recorder image corresponding to the end point sign in the discontinuous sign topological relation;

generating a corresponding first sign topological relation according to sign information in the person acquisition image or the automobile data recorder image;

and fusing the person acquisition image or the automobile data recorder image with the street view image corresponding to the end point signboard, and fusing the two corresponding first signboard topological relations to obtain a continuous signboard topological relation.

6. A sign topology construction apparatus comprising:

the first acquisition module is used for acquiring a plurality of street view images;

the first generation module is used for generating a corresponding first signboard topological relation according to the signboard information in the street view image, wherein the signboard information comprises a signboard frame center point and relative positions among all sides; and

the second generation module is used for generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images;

wherein the second generating module includes:

the first fusion unit is used for fusing the two first signboard topological relations corresponding to the two street view images with the same signboard to obtain a continuous signboard topological relation;

and the second fusion unit is used for determining that the acquisition track acquires the images on the same side of the same basic ground object according to the historical acquisition position, the road network, the shooting direction and the basic ground object of the acquisition track corresponding to the acquisition position of the street view image for the street view image which does not have the same signboard with other street view images in the plurality of street view images, and then fusing the continuous signboard topological relation after the street view images corresponding to the historical acquisition position and the first signboard topological relation of the street view image to obtain the discontinuous signboard topological relation.

7. The build apparatus of claim 6, further comprising:

and the correction module is used for carrying out image correction on the street view image so as to correct the street view image into a street view image with a forward viewing angle.

8. The build apparatus of claim 7, wherein the corrective module comprises:

a determining unit, configured to determine vanishing points in the street view image;

a calculation unit for calculating a perspective transformation matrix of the street view image according to the vanishing points;

and the first correction unit is used for carrying out image correction on the street view image according to the perspective transformation matrix.

9. The build apparatus of claim 7, wherein the corrective module comprises:

the input unit is used for inputting the street view image into an image correction model to obtain a perspective transformation matrix of the street view image;

and the second correcting unit is used for carrying out image correction on the street view image according to the perspective transformation matrix.

10. The build apparatus of claim 6, further comprising:

the second acquisition module is used for acquiring a person acquisition image or a vehicle recorder image corresponding to the end point signboard in the discontinuous signboard topological relation;

the third generation module is used for generating the corresponding first signboard topological relation according to the signboard information in the people acquisition image or the automobile data recorder image;

and the fusion module is used for fusing the person acquisition image or the automobile data recorder image with the street view image corresponding to the end point signboard, and the corresponding two first signboard topological relations to obtain a continuous signboard topological relation.

11. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-5.