CN113592980A

CN113592980A - Signboard topological relation construction method and device, electronic equipment and storage medium

Info

Publication number: CN113592980A
Application number: CN202110729812.XA
Authority: CN
Inventors: 甘露; 吴云鹏; 王洪志
Original assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Current assignee: Beijing Baidu Netcom Science and Technology Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2021-11-02
Anticipated expiration: 2041-06-29
Also published as: CN113592980B

Abstract

The disclosure provides a signboard topological relation construction method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, in particular to the fields of image processing and deep learning. The specific implementation scheme is as follows: acquiring a plurality of street view images; generating a corresponding first signboard topological relation according to signboard information in the street view image; and generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images. According to the method for constructing the signboard topological relation, the plurality of street view images are obtained, the single-picture topological structure pictures corresponding to the single street view images are generated, the multiple single-picture topological structure pictures corresponding to the multiple street view images are fused to expand the topological relation, the linear signboard topological relation is constructed, and the topological relation among the signboards can be accurately described in detail.

Description

Signboard topological relation construction method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of image processing and deep learning in the field of artificial intelligence technologies, and in particular, to a method and an apparatus for constructing a topological relationship of a signboard, an electronic device, and a storage medium.

Background

In reality, each signboard has a relative position relationship, and the relative position relationship plays an important role in navigation, Point of Interest (POI) status update and other services.

In the related art, the relative positional relationship of the signs can be described only by coordinates, but the coordinates are deviated to some extent, and the relative positional relationship between the signs cannot be accurately expressed.

Disclosure of Invention

The disclosure provides a signboard topological relation construction method and device, electronic equipment and a storage medium.

According to an aspect of the present disclosure, there is provided a method for constructing a topological relation of a signboard, including: acquiring a plurality of street view images; generating a corresponding first signboard topological relation according to signboard information in the street view image; and generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

According to another aspect of the present disclosure, there is provided a signboard topological relation constructing apparatus including: the first acquisition module is used for acquiring a plurality of street view images; the first generation module is used for generating a corresponding first signboard topological relation according to signboard information in the street view image; and the second generation module is used for generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

According to another aspect of the present disclosure, there is provided an electronic device including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform a method of constructing a signage topological relationship according to an aspect of the present disclosure.

According to another aspect of the present disclosure, there is provided a non-transitory computer readable storage medium storing computer instructions for causing a computer to perform the method of constructing a signboard topological relation according to an aspect of the present disclosure.

According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements a method of building a signage topological relationship according to an aspect of the present disclosure.

It should be understood that the statements in this section do not necessarily identify key or critical features of the embodiments of the present disclosure, nor do they limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The drawings are included to provide a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

fig. 1 is a schematic flow chart of a method of constructing a signboard topology relation according to a first embodiment of the present disclosure;

fig. 2 is a street view image artwork of a corresponding signboard "ABCD" according to a method for constructing a topological relation of a signboard according to a first embodiment of the present disclosure;

FIG. 3 is a flow chart diagram of a method of constructing a signboard topology relation according to a second embodiment of the present disclosure;

fig. 4 is a street view image rectification diagram of a corresponding signboard "ABCD" according to a construction method of a signboard topology relation according to a second embodiment of the present disclosure;

fig. 5 is a street view image rectification diagram of a corresponding signboard "EA" according to a construction method of a signboard topology relation according to a second embodiment of the present disclosure;

FIG. 6 is a diagram of a continuously acquired trajectory of a method of constructing a topological relationship of a signboard according to a second embodiment of the present disclosure;

fig. 7 is a street view image collected at the next track point a ', B' of the construction method of the signboard topology relation according to the second embodiment of the present disclosure;

fig. 8 is a human image corresponding to an end-point signboard "E" of the construction method of the signboard topology relation according to the second embodiment of the present disclosure;

FIG. 9 is a flow chart diagram of a method of constructing a signboard topology relation according to a third embodiment of the present disclosure;

FIG. 10 is a flow chart diagram of a method of constructing a signboard topology relation according to a fourth embodiment of the present disclosure;

fig. 11 is an overall flowchart schematic diagram of a construction method of a signboard topology relation according to a fifth embodiment of the present disclosure;

fig. 12 is a block diagram of a signboard topology relation constructing apparatus according to a first embodiment of the present disclosure;

fig. 13 is a block diagram of a signboard topology relation constructing apparatus according to a second embodiment of the present disclosure;

FIG. 14 is a block diagram of an electronic device used to implement the construction of the signage topology of an embodiment of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the disclosure are included to assist understanding, and which are to be considered as merely exemplary. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Artificial Intelligence (AI) is a technical science that studies and develops theories, methods, techniques and application systems for simulating, extending and expanding human intelligence. At present, the AI technology has the advantages of high automation degree, high accuracy and low cost, and is widely applied.

Image Processing (Image Processing) is a technique that analyzes an Image with a computer to achieve a desired result. The image processing is to process the image information by using a computer to meet the visual psychology of people or the behavior of application requirements, has wide application, and is mainly used for mapping, atmospheric science, astronomy, beautifying, image identification improvement and the like.

Deep Learning (DL) is a new research direction in the field of Machine Learning (ML), and learns the intrinsic rules and representation levels of sample data, and information obtained in the Learning process is helpful for interpreting data such as text, images, and sound. The final aim of the method is to enable the machine to have the analysis and learning capability like a human, and to recognize data such as characters, images and sounds. As for specific research content, the method mainly comprises a neural network system based on convolution operation, namely a convolution neural network; a multilayer neuron based self-coding neural network; and pre-training in a multilayer self-coding neural network mode, and further optimizing the deep confidence network of the neural network weight by combining the identification information. Deep learning has achieved many achievements in search technology, data mining, machine learning, machine translation, natural language processing, multimedia learning, speech, recommendation and personalization technologies, and other related fields.

The method, the apparatus, the electronic device, and the storage medium for constructing a signboard topology relation according to the embodiments of the present disclosure are described below with reference to the drawings.

Fig. 1 is a schematic flow chart of a method of constructing a signboard topology relation according to a first embodiment of the present disclosure.

As shown in fig. 1, the method for constructing a topological relationship of a signboard according to an embodiment of the present disclosure may specifically include the following steps:

s101, a plurality of street view images are obtained.

Specifically, the execution subject of the method for constructing a signboard topological relation according to the embodiment of the present disclosure may be the apparatus for constructing a signboard topological relation provided by the embodiment of the present disclosure, and the apparatus for constructing a signboard topological relation may be a hardware device having a data information processing capability and/or necessary software for driving the hardware device to operate. Alternatively, the execution body may include a workstation, a server, a computer, a user terminal, and other devices. The user terminal includes, but is not limited to, a mobile phone, a computer, an intelligent voice interaction device, an intelligent household appliance, a vehicle-mounted terminal, and the like.

In the disclosed embodiments, the captured street view image may be a street view image taken by an image capture device containing one or more signs. The image capturing device may specifically include, but is not limited to, a dedicated camera, a smartphone, glasses with a camera function, a helmet, and the like.

S102, generating a corresponding first signboard topological relation according to signboard information in the street view image.

Specifically, a corresponding single image signboard topological relation, that is, a first signboard topological relation, is generated according to the signboard information of the single street view image acquired in step S101. The signboard information may specifically include a center point of the signboard frame and a relative position between the edges. For example, according to the relative position relationship between the central point of each signboard frame and each edge in the street view image shown in fig. 2, the topological relationship of the single image signboard shown in table 1 can be obtained, and the topological relationship of the signboard includes the relative position relationship of each signboard in fig. 2.

TABLE 1 first signboard topological relation corresponding to streetscape image of FIG. 2



A	B	C	D

It should be noted here that the blank spaces in table 1 represent the topological relationship of the signboard that is not currently involved, for example, the topological relationship of the signboard obtained based on the historical image, or the topological relationship of the signboard that has not been obtained based on the current image and the historical image, and the blank spaces in the subsequent tables have the same meaning as the blank spaces in table 1, and are not described again.

S103, generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

Specifically, a plurality of image fusion signboard topological relations corresponding to the plurality of street view images, that is, a second signboard topological relation, are generated according to the plurality of first signboard topological information corresponding to the plurality of street view images generated in step S102.

In summary, according to the method for constructing the signboard topological relation in the embodiment of the disclosure, a plurality of street view images are acquired, a corresponding first signboard topological relation is generated according to the signboard information in the street view images, and a second signboard topological relation corresponding to the street view images is generated according to the first signboard topological relations corresponding to the street view images. According to the method for constructing the signboard topological relation, the plurality of street view images are obtained, the single-picture topological structure diagrams corresponding to the single street view image are generated, the multiple single-picture topological structure diagrams corresponding to the multiple street view images are fused to expand the topological relation, the linear signboard topological relation is constructed, and the topological relation among the signboards can be accurately described in detail.

Fig. 3 is a flow diagram of the construction of a signboard topology relation according to a second embodiment of the present disclosure. As shown in fig. 3, based on the embodiment shown in fig. 1, the building of the signboard topology relationship according to the embodiment of the present disclosure may specifically include the following steps:

s301, a plurality of street view images are obtained.

Specifically, step S301 in the embodiment of the present disclosure is the same as step S101 in the embodiment described above, and is not described again here.

And S302, carrying out image rectification on the street view image so as to rectify the street view image into a street view image with a front view angle.

Specifically, the single street view image acquired in step S301 is subjected to image rectification to rectify the street view image into a street view image with a front view angle. Due to different sources of the collected street view images and the influence of factors such as collection positions, angles, directions and the like, if the original image is directly used for constructing the topological relation, the result has larger deviation, and the images at different angles and different positions can be converted into a uniform front-view visual angle through an image correction algorithm. For example, the street view image shown in fig. 2 is subjected to image rectification to obtain the street view image with the front view angle shown in fig. 4.

S303, generating a corresponding first signboard topological relation according to the signboard information in the street view image.

Specifically, step S303 in the embodiment of the present disclosure is the same as step S102 in the embodiment described above, and is not repeated here.

Step S103 "generating a second signboard topological relation corresponding to a plurality of street view images according to a plurality of first signboard topological relations corresponding to a plurality of street view images" in the above embodiment may specifically include the following steps S304 and S305.

S304, fusing two first signboard topological relations corresponding to two street view images with the same signboard to obtain a continuous signboard topological relation.

Specifically, each signboard represents a unique shop in reality, and the topological relations of two first signboards corresponding to any two street view images with the same signboard can be fused by using the same shop in different images as a reference point to obtain a continuous topological relation of the signboards.

For example, the first signboard topological relation corresponding to the street view image shown in fig. 5 is shown in table 2, the same shop signboard "a" in fig. 5 and fig. 2, the topological relation of the signboard "EA" can be known according to the first signboard topological relation shown in table 2, and the topological relation of the signboard "ABCD" can be known from table 1, so that the continuous topological relation of the signboard "EABCD" shown in table 3 can be obtained by fusing table 2 and table 1, and the signboard topological relation in a plurality of continuous images can be constructed into a complete linear signboard topological structure.

TABLE 2 first signboard topological relation corresponding to streetscape image of FIG. 5



	E	A

TABLE 3 consecutive topological relations of the signboard EABCD



E	A	B	C	D

S305, for street view images which do not have the same signboard as other street view images in the street view images, determining that the collected track is an image of the same side of the same basic feature according to the historical collected position, the road network, the shooting direction and the basic feature of the collected track corresponding to the collected position of the street view image, and fusing the continuous signboard topological relation after fusing the street view images corresponding to the historical collected position and the first signboard topological relation of the street view images to obtain a discontinuous signboard topological relation.

Specifically, for street view images which do not have the same signboard as other street view images in the street view images, the topology of the signboard relationship cannot be performed by the method shown in step S304, and the topology of the signboard relationship can be performed by the method shown in this step, that is, whether the acquisition track is an image of the same side of the same basic feature is determined according to the historical acquisition position, the road network, the shooting direction and the basic feature of the acquisition track corresponding to the acquisition position of the street view image, and if so, the continuous signboard topological relationship obtained by fusing the street view images corresponding to the historical acquisition positions is fused with the first signboard topological relationship of the street view images, so as to obtain the discontinuous signboard topological relationship. For example, the continuously collected track diagram shown in fig. 6, a ', B ', C ' are track points in the continuously collected track, track point "a '" is the collection position of the street view image of fig. 2, track point "B '" is the collection position of the street view image of fig. 5, track point "C '" is the collection position of the street view image of fig. 7, the topological relation "EABCD" shown in table 3 can be obtained from table 2 corresponding to table 1 and fig. 5 of fig. 2, and from the continuous information of the track points, fig. 7 is the street view image collected by the next track point of a ', B ' ", and from the road network, the shooting direction and the base feature, it can be known that the points" a ' "," B ' "and" C ' "are taken of the same side of the same base feature, and further it can be known that the sign" F "is on the left side of the sign" E ", and from fig. 5 and fig. 7, the sign" F "and" E "lack a direct adjacent relation, therefore, a certain space is left in the topological relation, and only the left-right relation between the sign "F" and the sign "E" is shown, so that the discontinuous topological relation of the signs as shown in table 4 is obtained.

Table 4 discontinuous signboard topology relations from fig. 5 and 7

F

E

A

B

C

D

S306, acquiring a person acquisition image or a vehicle event data recorder image corresponding to the endpoint signboard in the discontinuous signboard topological relation.

Specifically, when the second signboard topological relation corresponding to the plurality of street view images generated from the obtained street view image is a discontinuous signboard topological relation, the person acquisition image or the tachograph image corresponding to the end signboard in the discontinuous signboard topological relation is continuously obtained, where the end signboard may specifically be a signboard on both sides of the vacancy, and for example, the person acquisition image corresponding to the end signboard "E" shown in fig. 8 is acquired according to the discontinuous signboard topological relation mined through the continuous track shown in table 4.

S307, generating a corresponding first signboard topological relation according to signboard information in the human acquisition image or the automobile data recorder image.

Specifically, the corresponding first signboard topological relation is generated according to the signboard information in the human capture image or the automobile data recorder image acquired in step S306. The first sign topology relation of the sign "GE" shown in table 5 can be generated from the human face image shown in fig. 8, for example.

And S308, fusing the two corresponding first signboard topological relations of the human acquisition image or the automobile data recorder image and the street view image corresponding to the endpoint signboard to obtain a continuous signboard topological relation.

Specifically, the topological relationship between the person-collected image or the tachograph image and the street view image corresponding to the end-point signboard is fused, for example, the topological relationship between the first signboard of the signboard "GE" corresponding to fig. 8 and the topological relationship between the first signboard of the signboard "EA" corresponding to fig. 5 are fused, so as to obtain the continuous topological relationship "GEA" of the signboard, and the topological relationship of the signboard "FGEABCD" shown in table 5 is obtained by combining the previously obtained topological relationship of the signboard "F EABCD".

TABLE 5 topological relationship of signboard FGEABCD

F

G

E

A

B

C

D

As a first possible implementation manner, as shown in fig. 9, based on the above-mentioned embodiment shown in fig. 3, the step S302 "performing image rectification on a street view image" may specifically include:

and S901, determining vanishing points in the street view image.

Specifically, vanishing points in the street view image are determined through line detection and line segment clustering. Wherein the vanishing point is the point at which two or more lines representing parallel lines extend far to the horizon until convergence.

And S902, calculating a perspective transformation matrix of the street view image according to the vanishing point.

Specifically, according to the vanishing point in the street view image determined in step S901, a pixel point of the perspective transformation matrix is calculated by using the vanishing point in the vertical direction and the vanishing point in the horizontal direction, and the perspective transformation matrix of the street view image is calculated by substituting the parallel points found by the vanishing point into a perspective transformation matrix calculation formula.

And S903, carrying out image rectification on the street view image according to the perspective transformation matrix.

Specifically, the street view image is subjected to image rectification according to the perspective transformation matrix of the street view image calculated in step S902, and all signboard frames in the original image of a single street view image are converted into the front view angle through the perspective transformation matrix.

As a second possible implementation manner, as shown in fig. 10, based on the embodiment shown in fig. 3, the step S302 "performing image rectification on the street view image" may specifically further include:

and S1001, inputting the street view image into the image correction model to obtain a perspective transformation matrix of the street view image.

Specifically, a data set is constructed according to the vanishing point method, and the data set comprises a plurality of street view images and a plurality of corresponding perspective transformation matrixes. The image correction model can be obtained by training in a Spatial Transformer Network (STN) mode and the like. And (4) inputting the street view image acquired in the step (S301) into an image rectification model, and obtaining a perspective transformation matrix of the street view image through model prediction.

And S1002, carrying out image rectification on the street view image according to the perspective transformation matrix.

Specifically, step S1002 in the present disclosure is the same as step S903 in the above embodiments, and is not described here again.

In summary, according to the method for constructing the signboard topological relation in the embodiment of the disclosure, a plurality of street view images are acquired, a corresponding first signboard topological relation is generated according to the signboard information in the street view images, and a second signboard topological relation corresponding to the street view images is generated according to the first signboard topological relations corresponding to the street view images. According to the method for constructing the signboard topological relation, the plurality of street view images are obtained, the single-picture topological structure diagrams corresponding to the single street view image are generated, the multiple single-picture topological structure diagrams corresponding to the multiple street view images are fused to expand the topological relation, the linear signboard topological relation is constructed, and the topological relation among the signboards can be accurately described in detail. Images are uniformly converted into uniform front-view visual angles through an image correction algorithm, and the accuracy of describing the topological relation among the signboards is improved by combining the topological relation that the direct adjacent relation is lacked by human-collected image information supplement.

Fig. 11 is an overall flow chart of the construction of a signboard topology relation according to an embodiment of the fifth aspect of the present disclosure. As shown in fig. 11, the construction of the signboard topology relationship according to the embodiment of the present disclosure specifically includes the following steps:

s1101, a plurality of street view images are obtained.

And S1102, selecting an image rectification algorithm to carry out image rectification.

If the detection vanishing point method is selected, executing the steps S1103 to S1104; if the deep learning algorithm is selected, steps S1105-S1106 are executed.

And S1103, determining vanishing points in the street view image.

And S1104, calculating a perspective transformation matrix of the street view image according to the vanishing point. Execution continues with S1106.

S1105, inputting the street view image into the image correction model to obtain the perspective transformation matrix of the street view image.

And S1106, carrying out image rectification on the street view image according to the perspective transformation matrix.

S1107, a corresponding first signboard topological relation is generated according to signboard information in the street view image.

S1108, fusing two first signboard topological relations corresponding to two street view images with the same signboard to obtain a continuous signboard topological relation.

S1109, for street view images which do not have the same signboard as other street view images in the street view images, determining that the acquired tracks are images of the same side of the same basic ground object according to the historical acquisition positions, the road network, the shooting direction and the basic ground object of the acquisition tracks corresponding to the acquisition positions of the street view images, and fusing the continuous signboard topological relation after fusing the street view images corresponding to the historical acquisition positions and the first signboard topological relation of the street view images to obtain the discontinuous signboard topological relation.

S1110, acquiring a person-collected image or a vehicle event data recorder image corresponding to the endpoint signboard in the discontinuous signboard topological relation.

S1111, generating a corresponding first signboard topological relation according to signboard information in the human acquisition image or the automobile data recorder image.

And S1112, fusing the topological relations of the two corresponding first signboards with the street view images corresponding to the human acquisition images or the automobile data recorder images and the endpoint signboards to obtain a continuous topological relation of the signboards.

Fig. 12 is a block diagram of a signboard topology relation constructing apparatus according to a first embodiment of the present disclosure.

As shown in fig. 12, the apparatus 1200 for constructing a topological relation of a signboard according to an embodiment of the present disclosure includes: a first obtaining module 1201, a first generating module 1202, and a second generating module 1203.

A first obtaining module 1201, configured to obtain a plurality of street view images.

The first generating module 1202 is configured to generate a corresponding first signboard topological relation according to signboard information in the street view image.

The second generating module 1203 is configured to generate a second signboard topological relation corresponding to the multiple street view images according to the multiple first signboard topological relations corresponding to the multiple street view images.

It should be noted that the explanation of the embodiment of the method for constructing a signboard topology relationship is also applicable to the apparatus for constructing a signboard topology relationship in the embodiment of the present disclosure, and the specific process is not described herein again.

In summary, the apparatus for constructing a topological relationship of signs in the embodiments of the present disclosure acquires a plurality of street view images, generates a corresponding first topological relationship of signs according to the sign information in the street view images, and generates a second topological relationship of signs corresponding to the street view images according to the plurality of first topological relationships of signs corresponding to the street view images. The device for constructing the signboard topological relation according to the embodiment of the present disclosure obtains a plurality of street view images, generates a single-drawing topological structure diagram corresponding to a single street view image, integrates the multiple single-drawing topological structure diagrams corresponding to the multiple street view images to expand the topological relation, constructs a linear signboard topological relation, and can accurately describe the topological relation between signboards in detail.

Fig. 13 is a block diagram of a signboard topology relation constructing apparatus according to a second embodiment of the present disclosure.

As shown in fig. 13, an apparatus 1300 for constructing a topological relation of a signboard according to an embodiment of the present disclosure includes: a first obtaining module 1301, a first generating module 1302, and a second generating module 1303.

The first obtaining module 1301 has the same structure and function as the first obtaining module 1201 in the previous embodiment, the first generating module 1302 has the same structure and function as the first generating module 1202 in the previous embodiment, and the second generating module 1303 has the same structure and function as the second generating module 1203 in the previous embodiment.

Further, the apparatus 1300 for constructing a topological relation of a signboard according to the embodiment of the present disclosure further includes: the rectification module 1304 is configured to perform image rectification on the street view image, so as to rectify the street view image into a street view image with a front view angle.

Further, the calibration module 1304 may specifically include: a determining unit 13041, configured to determine a vanishing point in the street view image; a calculating unit 13042, configured to calculate a perspective transformation matrix of the street view image according to the vanishing point; and a first rectification unit 13043, configured to perform image rectification on the street view image according to the perspective transformation matrix.

Further, the calibration module 1304 may specifically include: an input unit 13044 configured to input the street view image into the image correction model to obtain a perspective transformation matrix of the street view image; second correcting section 13045 performs image correction on the street view image based on the perspective transformation matrix.

Further, the second generating module 1303 specifically includes: a first fusing unit 13031 is configured to fuse two first signboard topology relations corresponding to two street view images with the same signboard to obtain a continuous signboard topology relation.

Further, the second generating module 1303 may further include: a second fusing unit 13032, configured to determine that the collection track of a street view image, which does not have the same signboard as another street view image in the multiple street view images, is an image of the same side of the same basic feature according to the historical collection position, the road network, the shooting direction, and the basic feature of the collection track corresponding to the collection position of the street view image, and fuse a continuous signboard topological relation after fusing the street view images corresponding to the historical collection position with the first signboard topological relation of the street view image to obtain a discontinuous signboard topological relation.

Further, the apparatus 1300 for constructing a topological relation of a signboard according to the embodiment of the present disclosure further includes: a second obtaining module 1305, configured to obtain a person-collected image or an automobile data recorder image corresponding to an endpoint signboard in a discontinuous signboard topological relation; a third generating module 1306, configured to generate a corresponding first signboard topological relation according to signboard information in the people collection image or the automobile data recorder image; the fusion module 1307 is configured to fuse the two first signboard topological relations corresponding to the street view image corresponding to the person-collected image or the vehicle event data recorder image and the endpoint signboard, so as to obtain a continuous signboard topological relation.

In summary, the apparatus for constructing a topological relationship of signs in the embodiments of the present disclosure acquires a plurality of street view images, generates a corresponding first topological relationship of signs according to the sign information in the street view images, and generates a second topological relationship of signs corresponding to the street view images according to the plurality of first topological relationships of signs corresponding to the street view images. The device for constructing the signboard topological relation according to the embodiment of the present disclosure obtains a plurality of street view images, generates a single-drawing topological structure diagram corresponding to a single street view image, integrates the multiple single-drawing topological structure diagrams corresponding to the multiple street view images to expand the topological relation, constructs a linear signboard topological relation, and can accurately describe the topological relation between signboards in detail. Images are uniformly converted into uniform front-view visual angles through an image correction algorithm, and the accuracy of describing the topological relation among the signboards is improved by combining the topological relation that the direct adjacent relation is lacked by human-collected image information supplement.

In the technical scheme of the disclosure, the acquisition, storage, application and the like of the personal information of the related user all accord with the regulations of related laws and regulations, and do not violate the good customs of the public order.

The present disclosure also provides an electronic device, a readable storage medium, and a computer program product according to embodiments of the present disclosure.

FIG. 14 shows a schematic block diagram of an example electronic device 1400 that can be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 14, the device 1400 includes a computing unit 1401 that can perform various appropriate actions and processes in accordance with a computer program stored in a Read Only Memory (ROM)1402 or a computer program loaded from a storage unit 1408 into a Random Access Memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the device 1400 can also be stored. The calculation unit 1401, the ROM 1402, and the RAM 1403 are connected to each other via a bus 1404. An input/output (I/O) interface 1405 is also connected to bus 1404.

Various components in device 1400 connect to I/O interface 1405, including: an input unit 1406 such as a keyboard, a mouse, or the like; an output unit 1407 such as various types of displays, speakers, and the like; a storage unit 1408 such as a magnetic disk, optical disk, or the like; and a communication unit 1409 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 1409 allows the device 1400 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.

The computing unit 1401 may be a variety of general purpose and/or special purpose processing components having processing and computing capabilities. Some examples of the computing unit 1401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, and the like. The calculation unit 1401 executes the respective methods and processes described above, such as the construction method of the signboard topology relation shown in fig. 1 to 11. For example, in some embodiments, the method of constructing the signboard topology relationship may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 1408. In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1400 via ROM 1402 and/or communication unit 1409. When the computer program is loaded into the RAM 1403 and executed by the computing unit 1401, one or more steps of the method of building a signboard topology relation described above may be performed. Alternatively, in other embodiments, the computing unit 1401 may be configured to perform the construction method of the signboard topological relation by any other suitable means (e.g. by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuitry, Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), system on a chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, receiving data and instructions from, and transmitting data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a back-end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), Wide Area Networks (WANs), the internet, and blockchain networks.

The computer system may include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server with a combined blockchain.

According to an embodiment of the present disclosure, there is also provided a computer program product including a computer program, wherein the computer program, when executed by a processor, implements the method for constructing a signboard topological relation according to the above-described embodiment of the present disclosure.

It should be understood that various forms of the flows shown above may be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure may be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

The above detailed description should not be construed as limiting the scope of the disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may be made in accordance with design requirements and other factors. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present disclosure should be included in the scope of protection of the present disclosure.

Claims

1. A method for constructing a topological relation of a signboard comprises the following steps:

acquiring a plurality of street view images;

generating a corresponding first signboard topological relation according to signboard information in the street view image; and

and generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

2. The construction method according to claim 1, wherein before generating the corresponding first signboard topological relation according to the signboard information in the street view image, the construction method further comprises:

and carrying out image correction on the street view image so as to correct the street view image into a street view image with a front view angle.

3. The construction method according to claim 2, wherein the image rectification of the streetscape image comprises:

determining vanishing points in the street view image;

calculating a perspective transformation matrix of the street view image according to the vanishing point;

and carrying out image rectification on the street view image according to the perspective transformation matrix.

4. The construction method according to claim 2, wherein the image rectification of the streetscape image comprises:

inputting the street view image into an image correction model to obtain a perspective transformation matrix of the street view image;

5. The construction method according to claim 1, wherein the generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images comprises:

and fusing two first signboard topological relations corresponding to two street view images with the same signboard to obtain a continuous signboard topological relation.

6. The construction method according to claim 5, wherein the generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images further comprises:

and for street view images which do not have the same signboard as other street view images in the street view images, determining that the acquisition track acquires images on the same side of the same basic feature according to the historical acquisition position, the road network, the shooting direction and the basic feature of the acquisition track corresponding to the acquisition position of the street view images, and fusing the continuous signboard topological relation after fusing the street view images corresponding to the historical acquisition position and the first signboard topological relation of the street view images to obtain a discontinuous signboard topological relation.

7. The build method of claim 6, further comprising:

acquiring a person collecting image or a vehicle event data recorder image corresponding to the endpoint signboard in the discontinuous signboard topological relation;

generating a corresponding first signboard topological relation according to signboard information in the human acquisition image or the automobile data recorder image;

and fusing the topological relations of the two corresponding first signboards with the human acquisition images or the automobile data recorder images and the street view images corresponding to the end signboards to obtain continuous topological relations of the signboards.

8. An apparatus for constructing a topological relationship for a sign, comprising:

the first acquisition module is used for acquiring a plurality of street view images;

the first generation module is used for generating a corresponding first signboard topological relation according to signboard information in the street view image; and

and the second generation module is used for generating a second signboard topological relation corresponding to the street view images according to the first signboard topological relations corresponding to the street view images.

9. The build device of claim 8, further comprising:

and the correcting module is used for carrying out image correction on the street view image so as to correct the street view image into the street view image with the front view as the visual angle.

10. The build device of claim 9, wherein the remediation module comprises:

the determining unit is used for determining vanishing points in the street view images;

the computing unit is used for computing a perspective transformation matrix of the street view image according to the vanishing point;

and the first correcting unit is used for correcting the image of the street view image according to the perspective transformation matrix.

11. The build device of claim 9, wherein the remediation module comprises:

the input unit is used for inputting the street view image into an image correction model to obtain a perspective transformation matrix of the street view image;

and the second correction unit is used for correcting the image of the street view image according to the perspective transformation matrix.

12. The build apparatus of claim 8, wherein the second generation module comprises:

and the first fusion unit is used for fusing two first signboard topological relations corresponding to two street view images with the same signboard to obtain a continuous signboard topological relation.

13. The build apparatus of claim 12, wherein the second generation module further comprises:

and the second fusion unit is used for determining that the acquisition track acquires images on the same side of the same basic ground object according to the historical acquisition position, the road network, the shooting direction and the basic ground object of the acquisition track corresponding to the acquisition position of the street view image, and fusing the continuous signboard topological relation after the street view image corresponding to the historical acquisition position is fused with the first signboard topological relation of the street view image to obtain the discontinuous signboard topological relation.

14. The build device of claim 13, further comprising:

the second acquisition module is used for acquiring a person acquisition image or a vehicle event data recorder image corresponding to the endpoint signboard in the discontinuous signboard topological relation;

the third generation module is used for generating the corresponding first signboard topological relation according to the signboard information in the human acquisition image or the automobile data recorder image;

and the fusion module is used for fusing the topological relations of the two corresponding first signboards of the person-collected image or the automobile data recorder image and the street view image corresponding to the endpoint signboards to obtain a continuous topological relation of the signboards.

15. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

16. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-7.

17. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-7.