CN117275030A

CN117275030A - Method and device for auditing map

Info

Publication number: CN117275030A
Application number: CN202311268326.8A
Authority: CN
Inventors: 左栋; 张文晖; 邹辉东; 狄琳; 宋欣; 陈达; 胡雅斯
Original assignee: Map Technology Examination Center Of Ministry Of Natural Resources
Current assignee: Map Technology Examination Center Of Ministry Of Natural Resources
Priority date: 2023-09-27
Filing date: 2023-09-27
Publication date: 2023-12-22
Anticipated expiration: 2043-09-27

Abstract

The disclosure provides a method and a device for auditing a map, relates to the field of artificial intelligence, and particularly relates to the field of image processing. The specific implementation scheme is as follows: obtaining auditing elements and the model of equipment for displaying a map to be audited; determining a scale set to be switched of the map to be examined according to the examination elements; determining a corresponding screenshot frequency according to the model of the equipment; the control equipment performs screenshot on a target to-be-examined map set with screenshot frequency to obtain a screenshot set, wherein the target to-be-examined map set is obtained by sequentially adjusting the scale of the to-be-examined map according to the scale set and positioning the scale to the coordinate position appointed by the auditing element; and comparing each screenshot in the screenshot set with a true value map of the same scale, and outputting an auditing result. According to the embodiment, the accuracy and efficiency of the drawing can be improved, and the labor cost is reduced.

Description

Method and device for auditing map

Technical Field

The disclosure relates to the field of artificial intelligence, in particular to the field of image processing, and specifically relates to a method and a device for auditing a map.

Background

With the continuous development of technology, the map products are presented in various modes, and strict auditing is required for the release of the map products. The current auditing process requires professional diagramming personnel to do a great deal of repeated mechanical work to check whether the map product meets the requirements. Not only is a great deal of time wasted, but also there is a risk of human misoperation.

Disclosure of Invention

The present disclosure provides a method, apparatus, device, storage medium, and computer program product for auditing a map.

According to a first aspect of the present disclosure, there is provided a method of auditing a map, comprising: obtaining auditing elements and the model of equipment for displaying a map to be audited; determining a scale set to be switched of the map to be examined according to the auditing element; determining screenshot frequency according to the model of the equipment; the equipment is controlled to sequentially adjust the scale of the map to be inspected according to the scale set according to the screenshot frequency, and screenshot is carried out after the scale is positioned to the coordinate position appointed by the auditing element, so that a screenshot set is obtained; and comparing each screenshot in the screenshot set with a true value map of the same scale to output an auditing result.

According to a second aspect of the present disclosure, there is provided an apparatus for auditing a map, comprising: an acquisition unit configured to acquire an audit element and a model of a device displaying a map to be audited; the switching unit is configured to determine a scale set of the map to be checked to be switched according to the checking element; a determining unit configured to determine a screenshot frequency according to a model of the device; the screenshot unit is configured to control the equipment to sequentially adjust the scale of the map to be examined according to the scale set at the screenshot frequency, position the scale to the coordinate position appointed by the auditing element and then screenshot the scale to obtain a screenshot set; and the auditing unit is configured to compare each screenshot in the screenshot set with a true value map of the same scale and output an auditing result.

According to a third aspect of the present disclosure, there is provided an electronic device comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of the first aspects.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the method of any one of the first aspects.

According to a fifth aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the method of any one of the first or second aspects.

The method for examining the map, which is provided by the embodiment of the disclosure, is an examining method combining computer vision and machine learning technology, can greatly improve the working efficiency of map examination, reduce the labor cost of map examination, and replace the traditional time-consuming and labor-consuming manual examining mode with simple and efficient machine examination.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.

Drawings

The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:

FIG. 1 is an exemplary system architecture diagram to which the present disclosure may be applied;

FIG. 2 is a flow chart of one embodiment of a method of auditing a map according to the present disclosure;

FIG. 3 is a schematic illustration of one application scenario of a method of auditing a map according to the present disclosure;

FIG. 4 is a flow chart of a process of training an aesthetic model in accordance with the method of auditing maps of the present disclosure;

FIG. 5 is a schematic structural view of one embodiment of an apparatus for auditing a map according to the present disclosure;

fig. 6 is a schematic diagram of a computer system suitable for use in implementing embodiments of the present disclosure.

Detailed Description

Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.

Fig. 1 illustrates an exemplary system architecture 100 of a device that may apply the methods of auditing maps of embodiments of the present disclosure.

As shown in fig. 1, the system architecture 100 may include terminals 101, 102, a network 103, a database server 104, and a server 105. The network 103 serves as a medium for providing a communication link between the terminals 101, 102, the database server 104 and the server 105. The network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

The user 110 may interact with the server 105 via the network 103 using the terminals 101, 102 to receive or send messages or the like. The terminals 101, 102 may have various client applications installed thereon, such as model training class applications, map auditing class applications, map class applications, shopping class applications, payment class applications, web browsers, instant messaging tools, and the like.

The terminals 101 and 102 may be hardware or software. When the terminals 101, 102 are hardware, they may be various electronic devices with display screens, including but not limited to smartphones, tablet computers, electronic book readers, MP3 players (Moving Picture Experts Group Audio Layer III, dynamic video experts compression standard audio layer 3), laptop and desktop computers, and the like. When the terminals 101, 102 are software, they can be installed in the above-listed electronic devices. Which may be implemented as multiple software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein.

When the terminals 101 and 102 are software, map-like software may be installed thereon, and a map may be displayed. And the screenshot map after the map class software is called by different terminals can be checked. And the map is used for verifying whether the map provided by the map class software meets the release standard.

Database server 104 may be a database server that provides various services. For example, a database server may have stored therein a sample set. The sample set contains a large number of samples. The sample may include a pending map, a truth map, and annotation similarities of the pending map and the truth map. Thus, the user 110 may also select samples from the sample set stored by the database server 104 via the terminals 101, 102.

The server 105 may also be a server providing various services, such as a background server providing support for various applications displayed on the terminals 101, 102. The background server can train the initial model by utilizing samples in the sample set stored in the database server 104, and the user can apply the trained aesthetic model to conduct map auditing.

The database server 104 and the server 105 may be hardware or software. When they are hardware, they may be implemented as a distributed server cluster composed of a plurality of servers, or as a single server. When they are software, they may be implemented as a plurality of software or software modules (e.g., to provide distributed services), or as a single software or software module. The present invention is not particularly limited herein. Database server 104 and server 105 may also be servers of a distributed system or servers that incorporate blockchains. Database server 104 and server 105 may also be cloud servers, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

It should be noted that, the method for auditing the map provided by the embodiments of the present disclosure is generally performed by the server 105. Accordingly, means for auditing the map is also typically provided in the server 105.

It should be noted that the database server 104 may not be provided in the system architecture 100 in cases where the server 105 may implement the relevant functions of the database server 104.

It should be understood that the number of terminals, networks, database servers, and servers in fig. 1 are merely illustrative. There may be any number of terminals, networks, database servers, and servers, as desired for implementation.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method of auditing maps in accordance with the present disclosure is shown. The method for auditing the map may include the steps of:

and step 201, obtaining auditing elements and the model of the equipment displaying the map to be audited.

In this embodiment, the execution subject (e.g., the server shown in fig. 1) of the method for auditing the map may receive the auditing element set by the auditor and receive the model fed back by the device displaying the map to be audited. The audit element includes at least one of: the national boundary line, the province boundary line, the south China sea islands, the port Australian stations and the province city names are line elements, and the south China sea islands, the port Australian stations and the province city names are point elements. The map to be audited is any map uploaded through a mobile phone, a tablet, a hard disk, a computer and other devices. The device may display the pending map by invoking a pending application (e.g., map class APP). For example, the device enables the pending application and invokes the pending map. And then the device performs screenshot and uploads the screenshot to the server.

And 202, determining a scale set to be switched of the map to be examined according to the examination elements.

And determining the scale set to be switched of the map to be examined according to the corresponding relation between the auditing element and the scale set. In this embodiment, the map scale generally contains 19 types, which are 1000km, 500km, 200km, 100km, 50km, 25km, 20km, 10km, 5km, 2km, 1km, 500m, 200m, 100m, 50m, 20m, 10m, 5m, 2m, respectively. According to the types of the auditing elements (at least one of national boundary line, provincial boundary line, south-sea islands, port Australian stations and provincial and urban names), determining the scale of the auditing of the map to be audited, if the national boundary line is not displayed on the map with the scale of less than 25km, the scale of the auditing of the national boundary line of the map to be audited is set to 1000km, 500km, 200km, 100km, 50km and 25km, which are the scale sets required to be switched for the auditing of the national boundary line. The shape displayed by the south sea islands at the scales of 1000km, 500km and 200km is only blurred and has little significance on the examination and drawing, so that the scales examined on the south sea islands are controlled within the range of 100km-2 m; and the corresponding POI information is required to be checked for the elements of the port Australian stations, and is not displayed at a small scale, so that the checked elements are 500km-25km, and the scale set for checking the port Australian stations to be switched comprises 500km, 200km, 100km, 50km and 25km.

A pending map may be provided with a plurality of audit elements. For example, if the map under review is a world map, the audit elements may include national borders, provincial borders, south-sea islands, harbor Australian stations, provincial names. If the map to be reviewed is only a map of a certain province of inland, the audit elements may include a province boundary and a province city name.

And 203, determining a corresponding screenshot frequency according to the model of the equipment.

In this embodiment, a screenshot is required for each scale for each audit element. In order to increase the auditing speed, the scale is switched at minimum time intervals and then the screenshot is taken. But different devices have different speeds for rendering the map for performance reasons. Poor performing devices require longer time to render a clear map. It is necessary to take screenshots at different frequency of screenshots for different models of equipment. The screenshot frequency may be calculated based on screenshot time, rendering time, and time to restart the user to be reviewed. For example, a better-performance machine type can support a mobile phone high-frequency continuous screenshot function, is set to be screenshot once in 1 second, a switching scale waits for 2 seconds to render, and a switching audit element restarts an application for 3 seconds, so that the screenshot frequency can be screenshot once in every 6 seconds when the audit element is switched and screenshot once in every 3 seconds when the audit element is not switched; and the android mobile phone has limitation in part, for example, high-frequency screenshot in part of the android mobile phone can trigger a system rendering alarm, and the system is adjusted to be restarted for 50 times per screenshot. In this case the frequency of the screen shots is plus the increased time to restart the application for the non-switching audit element.

And 204, the control equipment performs screenshot on the target to-be-examined map set with screenshot frequency to obtain a screenshot set, wherein the target to-be-examined map set is obtained by sequentially adjusting the scale of the to-be-examined map according to the scale set and positioning the scale to the coordinate position appointed by the auditing element.

In this embodiment, the server uses an element and a scale as a unit, and invokes the application to be checked through a debugging tool of the device (for example, ADB (Android Debug Bridge, android debug bridge)) according to the coordinate position (for example, longitude and latitude of the national boundary) of the checking element, locates to the specified scale and coordinate position, and then performs screenshot through the ADB. The screenshot is then returned to the server.

And 205, comparing each screenshot in the screenshot set with a true value map of the same scale, and outputting an auditing result.

In this embodiment, the server compares each screenshot in the screenshot set with a pre-stored truth value map of the same scale. For example, a screenshot of a 100km scale auditing a national boundary would need to be compared to a truth map comprising a 100km scale for the national boundary. The similarity between the screenshot and the truth value map can be calculated to obtain an auditing result, for example, the auditing result is that the similarity is 0.8, or the similarity exceeds 0.7, and the auditing result is that the auditing result passes the auditing. The truth value map is prepared according to the national boundary line drawing method standard of China and world, and can be used for news propaganda pictures, books and periodicals newspaper illustration, advertisement display background pictures, artwork design base pictures and the like, and also can be used as a reference base picture for preparing a public version map. The truth map may be scaled, cropped, etc. to the same scale, same size, and positioned at the same coordinate location as the screenshot.

According to the method provided by the embodiment of the disclosure, the working efficiency of map auditing can be greatly improved by switching the scale with different screenshot frequencies aiming at different equipment, the labor cost of map auditing is reduced, and the traditional time-consuming and labor-consuming manual auditing mode is replaced by simple and efficient machine auditing.

In some optional implementations of this embodiment, comparing each screenshot in the screenshot set with a true value map of the same scale, and outputting an audit result includes: if the type of the auditing element is a line element, respectively extracting line elements from each screenshot in the screenshot set and a truth value map of the same scale, and correspondingly generating a line element diagram to be audited and a truth value line element diagram under different scales according to the extracted line elements; inputting the to-be-examined line element graph and the true value line element graph under each scale into a pre-trained aesthetic graph model, and outputting the similarity of the line elements of the screenshot and the true value map under each scale.

The auditing of the line element is an auditing for a boundary line, for example, a national boundary line, a provincial boundary line, and the like. The image segmentation can be performed on the to-be-examined map and the truth value map according to the color information of the to-be-examined line elements so as to separate potential line elements. Optionally, noise and interference in the image may also be eliminated.

Color information of the line elements can be extracted from the pending map and the truth map by pre-trained pre-processing models, and the line elements in each image are separated from the background. Thus, a binary mask map or segmentation map containing potential line elements may be obtained, named a pending line element map and a true line element map, respectively.

The aesthetic model includes a feature extractor and a similarity comparison network based on a self-attention mechanism.

In this process, the processed review and truth maps are used as inputs to a self-attention feature extractor, which outputs intermediate feature maps of the review and truth maps, respectively, that contain rich local and global structural features (specifically, features of texture, color, etc. of lines). Specifically:

input data: the processed pending and truth maps are input into a self-attention feature extractor. The maps are preprocessed to divide the color information of the line elements and then filtered to reduce noise.

Self-attention feature extractor: the self-attention feature extractor is a deep learning model specifically used for processing pending and truth maps. The model is able to extract rich feature information from both maps. These features not only contain local details of the map, but also capture the overall structure of the map.

And (3) generating an intermediate feature map: after the self-attention feature extractor processes the pending map and the truth value map, an intermediate feature map of the pending map and an intermediate feature map of the truth value map are output. These feature maps contain information about local and global features at each pixel location. They can be regarded as abstract representations of a map.

The similarity comparison network can be of two types, one can directly compare the average pixel-by-pixel distances of two intermediate feature maps to obtain the predicted similarity. Another way is to calculate the similarity after converting the two feature maps into feature vectors, e.g. cosine similarity.

In some optional implementations of this embodiment, comparing each screenshot in the screenshot set with a true value map of the same scale, and outputting an audit result includes: if the type of the audit element is a point element, carrying out text region positioning on each screenshot in the screenshot set, and extracting text information from the text region; and carrying out similarity calculation on the text information in the screenshot under each scale and the corresponding text information in the truth value map, and outputting the similarity of the screenshot under each scale and the point elements of the truth value map.

The audit of the point element refers to the audit of the POI, for example, the name of the city of Kong Australian, province is the point element. For the point elements, checking key POI information points in the map, firstly detecting and positioning a text area on the image, extracting text features by adopting an OCR technology, comparing and judging the text features with corresponding text features in the truth value map, and finally outputting an aesthetic result. The similarity calculation can calculate the distance between the text information and the text information of the truth value map by a cosine similarity method. POI auditing is carried out through OCR recognition technology, and auditing speed and accuracy can be improved compared with manual auditing.

In some optional implementations of this embodiment, the method further includes: and controlling a display to display the comparison information of each auditing element under different scales. In the comparison process, the comparison of one element and one scale can be displayed on a display, so that audit information (elements, scales, coordinates and comparison pictures) can be conveniently checked.

In some optional implementations of this embodiment, the method further includes: and feeding back the model and the scale of the equipment corresponding to the screenshot which is not checked to pass to the map provider. And if the calculated similarity under any scale of any auditing element of the map to be audited is smaller than a preset threshold value, the auditing is considered not to be passed. For example, a map to be reviewed, while the national boundary review is higher than the threshold value at a 50km scale, the review is not passed at a 25km scale where the similarity is less than the threshold value. Or the similarity of the national boundary lines under all scales is higher than a threshold value, but the similarity of the port and australia station names during POI auditing is smaller than the threshold value, and the auditing is not passed. The map provider is facilitated to quickly locate problems in the map.

In some optional implementations of this embodiment, the method further includes: and marking the audit elements which do not pass the audit on the to-be-audited map of the equipment. For example, if the national boundary is wrong, the problem is identified by using the prominent color and can be displayed in real equipment by clicking, so that the problem can be conveniently and quickly positioned by a diagrammer.

In some optional implementations of this embodiment, the method further includes: and storing the screenshot passing the verification as a truth value map according to the verification elements and the scale. And updating the true value map if the map product is confirmed to be qualified. Therefore, the difference between the to-be-examined map and the true value map can be reduced, and the examination speed is increased. For example, the true value map of some business POIs is added, and the difference of the auditing elements can be highlighted only if the true value map is closer to the POIs of the map to be audited, although the auditing of the auditing elements is not influenced, so that the auditing speed can be increased.

In some optional implementations of this embodiment, the method further includes: determining a screenshot amount threshold when restarting a pending application according to the model of the equipment, wherein the pending application is used for calling the pending map; counting the number of screenshots after the last time of starting the application to be checked; and if the times reach the screenshot quantity threshold, controlling the equipment to restart the application to be checked. And the high-frequency screenshot of a part of machine types can cause the rendering alarm of the system, so that a screenshot amount threshold value is required to be set according to the machine types, and the application to be checked is restarted after the screenshot times reach the threshold value. Thus ensuring that the auditing process is carried out stably.

In some optional implementations of this embodiment, the audit element includes at least one of: the national boundary line, the province boundary line, the south China sea islands, the port Australian stations and the province city names are line elements, and the south China sea islands, the port Australian stations and the province city names are point elements. The auditing is carried out according to the types, so that the auditing speed can be increased, and if any auditing item is not successful, the whole map is unqualified.

With continued reference to fig. 3, fig. 3 is a schematic diagram of an application scenario of the method of auditing a map according to the present embodiment. In the application scenario of fig. 3, it is tested whether the maps displayed by various types of devices on the market meet the requirements. The server controls the map displayed on each device to switch the scale according to the requirement of the auditing element, and the switching frequency of different devices is different. And after the scale is switched, screenshot is sent to the server. And comparing the screenshot with the truth map through an audit model by the server, and outputting an audit result. The manual adjustment of the scale is not needed, the manual screenshot is performed again, and the full-automatic auditing is realized.

With further reference to FIG. 4, a flow 400 of a training aesthetic model of the audit map method is shown. The process 400 of training the aesthetic model includes the steps of:

And step 401, obtaining a to-be-examined sample map, a truth value map, and marking similarity of the to-be-examined sample map and the truth value map.

In this embodiment, the execution subject of the method of training the aesthetic model (e.g., the server shown in fig. 1) may obtain a sample set from a database server. Each sample comprises a to-be-examined sample map, a truth value map, and annotation similarity of the to-be-examined sample map and the truth value map. The truth value map is prepared according to the national boundary line drawing method standard of China and world, and can be used for news propaganda pictures, books and periodicals newspaper illustration, advertisement display background pictures, artwork design base pictures and the like, and also can be used as a reference base picture for preparing a public version map. During training, samples are randomly selected from the sample set for training. The pending sample map and the truth map have the same scale, the same size, and are positioned at the same coordinate location. In the sample collection process, the to-be-examined sample map and the truth value map can be adjusted to be in a state capable of being compared in a zooming and cutting mode and the like.

The similarity of the sample map to be inspected and the truth map can be automatically marked through a machine, for example, an aesthetic model trained through the method or other methods in the prior art can be used as an automatic marking tool for automatically marking the similarity of the sample map to be inspected and the truth map. The marking of the data can be realized through manual examination, the similarity between the sample map to be examined and the truth value map is scored manually, if the sample map to be examined and the truth value map are completely consistent, the score is 1, if the sample map to be examined and the truth value map are completely different, the score is 0, and the rest scores are distributed between 0 and 1 according to the similarity degree. And (3) carrying out manual examination on the truth value map and the sample map to be examined, wherein the score is 0.7 (representing that the similarity between the truth value map and the sample map to be examined is 0.7). In order to distinguish from the predicted similarity, the similarity noted in the sample is named as the noted similarity.

And step 402, respectively extracting line elements from the sample map to be inspected and the truth value map, and correspondingly generating a line element map to be inspected and a truth value line element map according to the extracted line elements.

In this embodiment, according to the color information of the line element to be inspected, the image segmentation is performed on the sample map to be inspected and the truth map to separate out the potential line element. Optionally, image processing techniques may also be applied to reduce or remove noise and interference in the pending line element graphs and the truth line element graphs. Image processing techniques such as morphological operations, filters, connected component analysis, etc. may be applied to eliminate noise and interference in the image.

Color information of the line elements can be extracted from the sample map and the truth map through a pre-trained preprocessing model, and the line elements in each image are separated from the background. Thus, a binary mask map or segmentation map containing potential line elements may be obtained, named a pending line element map and a true line element map, respectively.

And step 403, inputting the line element diagram to be examined and the true line element diagram into a feature extractor based on a self-attention mechanism in the initial examination model to obtain an intermediate feature diagram of the sample map to be examined and an intermediate feature diagram of the true value map.

In this embodiment, the aesthetic model includes a feature extractor and similarity comparison network based on a self-attention mechanism.

In this process, the processed sample and truth maps are used as inputs to a self-attention feature extractor, which outputs intermediate feature maps of the sample and truth maps, respectively, that contain rich local and global structural features (specifically, features of texture, color, etc. of lines). Specifically:

input data: the processed pending sample map and truth map are input into a self-attention feature extractor. The sample maps are preprocessed, color information of line elements is segmented, and steps such as filtering are performed to reduce noise.

Self-attention feature extractor: the self-attention feature extractor is a deep learning model and is specially used for processing the pending sample map and the truth map. The model is able to extract rich feature information from both sample maps. These features not only contain local details of the sample map, but also capture the overall structure of the sample map.

And (3) generating an intermediate feature map: after the self-attention feature extractor processes the pending sample map and the truth value map, an intermediate feature map of the pending sample map and an intermediate feature map of the truth value map are output. These feature maps contain information about local and global features at each pixel location. They can be regarded as abstract representations of the sample map.

And step 404, inputting the intermediate feature map of the sample map to be examined and the intermediate feature map of the truth value map into a similarity comparison network in the initial aesthetic model to obtain the predicted similarity.

In this embodiment, the similarity comparison network may be of two types, one of which can directly compare the average pixel-by-pixel distances of the two intermediate feature maps to obtain the predicted similarity. Another way is to calculate the similarity after converting the two feature maps into feature vectors, e.g. cosine similarity.

Step 405, adjusting network parameters of a self-attention mechanism based feature extractor and/or similarity comparison network in the initial aesthetic model based on differences in predicted and annotated similarities.

In this embodiment, the present application performs supervised training, uses the labeled similarity as a supervision signal, and calculates the loss value according to the difference between the predicted similarity and the labeled similarity. The method comprises the steps of adjusting network parameters of any one of the self-attention mechanism-based feature extractor and the similarity comparison network in the aesthetic model, or simultaneously adjusting the network parameters of the self-attention mechanism-based feature extractor and the similarity comparison network so that loss values are converged. The re-selection of samples repeats steps 401-405 until the loss value converges to a predetermined value.

In some optional implementations of this embodiment, the extracting line elements from the pending sample map and the truth map respectively, and generating a pending line element map and a truth line element map according to the extracted line element correspondence, includes: extracting color information of line elements from the sample map to be examined and the truth value map; and separating the line elements in the to-be-inspected sample map and the truth value map from the background by using an image segmentation technology based on the color information of the line elements to obtain a to-be-inspected line element map and a truth value line element map. First, color information of line elements is extracted from a sample map to be examined and a truth value map. This may be achieved by color space conversion, filtering and enhancement steps to ensure that the color of the line elements is highlighted in the image. The color information of the line elements in the image may then be separated from the background using image segmentation techniques, such as segmentation methods based on threshold, edge detection, region growing, or deep learning. This allows a binary mask or segmentation map to be obtained that contains potential line elements, named pending line element map and true line element map. The line element diagrams are directly extracted for comparison, so that the calculated amount can be reduced, and the prediction accuracy is improved.

In some optional implementations of this embodiment, the method further includes: image processing techniques are applied to reduce or remove noise and interference in the pending line element graphs and the truth line element graphs. Image processing techniques such as morphological operations, filters, connected component analysis, etc. may be applied to reduce or remove noise and interference in the image. This helps preserve the accuracy of the line elements under review.

In some optional implementations of this embodiment, the inputting the pending line element graph and the truth line element graph into the self-attention mechanism-based feature extractor in the initial aesthetic model obtains an intermediate feature graph of the pending sample map and an intermediate feature graph of the truth value map, including: dividing the line element diagram to be inspected and the true line element diagram into blocks with fixed sizes respectively to obtain a sub-image set to be inspected and a true sub-image set; and carrying out global feature extraction on the sub-image set to be inspected and the true sub-image set to obtain an intermediate feature map of the sample map to be inspected and an intermediate feature map of the true map. This is a self-attention mechanism based feature extractor of the VIT (Vision Transformer) approach. The VIT method has been successfully applied in the field of natural language processing, and local and global features in a map can be captured by dividing an image to obtain a plurality of local images and then extracting global features from each local image by using a transducer mechanism.

In some optional implementations of this embodiment, the inputting the pending line element graph and the truth line element graph into the self-attention mechanism-based feature extractor in the initial aesthetic model obtains an intermediate feature graph of the pending sample map and an intermediate feature graph of the truth value map, including: extracting a local feature map of a to-be-examined sample map and a local feature map of a truth value map from the to-be-examined line element map and the truth value line element map respectively based on a convolutional neural network; and carrying out global feature extraction on the local feature map of the sample map to be inspected and the local feature map of the truth value map respectively to obtain an intermediate feature map of the sample map to be inspected and an intermediate feature map of the truth value map. The method combines the advantages of CNN (convolutional neural network) and the advantages of a transducer, firstly utilizes CNN to extract a local feature map, and then utilizes the transducer to extract global features, so that features can be extracted at a higher level of abstraction, and the content of a map can be better understood. Either way, the self-attention feature extractor can effectively extract features from the pending sample map and the truth map. The characteristics have key roles in the subsequent comparison of the predicted similarity and the labeled similarity, and are used for judging the matching degree of the line elements of the sample map to be examined and the line elements of the true value map, so that an automatic picture examination process is realized.

In some optional implementations of this embodiment, inputting the intermediate feature map of the pending sample map and the intermediate feature map of the truth map into a similarity comparison network in the initial aesthetic model to obtain the predicted similarity includes: splicing the intermediate feature map of the sample map to be examined and the intermediate feature map of the truth value map to obtain a complete feature map; expanding the complete feature map into feature vectors, and inputting the feature vectors into a multi-layer perceptron in the similarity comparison network to obtain perception vectors; and inputting the perception vector into a Sigmoid function in the similarity comparison network to obtain the predicted similarity. The method adopts a deep learning technology and is used for comparing the similarity between the sample map to be examined and the truth value map. In the process, the intermediate features of the sample map to be examined and the intermediate features of the truth value map are firstly taken as inputs to the similarity comparison network. The two features are then stitched together to integrate the information of both. The feature map is then expanded into feature vectors to facilitate subsequent full join operations. The feature vectors are processed more deeply by a multi-layer perceptron (MLP) to learn more advanced feature expressions. Finally, by mapping the output of the MLP to between 0 and 1 by applying the Sigmoid function, a similarity score between the pending sample map and the truth map is obtained. This score can accurately measure the degree of similarity between the two, a score close to 1 indicates a high degree of similarity, and a score close to 0 indicates a low degree of similarity. The deep learning method comprehensively utilizes various technologies, so that the result of the automatic aesthetic drawing can be evaluated more accurately.

In some optional implementations of this embodiment, the expanding the complete feature map into feature vectors includes: rolling and pooling the complete feature map to obtain a compressed feature map; and expanding the compressed feature map into feature vectors. Downsampling and channel number reduction processes are performed on the stitched features by convolution and pooling operations, thereby compressing the features and capturing a higher level representation of the features while reducing the number of parameters for subsequent processing. Features of the compressed feature map may be extracted by convolutional neural networks and cyclic neural networks. The features of the compressed feature graphs are subjected to feature fusion through a fusion layer to obtain feature vectors.

In some optional implementations of this embodiment, the extracting line elements from the to-be-inspected sample map and the truth map respectively, and generating a to-be-inspected line element map and a truth line element map according to the extracted line element correspondence, includes: and respectively preprocessing the to-be-examined sample map and the truth value map through a deep learning segmentation model to obtain a to-be-examined line element map and a truth value line element map.

In some optional implementations of this embodiment, the method further includes: and adjusting network parameters of the segmentation model based on the difference between the predicted similarity and the labeled similarity. If the segmentation model is adopted for preprocessing, network parameters of the segmentation model can be adjusted in the process of training the aesthetic drawing model, so that the segmentation model suitable for the aesthetic drawing is obtained.

The method provided by the above embodiment of the present disclosure examines line elements on a sample map to check whether there are errors, omissions, overlaps, and the like by using an image self-attention mechanism algorithm and a computer vision technique. The automatic line element inspection can improve the efficiency and accuracy of sample map inspection, and meanwhile, the artificial errors are reduced.

With further reference to fig. 5, as an implementation of the method shown in the foregoing figures, the present disclosure provides an embodiment of an apparatus for auditing maps, where the apparatus embodiment corresponds to the method embodiment shown in fig. 2, and the apparatus may be specifically applied to various electronic devices.

As shown in fig. 5, the apparatus 500 for auditing a map of the present embodiment includes: an acquisition unit 501, a switching unit 502, a determination unit 503, a screenshot unit 504, and an auditing unit 505. Wherein, the obtaining unit 501 is configured to obtain the audit element and the model of the device displaying the map to be audited; a switching unit 502, configured to determine, according to the audit element, a set of scales that the map to be audited needs to be switched; a determining unit 503 configured to determine a corresponding screenshot frequency according to a model of the device; the screenshot unit 504 is configured to control the device to screenshot the target to-be-examined map set at the screenshot frequency to obtain a screenshot set, wherein the target to-be-examined map set is obtained by sequentially adjusting the scale of the to-be-examined map according to the scale set and positioning the scale to the coordinate position designated by the auditing element; and the auditing unit 505 is configured to compare each screenshot in the screenshot set with a true value map of the same scale and output an auditing result.

In this embodiment, specific processing of the obtaining unit 501, the switching unit 502, the determining unit 503, the screenshot unit 504, and the auditing unit 505 of the apparatus 500 for auditing a map may refer to steps 201 to 205 in the corresponding embodiment of fig. 2.

In some optional implementations of the present embodiment, the auditing unit 505 is further configured to: if the type of the auditing element is a line element, respectively extracting line elements from each screenshot in the screenshot set and a truth value map of the same scale, and correspondingly generating a line element diagram to be audited and a truth value line element diagram under different scales according to the extracted line elements; inputting the to-be-examined line element graph and the true value line element graph under each scale into a pre-trained aesthetic graph model, and outputting the similarity of the line elements of the screenshot and the true value map under each scale.

In some optional implementations of the present embodiment, the apparatus 500 further includes a training unit (not shown in the drawings) configured to: obtaining a to-be-examined sample map, a truth value map, and marking similarity of the to-be-examined sample map and the truth value map; extracting line elements from the to-be-inspected sample map and the truth value map respectively, and correspondingly generating a to-be-inspected line element map and a truth value line element map according to the extracted line elements; inputting the line element diagram to be examined and the truth line element diagram into a feature extractor based on a self-attention mechanism in an initial examination diagram model to obtain an intermediate feature diagram of a sample map to be examined and an intermediate feature diagram of a truth value map; inputting the intermediate feature map of the sample map to be examined and the intermediate feature map of the truth value map into a similarity comparison network in the initial trial model to obtain predicted similarity; network parameters of a self-attention mechanism-based feature extractor and/or similarity comparison network in the initial aesthetic model are adjusted based on the difference between the predicted similarity and the labeled similarity.

In some optional implementations of this embodiment, the training unit is further configured to: splicing the intermediate feature map of the sample map to be examined and the intermediate feature map of the truth value map to obtain a complete feature map; expanding the complete feature map into feature vectors, and inputting the feature vectors into a multi-layer perceptron in the similarity comparison network to obtain perception vectors; and inputting the perception vector into a Sigmoid function of the similarity comparison network to obtain the predicted similarity.

In some optional implementations of this embodiment, the training unit is further configured to: rolling and pooling the complete feature map to obtain a compressed feature map; and expanding the compressed feature map into feature vectors.

In some optional implementations of the present embodiment, the auditing unit 505 is further configured to: if the type of the audit element is a point element, carrying out text region positioning on each screenshot in the screenshot set, and extracting text information from the text region; and carrying out similarity calculation on the text information in the screenshot under each scale and the corresponding text information in the truth value map, and outputting the similarity of the screenshot under each scale and the point elements of the truth value map.

In some optional implementations of the present embodiment, the apparatus 500 further includes an output unit (not shown in the drawings) configured to: and controlling a display to display the comparison information of each auditing element under different scales.

In some optional implementations of the present embodiment, the apparatus 500 further includes a feedback unit (not shown in the drawings) configured to: and feeding back the model and the scale of the equipment corresponding to the screenshot which is not checked to pass to the map provider.

In some optional implementations of the present embodiment, the apparatus 500 further includes a marking unit (not shown in the drawings) configured to: and marking the audit elements which do not pass the audit on the to-be-audited map of the equipment.

In some optional implementations of the present embodiment, the apparatus 500 further includes a storage unit (not shown in the drawings) configured to: and storing the screenshot passing the verification as a truth value map according to the verification elements and the scale.

In some optional implementations of the present embodiment, the apparatus 500 further includes a restarting unit (not shown in the drawings) configured to: determining a screenshot amount threshold when restarting a pending application according to the model of the equipment, wherein the pending application is used for calling the pending map; counting the number of screenshots after the last time of starting the application to be checked; and if the times reach the screenshot quantity threshold, controlling the equipment to restart the application to be checked.

In some optional implementations of this embodiment, the audit element includes at least one of: the national boundary line, the province boundary line, the south China sea islands, the port Australian stations and the province city names are line elements, and the south China sea islands, the port Australian stations and the province city names are point elements.

In the technical scheme of the disclosure, the related processes of collecting, storing, using, processing, transmitting, providing, disclosing and the like of the personal information of the user accord with the regulations of related laws and regulations, and the public order colloquial is not violated.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of flow 200 or 400.

A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method of flow 200 or 400.

A computer program product comprising a computer program that when executed by a processor implements the method of flow 200 or 400.

Fig. 6 illustrates a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.

As shown in fig. 6, the apparatus 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a Read Only Memory (ROM) 602 or a computer program loaded from a storage unit 608 into a Random Access Memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 may also be stored. The computing unit 601, ROM 602, and RAM 603 are connected to each other by a bus 604. An input/output (I/O) interface 605 is also connected to bus 604.

Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606 such as a keyboard, mouse, etc.; an output unit 607 such as various types of displays, speakers, and the like; a storage unit 608, such as a magnetic disk, optical disk, or the like; and a communication unit 609 such as a network card, modem, wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

The computing unit 601 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 601 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the respective methods and processes described above, for example, a method of auditing a map. For example, in some embodiments, the method of auditing a map may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 600 via the ROM 602 and/or the communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the method of auditing maps described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured to perform the method of auditing the map by any other suitable means (e.g., by means of firmware).

Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.

The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel or sequentially or in a different order, provided that the desired results of the technical solutions of the present disclosure are achieved, and are not limited herein.

The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.

Claims

1. A method of auditing a map, comprising:

obtaining auditing elements and the model of equipment for displaying a map to be audited;

determining a scale set to be switched of the map to be examined according to the auditing element;

determining a corresponding screenshot frequency according to the model of the equipment;

controlling the equipment to carry out screenshot on a target to-be-examined map set according to the screenshot frequency to obtain a screenshot set, wherein the target to-be-examined map set is obtained by sequentially adjusting the scale of the to-be-examined map according to the scale set and positioning the scale to the coordinate position appointed by the auditing element;

And comparing each screenshot in the screenshot set with a true value map of the same scale to output an auditing result.

2. The method of claim 1, wherein comparing each screenshot in the set of shots to a true value map of a scale to output an audit result comprises:

if the type of the auditing element is a line element, respectively extracting line elements from each screenshot in the screenshot set and a truth value map of the same scale, and correspondingly generating a line element diagram to be audited and a truth value line element diagram under different scales according to the extracted line elements;

inputting the to-be-examined line element graph and the true value line element graph under each scale into a pre-trained aesthetic graph model, and outputting the similarity of the line elements of the screenshot and the true value map under each scale.

3. The method of claim 2, wherein the aesthetic model is trained by:

obtaining a to-be-examined sample map, a truth value map, and marking similarity of the to-be-examined sample map and the truth value map;

extracting line elements from the to-be-inspected sample map and the truth value map respectively, and correspondingly generating a to-be-inspected line element map and a truth value line element map according to the extracted line elements;

Inputting the line element diagram to be examined and the truth line element diagram into a feature extractor based on a self-attention mechanism in an initial examination diagram model to obtain an intermediate feature diagram of a sample map to be examined and an intermediate feature diagram of a truth value map;

inputting the intermediate feature map of the sample map to be examined and the intermediate feature map of the truth value map into a similarity comparison network in the initial trial model to obtain predicted similarity;

network parameters of a self-attention mechanism-based feature extractor and/or similarity comparison network in the initial aesthetic model are adjusted based on the difference between the predicted similarity and the labeled similarity.

4. A method according to claim 3, wherein said inputting the intermediate feature map of the pending sample map and the intermediate feature map of the truth map into a similarity comparison network in the initial aesthetic model yields a predicted similarity, comprising:

splicing the intermediate feature map of the sample map to be examined and the intermediate feature map of the truth value map to obtain a complete feature map;

expanding the complete feature map into feature vectors, and inputting the feature vectors into a multi-layer perceptron in the similarity comparison network to obtain perception vectors;

And inputting the perception vector into a Sigmoid function of the similarity comparison network to obtain the predicted similarity.

5. The method of claim 4, wherein the expanding the complete feature map into feature vectors comprises:

rolling and pooling the complete feature map to obtain a compressed feature map;

and expanding the compressed feature map into feature vectors.

6. The method of claim 1, wherein comparing each screenshot in the set of shots to a true value map of a scale to output an audit result comprises:

if the type of the audit element is a point element, carrying out text region positioning on each screenshot in the screenshot set, and extracting text information from the text region;

and carrying out similarity calculation on the text information in the screenshot under each scale and the corresponding text information in the truth value map, and outputting the similarity of the screenshot under each scale and the point elements of the truth value map.

7. The method of claim 1, wherein the method further comprises:

and controlling a display to display the comparison information of each auditing element under different scales.

8. The method of claim 1, wherein the method further comprises:

And feeding back the model and the scale of the equipment corresponding to the screenshot which is not checked to pass to the map provider.

9. The method of claim 1, wherein the method further comprises:

and marking the audit elements which do not pass the audit on the to-be-audited map of the equipment.

10. The method of claim 1, wherein the method further comprises:

and storing the screenshot passing the verification as a new truth value map according to the verification elements and the scale.

11. The method of claim 1, wherein the method further comprises:

determining a screenshot amount threshold when restarting a pending application according to the model of the equipment, wherein the pending application is used for calling the pending map;

counting the number of screenshots after the last time of starting the application to be checked;

and if the times reach the screenshot quantity threshold, controlling the equipment to restart the application to be checked.

12. An apparatus for auditing a map, comprising:

an acquisition unit configured to acquire an audit element and a model of a device displaying a map to be audited;

the switching unit is configured to determine a scale set of the map to be checked to be switched according to the checking element;

a determining unit configured to determine a corresponding screenshot frequency according to a model of the device;

The screenshot unit is configured to control the equipment to screenshot the target to-be-examined map set at the screenshot frequency to obtain a screenshot set, wherein the target to-be-examined map set is obtained by sequentially adjusting the scale of the to-be-examined map according to the scale set and positioning the scale to the coordinate position appointed by the auditing element;

and the auditing unit is configured to compare each screenshot in the screenshot set with a true value map of the same scale and output an auditing result.

13. The apparatus of claim 12, wherein the auditing unit is further configured to:

14. The apparatus of claim 13, wherein the apparatus further comprises a training unit configured to:

15. The apparatus of claim 14, wherein the training unit is further configured to:

16. The apparatus of claim 15, wherein the training unit is further configured to:

and expanding the compressed feature map into feature vectors.

17. The apparatus of claim 12, wherein the auditing unit is further configured to:

18. The apparatus of claim 12, wherein the apparatus further comprises an output unit configured to:

19. The apparatus of claim 12, wherein the apparatus further comprises a feedback unit configured to:

20. The apparatus of claim 12, wherein the apparatus further comprises a marking unit configured to:

21. The apparatus of claim 12, wherein the apparatus further comprises a holding unit configured to:

22. The apparatus of claim 12, wherein the apparatus further comprises a restart unit configured to:

23. An electronic device, comprising:

at least one processor; and

A memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-11.

24. A non-transitory computer readable storage medium storing computer instructions for causing the computer to perform the method of any one of claims 1-11.