CN116432633A

CN116432633A - Address error correction method, device, computer equipment and readable medium

Info

Publication number: CN116432633A
Application number: CN202111673270.5A
Authority: CN
Inventors: 张定棋; 周训飞; 王小龙
Original assignee: Fengtu Technology Shenzhen Co Ltd
Current assignee: Fengtu Technology Shenzhen Co Ltd
Priority date: 2021-12-31
Filing date: 2021-12-31
Publication date: 2023-07-14

Abstract

The application discloses an address error correction method, an address error correction device, computer equipment and a readable medium, wherein the method comprises the following steps: determining conflict levels in the address text to be processed, and calling a prediction model corresponding to the conflict levels; carrying out vectorization processing on each word and the corresponding word level in different address level information in the address text to obtain a spliced vector formed by combining the word vector and the word level vector, inputting the spliced vector into a prediction model corresponding to a conflict level, outputting address level data corresponding to the conflict level, and updating according to the address text; according to the method and the device, the corresponding prediction models are configured for different address levels, the levels with conflicts in the address text are predicted and corrected to obtain the standard address, the problem of route errors caused by the information of the conflicting address levels can be effectively reduced, the logistics cost is reduced, and meanwhile, the database maintenance cost is reduced without depending on a database.

Description

Address error correction method, device, computer equipment and readable medium

Technical Field

The application belongs to the technical field of geographic information, and in particular relates to an address error correction method, an address error correction device, computer equipment and a readable medium.

Background

In a complete logistics distribution system, the distribution system firstly needs to carry out hierarchical matching according to the address of the customer order, and after the matching is completed, the routing distribution system carries out routing order of goods. However, because of the diversity of Chinese address writing, some users cannot provide standard addresses which can be directly identified by the system, so that subsequent dispatch service is affected, and even wrong dispatch is caused.

The conventional solution is to build and maintain a white list address library (dictionary) according to the existing address, and pre-determine address text matching rules, and then match the address with the existing address in the white list address library based on the matching rules after obtaining the address provided by the client, so as to obtain the standard address. However, the rule matching is easy to have matching errors, so that express routing errors are caused, and logistics cost is increased; meanwhile, the dictionary and the matching rules need to be maintained in real time, chinese address writing methods are various, and the maintenance cost of the address library of the mapping relation between the comprehensive and exhaustive address text and the different address layers in many-to-one mode is high.

Disclosure of Invention

Aiming at least one defect or improvement requirement of the prior art, the application provides an address error correction method, an address error correction device, computer equipment and a readable medium, and aims to solve the problems that the conventional dictionary matching-based mode determines standard addresses, matching errors are easy to occur and the maintenance cost of an address library is high.

To achieve the above object, according to a first aspect of the present application, there is provided an address error correction method, comprising:

acquiring an address text to be processed;

determining a conflict level in the address text, and calling a prediction model corresponding to the conflict level;

carrying out vectorization processing on each word and the corresponding word level in different address level information in the address text to obtain a spliced vector formed by combining a word vector and a word level vector;

inputting the spliced vector into the prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data; the prediction model is obtained by training sample address texts with address level labels corresponding to conflict levels, and each sample address text is processed into a spliced vector and then input into the prediction model with the corresponding address level label.

In some embodiments of the present application, after the obtaining the address text to be processed, the method further includes:

and matching the address text with a standard address in a pre-configured white list address library, wherein the white list address library is used for storing the association relation between address level information of the standard address, and if the matching is successful, determining the address level information corresponding to the conflict level in the address text according to the association relation.

In some embodiments of the present application, the determining the conflict hierarchy in the address text includes:

preprocessing the address text, wherein the preprocessing comprises address text normalization, address word segmentation and/or word level filtering;

word segmentation processing is carried out on the address text after pretreatment, and address level information of different address levels is obtained;

and determining conflict levels in the address text according to the level relation among the address level information.

In some embodiments of the present application, inputting the splice vector into the prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data, including:

inputting the spliced vector into a prediction model corresponding to the conflict level, and carrying out convolution processing on the spliced vector through the prediction model to obtain candidate address level data corresponding to the conflict level and confidence corresponding to the candidate address level data; the prediction model comprises a primary city prediction model, a secondary street prediction model, a tertiary community prediction model, a quaternary website prediction model and a five-level interest surface prediction model;

Setting the candidate address level data with the highest confidence as the address level data corresponding to the conflict level, and updating the address text according to the address level data.

In some embodiments of the present application, the training process of the predictive model includes:

acquiring a first sample address text, wherein the first sample address text is provided with an address level label corresponding to a conflict level;

carrying out vectorization processing on each word of different address levels and the word level corresponding to each word in the first sample address text to obtain a sample splicing vector formed by combining a sample word vector and a sample word level vector;

obtaining a first training sample set according to the sample splicing vector corresponding to each first sample address text and the address level label corresponding to the conflict level;

and performing model training according to the first training sample set to obtain a trained prediction model.

In some embodiments of the present application, the training of the model according to the first training sample set to obtain a trained prediction model includes:

generating address level prediction data corresponding to the conflict level according to the sample splicing vector through a prediction model to be trained;

Calculating errors between address level data corresponding to the conflict levels and corresponding address level labels, and reversely adjusting model parameters of the prediction model to be trained according to the errors;

and returning to the prediction model to be trained, and continuously executing the step of generating address level prediction data corresponding to the conflict level according to the sample splicing vector until the iteration stopping condition is met, and stopping iteration to obtain a trained prediction model.

In some embodiments of the present application, the generating address level prediction data corresponding to the conflict level according to the sample concatenation vector specifically includes:

extracting features of the sample spliced vectors to obtain corresponding maximum pooling feature vectors, average pooling feature vectors and weight feature vectors;

generating at least one candidate address level data corresponding to a conflict level according to the maximum pooling feature vector, the average pooling feature vector and the weight feature vector, wherein each candidate address level data has a corresponding confidence;

and selecting the candidate address level data with the highest confidence as the address level prediction data corresponding to the conflict level.

In some embodiments of the present application, the address error correction method further includes:

When the model updating condition is met, a second training sample set is obtained; the second training sample set comprises sample splicing vectors corresponding to the second sample address text and address level labels corresponding to conflict levels;

and carrying out iterative updating on the prediction model according to the second training sample set to obtain an updated prediction model, and taking the updated prediction model as a trained prediction model.

According to a second aspect of the present application, there is also provided an address error correction apparatus, comprising:

the acquisition module is used for acquiring the address text to be processed;

the conflict judging module is used for determining a conflict level in the address text;

the vector generation module is used for carrying out vectorization processing on each word and the corresponding word level in different address level information in the address text to obtain a spliced vector formed by combining the word vector and the word level vector;

and the prediction module is used for inputting the splicing vector into the prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data.

According to a third aspect of the present application there is also provided a computer device comprising at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program which, when executed by the processing unit, causes the processing unit to perform the steps of any of the methods described above.

According to a fourth aspect of the present application there is also provided a computer readable medium storing a computer program executable by a computer device, the computer program, when run on the computer device, causing the computer device to perform the steps of any one of the methods described above.

In general, compared with the prior art, the above technical solutions conceived by the present application can achieve the following beneficial effects:

the address error correction method, the device, the computer equipment and the readable medium provided by the application determine conflict levels in the address text and call a prediction model corresponding to the conflict levels after acquiring the address text to be processed; carrying out vectorization processing on each word of different address levels and the word level corresponding to each word in the address text to obtain a spliced vector formed by combining word vectors and word level vectors; determining address level data corresponding to the conflict level according to the splicing vector, and updating the address text according to the address level data; the prediction models corresponding to different address levels are used for carrying out error correction processing on the level data with conflicts in the address text to obtain the standard address, so that the problem of route misdistribution caused by the conflict of the address level information can be effectively reduced, and the cost of the whole logistics can be directly saved. Meanwhile, the generalization capability and accuracy of hierarchical structure recognition can be improved by adopting a deep learning algorithm, and text matching does not need to be carried out depending on an address library, so that a database does not need to be updated in real time, the maintenance cost of the database is reduced, and the maintenance cost of the database is reduced.

Drawings

FIG. 1 is a schematic diagram of a component architecture of an address error correction system according to an embodiment of the present application;

fig. 2 is a schematic diagram of a composition structure of a server according to an embodiment of the present application;

FIG. 3 is a schematic flow chart of an address error correction method according to an embodiment of the present disclosure;

FIG. 4 is a schematic diagram of an address hierarchy prediction model provided by an embodiment of the present application;

FIG. 5 is a schematic diagram of a network structure of a prediction model according to an embodiment of the present application;

fig. 6 is a logic block diagram of an address error correction device provided in an embodiment of the present application.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. In addition, technical features described below in the various embodiments of the present application may be combined with each other as long as they do not conflict with each other.

For ease of understanding, a system scenario to which the address error correction scheme provided herein is applicable is first described herein, and referring to fig. 1, a schematic diagram of a component architecture of an address error correction system of the present application is shown.

The system can comprise: the terminal 100 and the server 200 are connected to each other by a network. The server 200 obtains the address text to be processed, and the address text may be directly input into the server 200 or the terminal 100 may be sent to the server 200 through a network; the server 200 performs error correction processing on address hierarchy information with conflicts in the address text to obtain accurate address data; the server 200 issues the error-corrected address data to the terminal 100 through a network, and an courier holding the terminal 100 can acquire the address data in time through an application interface on the terminal 100 to execute subsequent dispatch operations. In an alternative embodiment, the server 200 further integrates a routing and sorting function, performs dot sorting based on big data after performing error correction processing on the address text, and issues the sorted address data to the terminal 100.

The terminal 100 may be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, and portable wearable devices, and the terminal 100 may collect a mailing address provided by a user and send it to the server 200 for processing; further, the terminal 100 may have a function of scanning and recognizing a handwriting address, and processing the handwriting address into address text information, and the like.

It should be noted that the above description uses a server as an independent server, but it is understood that in practical application, the server may be replaced by a server cluster or a distributed cluster formed by a plurality of servers.

In order to implement the corresponding functions on the server, a computer program for implementing the corresponding functions needs to be stored in a memory of the server. In order to facilitate understanding of the hardware configuration of each server, the following description will be given by taking the server as an example. As shown in fig. 2, a schematic structural diagram of a server according to the present application, the server 200 in this embodiment may include: a processor 201, a memory 202, a communication interface 203, an input unit 204, a display 205, and a communication bus 206.

The processor 201, the memory 202, the communication interface 203, the input unit 204, the display 205, and the communication bus 206 are all used to perform communication.

In this embodiment, the processor 201 may be a central processing unit (Central Processing Unit, CPU), an asic, a dsp, an off-the-shelf programmable gate array, or other programmable logic device.

The processor 201 may call a program stored in the memory 202. In particular, the processor 201 may perform operations performed on the server side in the following embodiments of the address error correction method.

The memory 202 is used to store one or more programs, and the programs may include program code that includes computer operation instructions, and in this embodiment, at least the programs for implementing the following functions are stored in the memory:

acquiring an address text to be processed;

determining a conflict level in the address text;

inputting the spliced vector into the prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data.

In one possible implementation, the memory 202 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, and at least one application program required for functions (such as text vectorization processing), and the like; the stored data area may store data created during use of the computer, such as word vectors, word level vectors, splice vectors, and predictive models and samples, among others.

In addition, memory 202 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device or other volatile solid-state storage device.

The communication interface 203 may be an interface of a communication module, such as an interface of a GSM module.

Of course, the structure of the server shown in fig. 2 does not limit the server in the embodiments of the present application, and the server may include more or fewer components than shown in fig. 2 or may combine some components in practical applications.

In combination with the above commonalities, referring to fig. 3, the present embodiment shows a flow chart of an address error correction method, and the method in the present embodiment includes the following steps:

step 301, obtaining an address text to be processed;

the address text to be processed is generally address data with a hierarchy conflict condition; the server analyzes the address text and judges whether the address text is used as the address text to be processed, namely, the server determines whether the address text contains at least two identical address levels, for example, if the address text does not contain at least two identical address levels, the server directly performs route sorting processing on the address text; if the address text contains at least two identical address levels, the server takes the address text as the address text to be processed.

For example, the address text to be processed is: the new street Yingfeng mansion 1305, shenzhen city, guangzhou; in the address text, the conflict between Shenzhen city and Guangzhou city appears in the city level, and the conflict between Dragon lake street and New safety street appears in the street level, so that the express delivery personnel can not quickly lock the target delivery area.

In a specific example, the server receives an address text processing request sent by the terminal, and parses the address text processing request to obtain an address text to be processed. Of course, the address data without the level conflict can be input into the server for processing, and when the server judges that the address data is not abnormal, the address data can be directly subjected to route sorting processing.

In a specific example, when the address text processing condition is satisfied, the server obtains the address text to be processed from a preset dispatch waybill address library. It should be noted that the pick-up and dispatch waybill address library may be a word library independent of the full-scale address library, which is dedicated to storing address text that requires community/village group predictions.

The address text processing condition is a condition or basis for triggering an address text processing operation, specifically, may be that a request instruction of address text processing sent by a terminal is received, or a preset duration is reached since the previous triggering of the address text processing operation, or a newly added address text to be processed appears in a dispatch bill address library is detected, which is not limited herein specifically. And the terminal generates a request instruction of address text processing according to the address text processing triggering operation of the user and sends the request instruction to the server. The preset duration may be customized, such as 1 hour.

Step 302, determining a conflict level in the address text, and calling a prediction model corresponding to the conflict level.

In some embodiments of the present application, determining a conflict level in the address text includes: the server preprocesses the address text, wherein the preprocessing comprises address text standardization, address word segmentation and/or word level filtering; word segmentation processing is carried out on the address text after pretreatment, and address level information of different address levels is obtained; and determining conflict levels in the address text according to the level relation among the address level information.

In some embodiments of the present application, server normalization processing includes, but is not limited to, cleaning invalid illegal characters, digital English normalization, address assignment normalization, duplication reduction, deduplication completion, bracket content processing, suffix processing, and the like. Specifically, the server acquires an initial address text to be processed according to the mode, and normalizes the initial address text to obtain a corresponding address text. In some embodiments of the present application, the server invokes a preset normalization system to normalize the initial address text, and obtains a corresponding address text.

The address word segmentation process mainly splits address text into different address levels. Specifically, the server acquires the address text after normalization processing, and performs word segmentation processing on the address text. For example, the normalized address text is: the address data obtained by word segmentation processing of the address text in the new street english peak building 1305 room of the new street in the dragon lake of Guangzhou, guangdong province is: A1|Shenzhen City, 2|Guangzhou, 2|Dragon lake street 5|New England street 5|English peak building 13|1305 Chamber 17.

In some embodiments of the present application, the server invokes a preset address word segmentation system to perform word segmentation on the normalized address text, so as to obtain corresponding address data. The address word segmentation system can adopt an open source word segmentation system such as jieba word segmentation, halimasch word segmentation and the like, and the scheme is not particularly limited.

Because the open source word segmentation systems such as jieba word segmentation and halibut word segmentation belong to word segmentation tools for solving general texts, in the field of address word segmentation, a person skilled in the art can train a special address word segmentation model according to the naming rule of an address, so that the word segmentation effect of the address is better, a large number of address texts are acquired in the embodiment, planning processing is carried out on the address texts to serve as training samples, the address word segmentation model is obtained through training of the training samples, and later, word segmentation processing is carried out on the address texts after standardization processing through the address word segmentation model obtained through training, and the address texts are segmented into: province, city, district, street, road number, campus, building, unit, house number, etc.

Word level filtering is mainly used for filtering out some word levels which have no relevance to conflict level prediction according to data relevance analysis. When predicting and correcting the conflict level of the street, the address level of the province is far higher than that of the street, and the method has no auxiliary effect on street prediction, so that the province level information is filtered out in the embodiment. Specifically, the server acquires address data of word segmentation processing, and filters the provincial level information in the address data to obtain a filtered address text.

For example, the address data after word segmentation is: the method comprises the steps of (1) in Guangdong province, (2) in Guangzhou city, (2) in dragon lake street, (5) in new installation street, (5) in English peak building, (13) in 1305 room, (17) filtering the address data to obtain a filtered address text: shenzhen city, 2|Guangzhou city, 2|Dragon lake street, 5|New street, 5|English peak building, 13|1305 room, 17.

After the server acquires the address text and performs preprocessing, firstly judging whether a conflict address hierarchy exists in the address text according to the context relation in the address text; if not, the address text can be directly subjected to route separation processing; if yes, determining a conflict level in the address text, and calling a prediction model corresponding to the conflict level.

In an alternative embodiment, the server determines that the address text conflict level may determine that there is a conflicting address level in the address text based on a hierarchical relationship between address levels, for example, a first-order city, a second-order street, a third-order community, a fourth-order dot pre-and a fifth-order AOI (AOI, english-full: area of interest, chinese-full-scale information plane, also referred to as an area-like geographic entity in map data), if two identical levels occur, or there is no dependency relationship between any two adjacent first-order cities, second-order streets, third-order communities, fourth-order dot pre-and fifth-order AOI in the address text; on the contrary, if the address text contains the first-level city, the second-level street, the third-level community, the fourth-level net point pre-and the fifth-level AOI respectively, and the first-level city, the second-level street, the third-level community, the fourth-level net point pre-and the fifth-level AOI have a subordinate relationship between any two adjacent two, the address text is judged to have no conflict address hierarchy.

In an alternative embodiment, the scheme sets five-level prediction models corresponding to five address levels respectively; referring to fig. 4, the five-level prediction model is a primary city prediction model, a secondary street prediction model, a tertiary community prediction model, a four-level net point prediction model and a five-level AOI prediction model, wherein AOI is fully known in english: the chinese language fully refers to the information plane, also called the interest plane. The map data comprises regional geographic entities in map data, wherein the regional geographic entities are used for predicting and correcting urban levels, street levels, community levels, network point levels and AOI levels of conflicts in address texts respectively; those skilled in the art will appreciate that the above five-level prediction model is only one specific example of the present solution, and does not constitute a limitation of the present solution; those skilled in the art may set more or fewer hierarchical prediction models depending on the actual needs.

In an alternative embodiment, when two or more conflict levels exist in the address text, corresponding prediction models are sequentially called from top to bottom according to the level sequence of the plurality of conflict levels in the address text to process, and because the addresses of the upper level and the addresses of the lower level have a cascade relation, accurate address information can be obtained only by determining the upper conflict level and then determining the lower conflict level, and meanwhile, because the data of the address of the upper level is relatively less, the processing efficiency of the data can be improved and the prediction result can be obtained quickly when the prediction models corresponding to the conflict levels are called from top to bottom.

For example: the order address is Shenzhen Guangzhou city, new street and English peak building 1305 room, and the conflict level is Shenzhen Guangzhou city and new street of Dragon lake street, and then the first-level city model is called to judge city information, and the second-level street model is called to judge street information.

Step 303, carrying out vectorization processing on each word of different address levels and the word level corresponding to each word in the address text to obtain a first spliced vector formed by combining a word vector and a word level vector;

The word vector refers to a vector corresponding to a single hierarchical word of different address levels, such as a word vector corresponding to Shenzhen city. Specifically, the server traverses each hierarchical word of different address levels in the filtered address text and generates a word vector corresponding to each hierarchical word traversed.

The server in this embodiment generates a word vector for each word of different address levels in the address text, in order to convert the character features of high dimensions into vectors of low dimensions, for example, implementation one: a dictionary is preset in the server, massive words and corresponding word vectors are recorded in the dictionary, and the server queries the dictionary to obtain the word vectors corresponding to the words; the implementation mode II is as follows: representing the words as a binary string of K dimensions, wherein the exclusive OR sum (the number of 0 or 1 in the same dimension) of the vectors of each word in the words, each word calculates the general hash of the K dimensions, such as MD5, each word becomes a binary string, the original position of 0 becomes-1, then the weight of the word is multiplied by each dimension, the vectors of the K dimensions of each word with the weight are added up, the dimension is 0 when the value of each dimension is negative, and the value is regular and 1; a K-dimensional 01 string is obtained as a word vector.

The word level vector refers to a word level vector corresponding to a hierarchical word of different address levels in the address text, wherein the word level is formed by pre-configuration, such as a word level vector of word level "2" corresponding to a hierarchical word "Shenzhen city". Specifically, the server traverses word levels corresponding to hierarchical words of different address levels in the filtered address text, and generates word level vectors of the word levels corresponding to each hierarchical word.

The first splicing vector is formed by combining and splicing word vectors and word-level vectors corresponding to the word vectors, the vector combination and splicing are used for fusing the corresponding features of the two vectors, and in the embodiment, the mode of splicing the first splicing vector is not limited, for example, the word vectors and the word-level vectors corresponding to the word vectors are all one-dimensional vectors, and the server splices the word vectors and the word-level vectors corresponding to the word vectors back and forth to form the first splicing vector; and then, for example, the server performs product operation on the word vector and the word level vector corresponding to the word vector to obtain a matrix, and takes the matrix as a first splicing vector.

For example, the server splices the word vector corresponding to Shenzhen and the word level "2" to obtain a spliced vector. Specifically, the server traverses word vectors corresponding to each hierarchical word of different address levels in the filtered address text and word level vectors thereof, and splices the word vectors to generate a first spliced vector corresponding to each hierarchical word.

In some embodiments of the present application, the server generates a concatenation vector corresponding to each hierarchical word in the address text through a trained vector model. And the server inputs the address text into the trained vector model to obtain the spliced vector corresponding to each grading word in the address text. The training step of the vector model comprises the following steps: obtaining a plurality of sample address texts, obtaining an address text corpus according to the plurality of sample address texts, training an initialized vector model by utilizing each grading word in the address text corpus, and obtaining a trained vector model when each grading word in the address text corpus is trained. In this embodiment, the machine learning algorithm adopted in the training process of the vector model is a network structure of Word2Vec, doc2Vec, CRNN and Text-CNN.

Since the word levels set by different word segmentation systems for hierarchical words of different address levels are likely to be different, the hierarchical manner of the 18-level address word segmentation system is taken as an example for illustration, and the filtered address text obtained after word level filtering is: the word vector generated for each hierarchical word in the filtered address text is in turn: v (Shenzhen City), V (Guangzhou City), V (Dragon lake street), V (Xin An street), V (Yingfeng mansion), V (chamber 1305), take "Shenzhen City" as an example, and "V (Shenzhen City)" represents the word vector corresponding to the hierarchical word "Shenzhen City".

The word-level vectors generated for the word level of each hierarchical word are in turn: v (2), V (2), V (5), V (5), V (13), V (17), taking "V (13)" as an example, the "V (13)" characterizes the word vector of the word class "13" corresponding to the hierarchical word "Yingfeng mansion".

The first splicing vector obtained by combining each word vector with the corresponding word level vector is as follows: v (Shenzhen city ζ2), V (Guangzhou city ζ2), V (Dragon lake street 5), V (Xin An street 5), V (Ying Feng mansion 13), V (1305 Chamber 17).

According to the scheme, word level vectorization is added on the basis of word vectorization, and a word vector and a first spliced vector of the word level vector are used as input parameters of a prediction model. The method has the advantages that: each hierarchical word after the address word segmentation has a hierarchical relationship, word level information representing the hierarchical relationship is substituted into word vectors, so that the model is more sensitive to the word segmentation position of the address, and address information with the same hierarchical word but different corresponding word levels in different address texts can be effectively distinguished.

In some embodiments of the present application, the text vectorization process further includes:

generating each word and the corresponding word level by each hierarchical word of different address levels in the address text, and carrying out vectorization processing to obtain a word vector and a word level vector; splicing the word vectors and the word-level vectors corresponding to the word vectors to obtain second spliced vectors; and combining the second splicing vector with the first splicing vector to serve as a target splicing vector of the input prediction model.

The word vector refers to a vector corresponding to a single word, specifically, the server traverses each word in each hierarchical word of different address hierarchy in the address text, and generates a word vector corresponding to each word traversed. For example: the word vector corresponding to each word in the hierarchical word Shenzhen city is: v (deep), V (zhen), V (city);

since the word level corresponding to each word in each hierarchical word is the same as the word level of the hierarchical word to which it belongs, the word level vectors corresponding to the word vectors of the plurality of words belonging to the same hierarchical word are also the same. Specifically, the server traverses word levels corresponding to each word in hierarchical words of different address levels in the address text, and generates a corresponding word level vector for each word in each hierarchical word. For example: the word class corresponding to the grading word "Shenzhen city" is "2", and the word vector V (deep) corresponding to each word in the grading word "Shenzhen city" is V (2), and the word vector V (zhen) corresponding to V (city) is V (2).

The server traverses and filters the word vector corresponding to each word and the word level vector of each word in the hierarchical words in the address text, and splices the word vectors to generate a spliced vector corresponding to each word. For example: the second splicing vector obtained by combining each word vector with the corresponding word level vector is: v (deep 2), V (zhen 2), V (city 2), V (Guangzhong 2), V (Zhou 2), V (Federal 2), V (Dragon 5), V (lake 5), V (street 5), V (New 5), V (amp. Sup.5), V (street. Sup.5), V (English. Sup.13), V (peak. Sup.13), V (large. Sup.13), V (mansion. Sup.13), V (1. Sup.17), V (3. Sup.17), V (0. Sup.17), V (5. Sup.17) and V (room. Sup.17).

In some embodiments of the present application, a server generates each word and its corresponding word level word by word for each word of different address levels in an address text, and performs vectorization processing to obtain a word vector, a word vector and a word level vector; the server combines the word vector, the word vector and the word level vector to serve as a spliced vector of the input prediction model. That is, the server splices the word vector, the word vector and the word level vector, so that the text features need to be deeply mined, the model is more sensitive to the word segmentation position of the address, and data of different categories can be effectively divided by the model.

In some embodiments of the present application, the server combines the second concatenation vector composed of the word vector and the word level vector with the first concatenation vector composed of the word vector and the word level vector to serve as a target concatenation vector of the input prediction model.

The above various splicing modes are all within the protection scope of the scheme.

In some embodiments of the present application, before inputting the target stitching vector into the prediction model, each stitching vector in the target stitching vector needs to be sequenced, so as to generate a corresponding target stitching vector sequence.

Taking a target splicing vector formed by a first splicing vector and a second splicing vector as an example, the target splicing vector sequence comprises a first splicing vector sequence and a second splicing vector sequence;

The first spliced vector sequence is a vector sequence formed by a plurality of word vectors according to the ordering of the words in the address text. The second concatenated vector sequence is a vector sequence consisting of a plurality of word vectors, the plurality of words being ordered in their associated hierarchical words.

For example, the target splice vector is: the target splice vector is: v (Shenzhen city), V (Guangzhou city), V (Dragon lake street 5), V (New safety street 5), V (British peak building 13), V (1305 room 17), V (SEP), V (deep 2), V (Zhen 2), V (city 2), V (Guangzhou 2), V (city 2), V (Dragon 5), V (lake 5), V (street 5), V (New 5), V (Ann 5), V (street 5), V (English 13), V (peak 13), V (large 13), V (1 17), V (3 17), V (0) and V (17), V (17) are the corresponding splicing vectors of the target sequences: { V (Shenzhen city ≡2), V (Guangzhou city ≡2), V (Dragon lake street ≡5), V (Xin lan street ≡5), V (Ying Feng mansion ≡13), V (1305), V (SEP), { V (Shen type 2), V (Guangzhou type 2), V (Zhou type 2), V (Guangzhou type 2), V (Longyi type 5), V (Hu type 5), V (street type 5), V (Xin type 5), V (Ying type 13), V (Feng type 13), V (Da type 13), V (mansion type 13), V (1) and V (17), V (17) and V (17).

And step 304, inputting the spliced vector into the prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data.

The server inputs the spliced vector into a prediction model corresponding to the conflict level, and the prediction model convolves the spliced vector to obtain candidate address level data corresponding to the conflict level and confidence corresponding to the candidate address level data; the prediction model comprises a primary city prediction model, a secondary street prediction model, a tertiary community prediction model, a quaternary website prediction model and a five-level interest surface prediction model; setting the candidate address level data with the highest confidence as the address level data corresponding to the conflict level, and updating the address text according to the address level data.

In some embodiments of the present application, updating the address text by the server according to the predicted address hierarchy data specifically includes the server adjusting address hierarchy data corresponding to the conflict hierarchy in the address text to the predicted address hierarchy data. In the embodiment, error correction of the address text is realized, and standard address information is obtained, so that routing can be accurately performed, and the cost of the whole logistics can be directly saved.

The prediction model is obtained by training sample address texts with address level labels corresponding to conflict levels, and each sample address text is processed into a spliced vector and then input into the prediction model with the corresponding address level label.

In the scheme, the prediction model is a model which is obtained by training based on a first training sample set obtained in advance and can be used for predicting address level data corresponding to a conflict level according to a target spliced vector sequence. The first training sample set comprises address level labels corresponding to conflict levels of a first sample stitching vector sequence corresponding to the first sample address text. The first sample spliced vector sequence is a vector sequence formed by combining vectors of each grading word and corresponding word level in the first sample address text, and the forming process of the first sample spliced vector sequence is the same as that of the target spliced vector sequence. The address hierarchy label corresponding to the conflict hierarchy is the address hierarchy data which is expected to be output.

Illustrating: the first sample address text is: the conflict level is the city level of "Shenzhen Guangzhou City", the address level label corresponding to the address text of the first sample is the "Shenzhen City" which is expected to be output. If the conflict level in the address text is other address levels, the address level label corresponding to the first sample address text is address level data expected to be output by other address levels, for example, the conflict level is a street level of ' new street in dragon lake street ', the address level label corresponding to the first sample address text is a street in dragon lake ' expected to be output, and the label setting methods of other conflict levels are analogized in sequence and are not repeated.

Specifically, in a model training stage, a server acquires first sample address texts, each first sample address text has a preconfigured first sample label, vectorization processing is carried out on each first sample address text to obtain a corresponding first sample spliced vector sequence, a first training sample set is obtained according to the first sample spliced vector sequences corresponding to the first sample address texts and the first sample labels, and model training is carried out according to the first training sample set to obtain a trained prediction model. In the model application stage, after a server generates a corresponding target splicing vector sequence aiming at an address text to be processed, the target splicing vector sequence is input into a trained prediction model, the address text is processed according to the target splicing vector sequence through the prediction model, and address level data corresponding to a conflict level is predicted and output. In this embodiment, a prediction model is trained based on a first training sample set, address texts with conflicting levels are processed through the trained prediction model, address level data corresponding to conflicting levels is output, and the address texts are updated according to the address level data, so that wrong address level data in the address texts can be quickly and accurately identified and corrected, and the problem of routing wrong classification caused by conflicting address level information is solved.

In some embodiments of the present application, the training process of the predictive model includes: acquiring a large number of first sample address texts, wherein each first sample address text is provided with an address level label corresponding to a conflict level; carrying out vectorization processing on each word of different address levels and the word level corresponding to each word in the first sample address text to obtain a sample splicing vector formed by combining a sample word vector and a sample word level vector; obtaining a first training sample set according to the sample splicing vector corresponding to the first sample address text and the address level label corresponding to the conflict level; and performing model training according to the first training sample set to obtain a trained prediction model.

In some embodiments of the present application, the server obtains full-size address data for at least half a year from a shipping bill address library, obtains full-size address data from a national address standard library, and obtains a plurality of first sample address texts according to the obtained full-size address data. Wherein, the full address data refers to all address data meeting the acquisition requirement.

In some embodiments of the present application, the sample concatenation vector further includes a sample word vector and a sample word level vector corresponding to the sample word vector; specifically: carrying out vectorization processing on each word generated by each word of different address levels in the first sample address text and the word level corresponding to each word to obtain a sample word vector and a sample word level vector; and splicing the sample word vector with the corresponding sample word level vector, and combining the sample word vector with the corresponding sample word level vector to serve as a sample splicing vector of the input prediction model.

Similarly, the sample splicing vector needs to be processed into a sample splicing vector sequence and then output a prediction model for model training.

In a specific example, performing model training according to the first training sample set to obtain a trained prediction model specifically includes:

generating address level prediction data corresponding to the conflict level according to the sample splicing vector sequence through a prediction model to be trained; calculating an error between address hierarchy prediction data corresponding to the conflict hierarchy and a corresponding address hierarchy label, specifically includes: the prediction model is correspondingly provided with a loss function, the loss function is a key element which is the most basic in machine learning, and the loss function is used for representing the difference degree between prediction and actual data; the effect of the loss function is used for measuring the quality of model prediction; the error between the predicted data and the address hierarchy label corresponding to the conflict hierarchy can be obtained by bringing the predicted data of the predicted model and the corresponding address hierarchy label into a loss function, and model parameters of the predicted model to be trained are reversely adjusted according to the error; address hierarchy labels corresponding to conflict hierarchies; returning to the step of generating address level prediction data corresponding to the conflict level according to the sample splicing vector sequence through the prediction model to be trained, and continuing to execute until an iteration stopping condition is met, and stopping iteration to obtain the trained prediction model, wherein the iteration stopping condition can be set according to specific situations, for example, the iteration stopping condition is that the iteration is performed for 1000 times, or the iteration stopping condition is that: the identification accuracy of the prediction model is higher than an accuracy threshold, the accuracy threshold can be set according to specific scenes, and the accuracy threshold is set to be 95 percent

Further, generating address level prediction data corresponding to the conflict level according to the sample splicing vector sequence specifically includes: extracting features of the sample spliced vector sequence to obtain corresponding maximum pooling feature vectors, average pooling feature vectors and weight feature vectors; the prediction model comprises a pooling layer, wherein the pooling layer can acquire a maximum pooling feature vector and an average pooling feature vector; the maximum pooling feature vector is used for recording the maximum feature and reducing interference information, so that the prediction capability is improved; the average pooling feature vector is used for recording the reserved local information, so that the prediction error is reduced; the weight feature vector is used for reasonably configuring the importance of different features, so that the model prediction comprehensively considers each factor, and address level prediction data corresponding to the conflict level is generated according to the maximum pooling feature vector, the average pooling feature vector and the weight feature vector. And reversely adjusting model parameters of the prediction model to be trained according to errors between the address level prediction data and the address level labels, and finishing single iteration training of the prediction model. Repeating the steps until the iteration stopping condition is met, and completing model training. The iteration stop condition, such as the number of iterations being greater than or equal to the iteration number threshold, and further, such as the loss function corresponding to a single iteration having been minimized, is not specifically limited herein.

FIG. 5 is a schematic diagram of a network structure of a prediction model provided in the present embodiment, and referring to FIG. 5, the prediction model includes a first neural network layer and a second neural network layer;

the first neural network layer is used as an input node of a prediction model and is mainly used for receiving a sample spliced vector sequence and carrying out recursive feature extraction and convolution feature extraction on the sample spliced vector sequence to generate a spliced and fused global feature vector;

the second neural network layer is mainly used for receiving the global feature vector generated by the first neural network layer, obtaining a maximum pooling feature vector sequence, an average pooling feature vector sequence and a weight feature vector sequence through maximum pooling, average pooling and attention weight distribution respectively, and performing splicing and fusion on the maximum pooling feature vector sequence, the average pooling feature vector sequence and the weight feature vector sequence to generate address level prediction data corresponding to a conflict level.

In a specific example, the first neural network layer includes an LSTM network layer and an IDCNN network layer, and the second neural network layer includes an average pooling layer, a maximum pooling layer, and an Attention network layer; LSTM is a long and short term memory network, which is used as a variant of a cyclic neural network and can better explain the relation between long sequence texts and sequences; IDCNN is a hollow convolutional neural network, mainly a variant of the convolutional neural network, and can extract relevant information in a key way; attention is Attention weight calculation, and important input points can be focused in the network iteration process. Because of the hierarchical relationship of the segmentation words in the address text, the LSTM and IDCNN network is better.

The main function of the Attention network layer is to calculate the influence degree of each word level information in the global input address vector on the finally predicted address level data, and different weights are given to different word levels according to the influence degree. The weight of the words with high correlation with the address level data of the expected output pair is added in the Attention network layer, for example, 13-level POI words, 6-level community/village groups and 11-level road number words are word levels with great influence on street level prediction, and the weight of the word levels is added, so that the accuracy rate of street prediction is improved.

In this embodiment, maximum pooling and average pooling are used simultaneously, and connection operation is performed on the respective output maximum pooling feature vector and average pooling feature vector, and because the address belongs to a short text, the pooling operation is beneficial to preserving more upper-layer feature information.

Further, a Mask layer is added in the Attention network layer, and the Mask layer is mainly used for filtering word levels which frequently occur but are not important for address level data prediction corresponding to conflict levels, for example, when the conflict level is a street, word levels such as a floor number, a building number and the like can be filtered.

Further, dropout layers are added in the average pooling layer and the maximum pooling layer respectively, and are used for randomly inactivating neurons, so that overfitting of the model can be prevented, and generalization capability of the model is improved.

In some embodiments of the present application, when training a prediction model according to the model training manner provided in one or more embodiments of the present application, training parameters of the model, such as a learning rate of a network, a model update rate, an empirical cache size, an action selection coefficient, a coefficient decay rate, and the like, are adaptively adjusted, which are not specifically limited herein.

In order to accelerate the model training time as much as possible and reduce the model size as much as possible, the embodiment performs self-optimization on the learning rate as the training parameter in the model training process; specifically, an initial learning rate is predefined, a loss function of the iteration is calculated in each iteration training, and if the loss function of a plurality of continuous iterations is unchanged, the initial learning rate is reduced according to a preset attenuation amplitude. In this embodiment, when a certain batch of iterative training is performed or it is monitored that the change of the loss function is not large, the learning rate is automatically reduced, so that the best convergence point is mainly found, because the gradient change is small when the loss function is about to converge, and if the learning rate is kept unchanged, the found convergence point has an error. For example: defining the initial learning rate as 0.1, and if no change or little change of the loss function of the continuous three-time iterative training is detected, attenuating the initial learning rate by 10%, and updating to 0.9.

It can be appreciated that, in an embodiment, the prediction model is deployed by using a labstack/echo framework of the Golang language, and the reason for selecting Golang is that Golang has an API for specifically calling the prediction model, so that the use is convenient, and the labstack/echo framework is very good for high concurrency multithreading optimization, so that the Web service performance after model deployment can be maximally realized.

In some embodiments of the present application, the address error correction method further includes: when the updating condition of the prediction model is met, a second training sample set is obtained; the second training sample set comprises a second sample splicing vector sequence and a sample label corresponding to a second sample address text; and carrying out iterative updating on the prediction model according to the second training sample set to obtain an updated prediction model, and taking the updated prediction model as a trained prediction model.

The model update condition is a condition or basis for triggering the model update operation, specifically, a model update instruction sent by the terminal is received, or a specified duration is reached from the last time of triggering the model update operation. The appointed duration is 6 months, and as new addresses are continuously appeared, the trained prediction model needs to be updated periodically according to a preset period, and the updated prediction model is used as the trained prediction model in the subsequent address text processing. In the above embodiment, the trained prediction model is iteratively updated and trained according to the model updating condition, so as to further improve the accuracy of model prediction, the generalization capability and the robustness of the new address.

In some embodiments of the present application, the following processing steps are further included before step S302:

and (3) comparing the address text processed in the step (S301) with addresses in a pre-configured white list address library, wherein the white list address library is mainly used for storing association relations between address hierarchy information of standard addresses, wherein the standard addresses can be address texts with model prediction errors and address hierarchy data corresponding to conflict hierarchies contained in the standard addresses, namely, storing the address texts with model past prediction errors or lower accuracy in the white list address library, and establishing mapping relations for address hierarchy data corresponding to the conflict hierarchies with accurate address text matching in a manual searching or dictionary matching mode. Specifically, after acquiring a new address text to be processed, the server firstly matches the address text with addresses in the white list address library, judges whether the address text to be processed belongs to an address where the model is likely to predict errors, and if so, identifies address level data corresponding to conflict levels according to the white list address library without inputting a prediction model to process, thereby avoiding wasting calculation resources of the model.

In some embodiments of the present application, for address texts that do not match successfully with addresses in the whitelist address library, the server processes the address text to be processed according to a pre-configured filtering rule, and aims to filter out invalid addresses; if the address text to be processed does not belong to the invalid address, the address text to be processed is input into a trained prediction model, and prediction and error correction of address level data corresponding to the conflict level are carried out.

In some embodiments of the present application, the server uses the tensorflow platform as a main framework for model training, that is, trains a prediction model based on the tensorflow platform, and stores the trained prediction model as a tensorflow platform savedmodel. When the trained predictive model is deployed online to provide Web services, the server deploys the savedmodel of the tensorflow platform using the labstack/echo framework of the Golang language. The Golang is selected because the Golang has an API (application program interface) specially calling a tensorf low platform, is convenient to use, and the labstack/echo framework is good in high-concurrency multithreading optimization, so that Web service performance after model deployment can be maximally realized.

In some embodiments of the present application, as shown in fig. 6, there is provided an address error correction apparatus 600, including: an acquisition module 601, a collision judgment module 602, a vector generation module 603, and a prediction module 604, wherein:

An obtaining module 601, configured to obtain an address text to be processed;

a conflict judging module 602, configured to determine a conflict level in the address text;

the vector generation module 603 is configured to perform vectorization processing on each word of different address levels and its corresponding word level in the address text, so as to obtain a first spliced vector formed by combining the word vector and the word level vector;

and the prediction module 604 is configured to input the splice vector into the prediction model corresponding to the conflict level, output address level data corresponding to the conflict level, and update the address text according to the address level data.

According to the method and the device, the prediction models corresponding to different address levels are used for carrying out error correction processing on the level data with conflicts in the address text, so that the problem of route misdistribution caused by conflict address level information can be effectively reduced, and the cost of the whole logistics can be directly saved. In addition, the deep learning algorithm is adopted to improve generalization capability and accuracy of hierarchical structure recognition, text matching does not need to be carried out depending on an address library, therefore, a database does not need to be updated in real time, the maintenance cost of the database is reduced, and the maintenance cost of the database is reduced.

In some embodiments of the present application, the conflict determination module 602 is further configured to: preprocessing the address text, wherein the preprocessing comprises address text normalization, address word segmentation and/or word level filtering; word segmentation processing is carried out on the address text after pretreatment, and address level information of different address levels is obtained; and determining conflict levels in the address text according to the level relation among the address level information.

In some embodiments of the present application, the vector generation module 603 is further configured to: generating each word and the corresponding word level by each hierarchical word of different address levels in the address text, and carrying out vectorization processing to obtain a word vector and a word level vector; splicing the word vectors and the word-level vectors corresponding to the word vectors to obtain second spliced vectors; and combining the second splicing vector with the first splicing vector to serve as a target splicing vector of the input prediction model.

In some embodiments of the present application, the prediction module 604 in the address error correction device 600 further includes: inputting the spliced vector into a prediction model corresponding to the conflict level, and carrying out convolution processing on the spliced vector through the prediction model to obtain candidate address level data corresponding to the conflict level and confidence corresponding to the candidate address level data; the prediction model comprises a primary city prediction model, a secondary street prediction model, a tertiary community prediction model, a quaternary website prediction model and a five-level interest surface prediction model; setting the candidate address level data with the highest confidence as the address level data corresponding to the conflict level, updating the address text according to the address level data,

In some embodiments of the present application, the address error correction device 600 further includes: a model training module;

the model training module is used for: acquiring a first sample address text, wherein the first sample address text is provided with an address level label corresponding to a conflict level; vectorizing the first sample address text to obtain a sample splicing vector; obtaining a first training sample set according to the sample splicing vector corresponding to the first sample address text and the address level label corresponding to the conflict level; and performing model training according to the first training sample set to obtain a trained prediction model.

In some embodiments of the present application, the model training module is further to: generating address level prediction data corresponding to the corresponding conflict level according to the sample splicing vector through a prediction model to be trained; calculating errors between the address level prediction data and the corresponding address level labels, and reversely adjusting model parameters of the prediction model to be trained according to the errors; and continuing to execute the step of generating address level prediction data corresponding to the corresponding conflict level according to the sample splicing vector through the prediction model to be trained until the iteration stopping condition is met, stopping iteration, and obtaining the trained prediction model.

In some embodiments of the present application, the model training module is further to: when the updating condition of the prediction model is met, a second training sample set is obtained; the second training sample set comprises a second sample splicing vector sequence and a sample label corresponding to a second sample address text; and carrying out iterative updating on the prediction model according to the second training sample set to obtain an updated prediction model, and taking the updated prediction model as a trained prediction model.

In some embodiments of the present application, the address error correction device 600 further includes a preprocessing module 605;

the preprocessing module 605 is configured to: and matching the address text provided by the acquisition module 601 with the address in the white list address library, judging whether the address text to be processed belongs to an address where the model is likely to predict errors, and if so, identifying address level data corresponding to the conflict level according to the white list address library. The white list address library is mainly used for storing address texts with model prediction errors and address level data corresponding to conflict levels thereof, matching the address texts with the address level data with the conflict levels being accurate in a manual searching or dictionary matching mode, and establishing a mapping relation.

The preprocessing module 605 is configured to: and matching the address text with a standard address in a pre-configured white list address library, wherein the white list address library is used for storing an association relation between address level information of the standard address, and if the matching is successful, determining address level information corresponding to a conflict level in the address text according to the association relation.

Further, the preprocessing module 605 is further configured to: processing the address text to be processed according to a preset filtering rule for the address text which is not successfully matched with the address in the white list address library, and filtering invalid addresses; if the address text to be processed does not belong to an invalid address, the preprocessing module 605 sends the address text to the vector generation module 603 for vectorization processing, the processed address data is input into a trained prediction model, and the prediction model performs conflict-level prediction and error correction.

For specific limitations of the address error correction device, reference may be made to the above limitations of the address error correction method, and no further description is given here. The respective modules in the above-described address error correction apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

It will be readily appreciated by those skilled in the art that the foregoing description is merely a preferred embodiment of the present application and is not intended to limit the present application, but any modifications, equivalents, improvements or the like which fall within the spirit and principles of the present application are intended to be included within the scope of the present application.

Claims

1. An address error correction method, comprising:

acquiring an address text to be processed;

2. The method for correcting an address according to claim 1, wherein the step of obtaining the address text to be processed further comprises:

and matching the address text with a standard address in a pre-configured white list address library, wherein the white list address library stores the association relation between address level information of the standard address, and if the matching is successful, the address level information corresponding to the conflict level in the address text is determined according to the association relation.

3. The address correction method of claim 1, wherein said determining a conflict level in said address text comprises:

4. The address correction method as claimed in any one of claims 1 to 3, wherein inputting the splice vector into the prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data, comprises:

5. The address error correction method of claim 4, wherein the training process of the predictive model comprises:

6. The method for address correction as claimed in claim 5, wherein said model training based on said first training sample set to obtain a trained predictive model comprises:

7. The method of address error correction as set forth in claim 6, wherein said generating address level prediction data corresponding to a collision level from said sample concatenation vector specifically includes:

8. An address error correction apparatus, comprising:

the acquisition module is used for acquiring the address text to be processed;

the conflict judging module is used for determining a conflict level in the address text and calling a prediction model corresponding to the conflict level;

the prediction module is used for inputting the spliced vector into a prediction model corresponding to the conflict level, outputting address level data corresponding to the conflict level, and updating the address text according to the address level data; the prediction model is obtained by training sample address texts with address level labels corresponding to conflict levels, and each sample address text is processed into a spliced vector and then input into the prediction model with the corresponding address level label.

9. A computer device comprising at least one processing unit, and at least one storage unit, wherein the storage unit stores a computer program which, when executed by the processing unit, causes the processing unit to perform the steps of the method of any of claims 1 to 7.

10. A computer readable medium, characterized in that it stores a computer program executable by a computer device, which computer program, when run on the computer device, causes the computer device to perform the steps of the method according to any one of claims 1-7.