TWI847497B - Store deduplication processing method, device, equipment and storage medium - Google Patents

Abstract

The present invention discloses a store deduplication processing method, device, equipment and storage medium, belonging to the field of data processing. The method includes: obtaining a first store name and first store location information of a target store; determining a target grid area where the target store is located according to the first store location information; obtaining a second store name and second store location information of stock stores located in the target grid area and the neighboring grid area from a pre-stored stock store database; obtaining a target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name, the first store location information, the second store name and the second store location information; and removing the target store as a duplicate store when the target similarity is greater than or equal to a preset deduplication similarity threshold. According to the embodiment of the present invention, the efficiency of store deduplication processing can be improved.

Description

Store deduplication processing method, device, equipment and storage medium

The present invention belongs to the field of data processing, and in particular relates to a store deduplication processing method, device, equipment and storage medium.

With the promotion of electronic payment technology, users can use electronic payment technology to pay in offline stores of merchants. In order to facilitate the processing of electronic payments in offline stores of merchants, it is necessary to manage the information of offline stores of merchants. However, when store data is uploaded by different sources, different sources may upload store data of the same store, and the store data of the same store uploaded by different sources may be different, resulting in the same store being misjudged as two different stores based on the store data, that is, the same store is counted repeatedly.

In order to prevent the same store from being counted repeatedly, personnel need to be dispatched to the store site for on-site inspections to manually determine whether the same store is counted repeatedly. However, manual inspections are very time-consuming and labor-intensive, and the efficiency of store deduplication processing is very low.

The embodiment of the present invention provides a store deduplication processing method, device, equipment and storage medium, which can improve the efficiency of store deduplication processing.

In the first aspect, the embodiment of the present invention provides a method for deduplicating stores, including: obtaining a first store name and first store location information of a target store; determining a target grid area where the target store is located according to the first store location information; obtaining a second store name and second store location information of stock stores located in the target grid area and the neighboring grid area from a pre-stored stock store database, wherein the neighboring grid area is adjacent to the target grid area; obtaining a target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name, the first store location information, the second store name and the second store location information; and removing the target store as a duplicate store when the target similarity is greater than or equal to a preset deduplication similarity threshold.

In a second aspect, an embodiment of the present invention provides a store deduplication processing device, comprising: a first acquisition module, used to obtain a first store name and first store location information of a target store; a grid area determination module, used to determine a target grid area where the target store is located according to the first store location information; a second acquisition module, used to obtain a second store name of the stock stores located in the target grid area and the neighboring grid area from a pre-stored stock store database and the location information of the second store, the neighboring grid area is adjacent to the target grid area; the calculation module is used to obtain the target similarity of the target store and the existing stores located in the target grid area and the neighboring grid area based on the name of the first store, the location information of the first store, the name of the second store and the location information of the second store; the deduplication module is used to remove the target store as a duplicate store when the target similarity is greater than or equal to the preset deduplication similarity threshold.

In the third aspect, the embodiment of the present invention provides a store deduplication processing device, the device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the store deduplication processing method of the first aspect is implemented.

In the fourth aspect, the embodiment of the present invention provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, the store deduplication processing method of the first aspect is implemented.

The embodiments of the present invention provide a method, device, equipment and storage medium for deduplication of stores, which can determine the grid area where the target store is located according to the store location information of the target store. The grid area is an area divided in a map. Based on the existing stores in the target grid area where the target store is located, the existing stores in the grid area surrounding the target grid area, and the store name and store location information of the target store in the database, the similarity between the target store and the existing stores is obtained, and based on the similarity, it is determined whether the newly acquired store is the same store as the existing store. If the newly acquired store is the same store as the existing store, the newly acquired store is considered to be a duplicate store and is removed. The deduplication process does not require manual participation, and the store locations can be used to narrow the scope of existing stores for comparison, improving the efficiency of store deduplication processing.

21: Existing stores

300,400: Store deduplication processing device

301: First acquisition module

302: Grid area determination module

303: Second acquisition module

304: Computing module

305: Deduplication module

401: Memory

402:Processor

403: Communication interface

404:Bus

A1,A2,A3,A4,A5,A6,A7,A8,A9: Grid area

S101,S102,S103,S104,S1041,S1042,S1043,S105,S106,S107,S108,S109,S110,S111,S112,S113,S114,S115: Steps

In order to more clearly explain the technical solution of the embodiment of the present invention, the following will briefly introduce the diagrams required for use in the embodiment of the present invention. For ordinary technicians in this field, other diagrams can be obtained based on these diagrams without creative labor.

FIG1 is a flow chart of a store deduplication processing method provided by an embodiment of the present invention; FIG2 is a schematic diagram of an example of a grid area in an embodiment of the present invention; FIG3 is a flow chart of a store deduplication processing method provided by another embodiment of the present invention; FIG4 is a schematic diagram of an example of a coding table in an embodiment of the present invention; FIG5 is a flow chart of a store deduplication processing method provided by another embodiment of the present invention; FIG6 is a structural schematic diagram of a store deduplication processing device provided by an embodiment of the present invention; FIG7 is a structural schematic diagram of a store deduplication processing device provided by an embodiment of the present invention.

The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in combination with drawings and specific embodiments. It should be understood that the specific embodiments described here are only intended to explain the present invention, not to limit the present invention. For those skilled in the art, the present invention can be implemented without some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by showing examples of the present invention.

With the promotion of electronic payment technology, users can use electronic payment technology to make payments in offline stores. In order to facilitate the processing of electronic payments in offline stores, it is necessary to manage the information of offline stores. However, when store data is uploaded by different sources, different sources may upload store data of the same store, and the store data of the same store uploaded by different sources may be different, resulting in the same store being misjudged as two different stores based on the store data, that is, the same store is repeatedly counted. In some cases, the situation where the same store is repeatedly counted may also be exploited, thereby causing loopholes in the database storing store information.

In order to prevent the same store from being counted repeatedly, personnel need to be dispatched to the store site for on-site inspections to manually determine whether the same store is counted repeatedly. However, manual inspections are very time-consuming and labor-intensive, and the efficiency of store deduplication processing is very low.

The present invention provides a store deduplication processing method, device, equipment and storage medium, which can determine the grid area where the store is located according to the store location information of the newly acquired store. The grid area is the area divided in the map. The data of the existing stores in the target grid area where the newly acquired store is located and the grid area around the target grid area, as well as the data of the newly acquired store, are used in the database to obtain the similarity between the newly acquired store and the existing store. Based on the similarity, it is judged whether the newly acquired store is the same store as the existing store. If the newly acquired store is the same store as the existing store, the newly acquired store is considered to be a duplicate store and is removed. The deduplication process does not require manual participation, and uses the store location to narrow the range of existing stores for comparison, thereby improving the efficiency of store deduplication processing.

The following is a description of the store deduplication processing method, device, equipment and storage medium provided by the present invention.

The first aspect of the present invention provides a store deduplication processing method, which can be applied to the scene of store deduplication based on store information collected from different sources, and can be executed by store deduplication devices, equipment, etc., which is not limited here. Figure 1 is a flow chart of the store deduplication processing method provided by an embodiment of the present invention. As shown in Figure 1, the store deduplication processing method may include steps S101 to S105.

In step S101, the first store name and first store location information of the target store are obtained.

The target store is the store to be determined whether it is a duplicate store, and can be the store corresponding to the newly acquired store information, such as a new store to be added to the existing store database. The first store name can be the store name of the target store. The first store location information can be the store location information of the target store. The store location information is used to characterize the location of the store, and can include the store address, store latitude and longitude, etc., which are not limited here.

In step S102, the target grid area where the target store is located is determined based on the first store location information.

For ease of processing, the map can be pre-divided into multiple grid areas. The sizes of different grid areas can be the same or different, which is not limited here. The shape of the grid area can be a regular shape such as a rectangle, or an irregular shape, which is not limited here. For example, the grid area can be a rectangular area with a length of 150 meters and a width of 150 meters.

The target grid area is the grid area where the target store is located. The first store location information can represent the location of the target store, and the grid area where the target store is located can be determined based on the first store location information, i.e. the target grid area.

In step S103, the second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area are obtained from the pre-stored stock store database.

The existing store database includes relevant data of existing stores. Existing stores are stores that have been determined to be non-duplicate stores. The relevant data of existing stores may include but is not limited to the store name, store location information, grid area, etc. of the existing stores.

In order to narrow the scope of the stock stores to be compared with the target store, a geographical area where the stock stores that are the same as the target store may be located can be first delineated according to the location. The geographical area is the surrounding area of the location of the target store. The target grid area and the neighboring grid area can be determined as the surrounding area of the location of the target store. The neighboring grid area is adjacent to the target grid area, that is, the neighboring grid area is the grid area adjacent to the target grid area.

For example, FIG2 is a schematic diagram of an example of a grid area in an embodiment of the present invention. FIG2 shows nine grid areas in dashed squares, namely grid areas A1 to A9. FIG2 also shows a plurality of existing stores 21. If grid area A5 is the target grid area, correspondingly, grid area A1, grid area A2, grid area A3, grid area A4, grid area A6, grid area A7, grid area A8 and grid area A9 are all neighboring grid areas of the target grid area. Taking the deduplication process for the target store located in the grid area A5 as an example, the store name and store location information of each stock store 21 in the grid area A1, the store name and store location information of each stock store 21 in the grid area A2, the store name and store location information of each stock store 21 in the grid area A3, the store name and store location information of each stock store 21 in the grid area A4, the store name and store location information of each stock store 21 in the grid area A6, the store name and store location information of each stock store 21 in the grid area A7, the store name and store location information of each stock store 21 in the grid area A8, and the store name and store location information of each stock store 21 in the grid area A9 can be obtained.

The number of existing stores in the existing store database is very large. If the target store is compared with all the existing stores in the existing store database one by one, it will take a long time to perform store deduplication processing. Since the target grid area and the neighboring grid area are the surrounding areas of the target store, the existing stores in the surrounding areas of the target store are more likely to be the same store as the target store. The relevant data of the existing stores in the target grid area and the neighboring grid area in the existing store database can be first screened out, and the relevant data of the existing stores in the target grid area and the neighboring grid area and the relevant data of the target store can be used to compare the existing stores with the target store, so as to shorten the time required for store deduplication processing and improve the efficiency of store deduplication processing.

The existing stores located in the target grid area and the neighboring grid area include the existing stores located in the target grid area and the existing stores located in the neighboring grid area. The second store name includes the store name of the existing store located in the target grid area and the store name of the existing store located in the neighboring grid area. The second store location information includes the store location information of the existing stores located in the target grid area and the store location information of the existing stores located in the neighboring grid area.

In step S104, based on the first store name, the first store location information, the second store name and the second store location information, the target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area is obtained.

Based on the first store name and the second store name, the similarity between the target store and the existing stores in terms of store name can be obtained. Based on the first store location information and the second store location information, the similarity between the target store and the existing stores in terms of geographical location can be obtained. According to the similarity between the target store and the existing stores in terms of store name and geographical location, the target similarity can be obtained. The target similarity is the similarity between the target store and the existing stores. The similarity between the target store and each existing store located in the target grid area and the neighboring grid area can be calculated, and according to the target similarity, it is determined whether the target store is a duplicate store that is the same as the existing stores located in the target grid area and the neighboring grid area.

In step S105, when the target similarity is greater than or equal to the preset deduplication similarity threshold, the target store is removed as a duplicate store.

The deduplication similarity threshold is a threshold for confirming that the target store and the existing store are the same store. It can be set according to the scenario, demand, experience, etc., and is not limited here. For example, the deduplication similarity threshold can be 0.6. If the target similarity is greater than or equal to the deduplication similarity threshold, it means that the target store and the existing store are the same store, that is, the target store is a duplicate store, and the target store can be removed. Removing the target store can mean discarding the relevant data of the target store. If the target similarity is less than the deduplication similarity threshold, it means that the target store and the existing store are different stores, that is, the target store is not a duplicate store, and the relevant data of the target store can be stored in the existing store database, that is, the target store can be regarded as an existing store newly added to the existing store database.

In the embodiment of the present invention, the grid area where the target store is located can be determined based on the store location information of the target store. The grid area is the area divided in the map. Based on the existing stores in the target grid area where the target store is located, the existing stores in the grid area around the target grid area, and the store name and store location information of the target store in the database, the similarity between the target store and the existing stores is obtained, and the newly acquired store is judged whether it is the same store as the existing store based on the similarity. If the newly acquired store is the same store as the existing store, the newly acquired store is considered to be a duplicate store and is removed. The deduplication process does not require manual participation, and the location of the store can be used to narrow the range of the existing stores used for comparison, thereby improving the efficiency of store deduplication processing.

Moreover, in addition to comparing the target store with the existing stores in the target grid area, the target store is also compared with the existing stores in the neighboring grid area to avoid missing the existing stores that are located near the boundary of the target grid area and are the same store as the target store, further improving the comprehensiveness and accuracy of store deduplication processing.

In some embodiments, the grid area has a grid code, and the neighboring grid area of the target grid area can be determined based on the grid code of the target grid area and the grid coding algorithm. FIG. 3 is a flow chart of a store deduplication processing method provided by another embodiment of the present invention. FIG. 3 is different from FIG. 1 in that the store deduplication processing method shown in FIG. 3 may further include steps S106 to S108, and the store deduplication processing method shown in FIG. 3 may further include steps S109 to S112, or steps S113 to S115.

In step S106, the map is divided into a plurality of grid regions, and a grid code is assigned to each grid region using a grid coding algorithm.

A geographic map can be obtained and divided into a plurality of grid regions. A grid code is assigned to each grid region, and the grid code can characterize the grid region, that is, different grid regions have different grid codes. The grid code can be obtained according to a grid coding algorithm, and the type of the grid coding algorithm is not limited here. The grid codes calculated according to the position information of different locations in the same grid region are the same.

In some examples, the grid code may be an m-bit string, and the characters of the first m1 bits in the grid code may represent a province, city, district, etc. The characters of the first m1 bits of multiple adjacent grid areas are consistent, and the characters of the last m-m1 bits are different. The characters of the last m-m1 bits of the grid codes of different grid areas may be selected according to a preset coding table. The coding table includes multiple coding characters arranged in a certain order. According to the correspondence between the arrangement order of the coding characters and the grid area, the corresponding coding characters may be selected as the characters of the last m-m1 bits of the grid code. Each bit in the last m-m1 bits of the grid code may correspond to a coding table, and the coding tables corresponding to different bits may be the same or different. According to the grid codes of multiple grid areas, it can be determined whether the multiple grid areas are adjacent. Furthermore, according to the grid codes of multiple grid areas, the orientation relationship between the grid areas can be determined.

For example, FIG4 is a schematic diagram of an example of a coding table in an embodiment of the present invention. The grid area is shown in FIG2, and the grid code is a 7-bit string. If the characters of the first 6 bits in the grid codes of the adjacent grid areas are the same, both are wk2vu1, and the characters of the last bit are encoded according to the coding table shown in FIG4, the grid code of the grid area A1 is wk2vu1E, the grid code of the grid area A2 is wk2vu1R, and the grid code of the grid area A3 is wk2vu1E. The grid code of grid area A4 is wk2vu1D, the grid code of grid area A5 is wk2vu1F, the grid code of grid area A6 is wk2vu1G, the grid code of grid area A7 is wk2vu1C, the grid code of grid area A8 is wk2vu1V, and the grid code of grid area A9 is wk2vu1B.

In step S107, the store location information of the existing stores is obtained, and the grid area where the existing stores are located is determined based on the store location information of the existing stores.

In step S108, a first correspondence between the existing stores and the grid codes of the grid areas where the existing stores are located is established, and the first correspondence is stored in the existing store database.

The first correspondence includes the correspondence between the existing stores and the grid codes of the grid areas where the existing stores are located. In order to further shorten the time required for store deduplication processing, the data of existing stores can be processed in advance, and the grid codes of the grid areas where the existing stores are located can be mapped to the existing stores, and the mapping relationship can be stored in the existing store database, so that in the process of store deduplication processing, the existing stores corresponding to the grid codes of the target grid area and the existing stores corresponding to the grid codes of the neighboring grid areas can be directly found in the existing store database. The existing stores corresponding to the grid codes of the target grid area are the existing stores located in the target grid area, and the existing stores corresponding to the grid codes of the neighboring grid areas are the existing stores located in the neighboring grid areas.

In step S109, the grid code of the target grid area is obtained.

After determining the target grid area, the grid code of the target grid area can be obtained.

In step S110, the position of the vertex of the target grid area is obtained according to the grid coding of the target grid area and the grid coding inverse algorithm.

The grid coding inverse algorithm is the inverse algorithm of the grid coding algorithm. Based on the location information of one or more locations in the grid area, the grid coding of the grid area can be obtained by using the grid coding algorithm. Based on the grid coding of the grid area, the location information of the vertices of the grid area can be obtained by using the grid coding inverse algorithm.

In step S111, the location information of the auxiliary point in the neighboring grid area is determined based on the location information of the vertex of the target grid area.

The neighboring grid area shares some vertices with the target grid area, and obtaining the location information of the vertices of the target grid area is equivalent to obtaining the location information of some vertices of the neighboring grid area. According to the location information of some vertices of the neighboring grid area, the location information of the auxiliary points in the neighboring grid area can be obtained. The auxiliary points can be any one or more points in the neighboring grid area except the vertices shared with the target grid area, and are not limited here. The auxiliary points can be determined in each neighboring grid area, so that the location information of the auxiliary points can be used to determine the neighboring grid area later.

In step S112, based on the location information of the auxiliary points in each neighboring grid area and the grid coding algorithm, the grid code of each neighboring grid area is calculated to determine the neighboring grid area.

The grid code and the grid area have a corresponding relationship. According to the location information of the auxiliary point in the neighboring grid area, the grid code calculated by the grid coding algorithm is the grid code of the neighboring grid area. The neighboring grid area can be determined by using the corresponding relationship between the grid code and the grid area.

In step S113, the grid code of the target grid area is obtained.

In step S114, the grid code of the candidate grid area is obtained according to the grid code of the target grid area.

In some examples, the characters of a portion of the digits in the grid codes of the adjacent grid regions are the same, and this feature can be used to screen out the grid regions adjacent to the target grid region from a large number of grid regions, namely the candidate grid regions. The candidate grid regions include grid regions where the characters of a portion of the digits in the grid codes are the same as the characters of a portion of the digits in the grid codes of the target grid regions. For example, the characters of the first m1 digits in the grid codes of the adjacent grid regions are the same, and the grid regions where the characters of the first m1 digits in the grid codes are the same as the characters of the first m1 digits in the grid codes of the target grid regions can be determined as candidate grid regions.

In step S115, according to the correspondence between the grid area arrangement and the characters of the coded digits in the grid coding algorithm, the grid code of the neighboring grid area is determined in the grid code of the candidate grid area to determine the neighboring grid area.

The grid coding algorithm may include a correspondence between the grid area arrangement and the characters of the coded digits. For example, the grid area arrangement is shown in FIG2 , the grid code is a 7-bit string, the first 6 bits of the grid code of the candidate grid area are the same as the first 6 bits of the grid code of the target grid area, the target grid area is grid area A5, and its grid code is wk2vu1D. The correspondence between the grid area arrangement and the last bit of the grid code in the grid coding algorithm is specifically implemented as a coding table as shown in FIG4 . It can be seen that the target grid area has 8 neighboring grid areas, and the 8 neighboring grid areas are located at the upper left, upper, upper right, left, right, lower left, lower, and lower right of the target grid area, respectively. According to the coding table shown in FIG4 , the position The characters at the upper left, upper right, left, right, lower left, lower right, and lower right of the character D are W, E, R, S, F, X, C, and V. Correspondingly, the grid codes of the eight neighboring grid areas located at the upper left, upper right, left, right, lower left, lower, and lower right of the target grid area, namely, grid area A1, grid area A2, grid area A3, grid area A4, grid area A6, grid area A7, grid area A8, and grid area A9, are wk2vu1W, wk2vu1E, wk2vu1R, wk2vu1S, wk2vu1F, wk2vu1X, wk2vu1C, and wk2vu1V, respectively.

The grid code represents the grid area. By determining the grid code of the neighboring grid area, the neighboring grid area can be determined.

The method of using the correspondence between the grid area arrangement and the characters of the coded digits to determine the neighboring grid area is simpler, less time-consuming and more efficient.

In some embodiments, the target similarity can be obtained based on the similarity related to the store name and the similarity related to the store location information. FIG5 is a flow chart of a store deduplication processing method provided by another embodiment of the present invention. FIG5 is different from FIG1 in that step S104 in FIG1 can be specifically refined into steps S1041 to S1043 in FIG5.

In step S1041, based on the first store name and the second store name, the similarities between the target store and the N existing stores located in the target grid area and the neighboring grid area are obtained.

N is an integer greater than or equal to 1. Name-related similarity is the similarity related to the store name, which can be obtained based on the first store name and the second store name. Name-related similarity may include but is not limited to any one or more of character similarity, semantic similarity, and store type similarity. Character similarity is the similarity of the characters constituting the store name. Semantic similarity is the similarity of the semantics of the store name. Store type similarity is the similarity of the store type obtained based on the store name.

In some examples, the name-related similarity includes character similarity. The first store name and the second store name may be segmented respectively to obtain the vocabulary corresponding to the first store name and the vocabulary corresponding to the second store name; the term frequency (TF) and inverse document frequency (IDF) of the vocabulary corresponding to the first store name and the vocabulary corresponding to the second store name are calculated; the vocabulary whose term frequency is less than or equal to the redundant term frequency threshold and whose inverse document frequency is greater than the redundant frequency index threshold is selected; based on the selected vocabulary corresponding to the first store name and the selected vocabulary corresponding to the second store name, the character similarity of the target store and the stock stores located in the target grid area and the neighboring grid area is obtained.

The word segmentation tool can be used to segment the first store name to obtain the vocabulary corresponding to the first store name; the word segmentation tool can be used to segment the second store name to obtain the vocabulary corresponding to the second store name. The word frequency represents the frequency of the occurrence of a word. The reverse document frequency is used to represent the distinguishing ability of a word. The redundant word frequency threshold is the threshold of the word frequency used to distinguish whether a word is a redundant word. The redundant frequency index threshold is the threshold of the reverse document frequency used to distinguish whether a word is a redundant word. If the frequency of a word is greater than the redundant word frequency threshold, it means that the word is a redundant word; if the reverse document frequency of a word is less than or equal to the redundant frequency index threshold, it means that the word is a redundant word. Redundant words are not helpful for the calculation of character similarity, and may even have adverse effects, so they do not need to participate in the calculation of character similarity. Words whose frequency is less than or equal to the redundant word frequency threshold and whose reverse document frequency is greater than the redundant frequency index threshold are valid words participating in the character similarity calculation. The character similarity calculation can refer to the Bilingual Evaluation Understudy (BLEU) algorithm used in machine translation. The character similarity between the first store name and the second store name is evaluated by the N-gram overlap between the words corresponding to the first store name and the words corresponding to the second store name.

In some examples, the name-related similarity includes semantic similarity. The first store name and the second store name are converted into a first name digital sequence and a second name digital sequence respectively; the first name digital sequence and the second name digital sequence are input into the first model to obtain the semantic similarity between the target store output by the first model and the stock stores located in the target grid area and the neighboring grid area.

The first model is used to output the semantic similarity of the two store names according to the digital sequences converted from the two store names. A certain number of annotated store names can be obtained in advance as positive samples of the training set, and a corresponding number of store names can be randomly selected as negative samples of the training set. The positive samples of the training set and the negative samples of the training set are converted into digital sequences respectively, and the first model is obtained by training with the digital sequences. The first model may include a classification model, which may be a deep learning classification model or other types of classification models, which are not limited here. For example, the Bidirectional Encoder Representations from Transformer (BERT) model can be used to take "[CLS] + a digital sequence corresponding to a store name + [SEP] + a digital sequence corresponding to another store name" as input to train the first model so that the first model can fit the semantic similarity between one store name and another store name, that is, the first model can output the semantic similarity between one store name and another store name based on the input.

The first name digital sequence is the digital sequence converted from the first store name. The second name digital sequence is the digital sequence converted from the second store name. Specifically, the store name can be segmented by characters, the segmented characters can be converted into numbers, and the numbers corresponding to each character can be combined to obtain a digital sequence. The first name digital sequence and the second name digital sequence corresponding to an existing store located in the target grid area and the neighboring grid area are input into the first model, and the first model can output the semantic similarity between the store name of the target store and the store name of this existing store.

In some examples, the name-related similarity includes store type similarity. In the store deduplication process, there may be a possibility of false deduplication due to the close proximity of chain stores and the similar names of different stores. In order to reduce or even avoid the possibility of false deduplication, store type similarity can be introduced to improve the accuracy of store deduplication. According to the first store name, the first store name information can be obtained; the first store name information is input into the second model to obtain the store type probability vector of the target store output by the second model; the store type probability vector corresponding to the second store name is searched in the stock store database; the similarity between the store type probability vector of the target store and the store type probability vector corresponding to the second store name is calculated, and the similarity is determined as the store type similarity between the target store and the stock stores located in the target grid area and the neighboring grid area.

The second model is used to output a store type probability vector based on the input store name information. The store type probability vector is used to represent the probability that the store indicated by the store name belongs to each store type. Each element in the store type probability vector can represent the probability that the store belongs to a store type, and the store type corresponding to the element with the maximum probability represented in the store type probability vector can be determined as the store type of the store. The store type probability vector can be a normalized vector with a length of M, but is not limited to this. A certain number of annotated store names and store types can be obtained in advance as a training set, such as <XXXX1 (B1 regional store), supermarket>, <YYYY2 (B2 regional store), cafe>, where XXXX1 (B1 regional store) and YYYY2 (B2 regional store) are store names, and supermarket and cafe are store types. The second model is trained using the training set. The second model may include a classification model, which may be a deep learning classification model or other types of classification models, which are not limited here. For example, the BERT model can be used to take "[CLS] + a digital sequence corresponding to a store name" as input to train the second model so that the second model can fit the correspondence between the store name and the store type, that is, the second model can output the store type probability vector of the store name based on the input.

The first store name information is obtained based on the first store name, which may be the first store name or information of the first store name after processing, such as a digital sequence. The method of converting the store name into a digital sequence can be referred to the relevant description in the above embodiment, which will not be repeated here. The store type probability vector corresponding to the second store name includes the store type probability vector corresponding to the stock stores located in the target grid area and the neighboring grid area. In some examples, the similarity between the store type probability vector of the target store and the store type probability vector corresponding to the second store name may be the cosine similarity of the two store type probability vectors.

In order to further shorten the time required for store deduplication processing, the store type probability vector of the existing stores can be obtained in advance based on the store names of each existing store, so that when the store type similarity needs to be calculated, it can be directly obtained from the existing store database. Specifically, the store name of the existing store can be obtained, and the store name information can be obtained based on the store name; the store name information of the existing store is input into the second model to obtain the store type probability vector of the existing store output by the second model; a second correspondence relationship between the existing store and the store type probability vector of the existing store is established, and the second correspondence relationship is stored in the existing store database. When calculating the store type similarity, the store type probability vector corresponding to the second store name can be found in the existing store database based on the second correspondence relationship.

In step S1042, based on the first store location information and the second store location information, the location similarity between the target store and the existing stores located in the target grid area and the neighboring grid area is obtained.

The location similarity is the similarity related to the store location information, which can be obtained based on the first store location information and the second store location information. The location similarity can be determined based on the distance between the two store locations indicated by the two store location information and the deviation amount that may be caused by the location information. Specifically, the geographical distance between the target store and the existing store can be obtained based on the first store location information and the second store location information; and the location similarity between the target store and the existing stores located in the target grid area and the neighboring grid area can be obtained based on the ratio of the geographical distance and the location deviation threshold. The first store location information and the second store location information can be positioning coordinate information, such as Global Positioning System (GPS) coordinate information. If the first store location information and the second store location information are address information, the address information can be converted into coordinate information, such as latitude and longitude information, and then the geographical distance between the target store and the existing store can be determined based on the coordinate information. The location deviation threshold can be the maximum value of the deviation that may be caused by the location information. The ratio of the geographical distance and the location deviation threshold can be used for normalization to obtain the location similarity. For example, the location similarity can be obtained according to the following formula (1):

In step S1043, the target similarity is calculated based on the N name-related similarities, position similarities and corresponding weight coefficients.

The weight coefficient can be used as an index or a product coefficient to participate in the calculation of target similarity, which is not limited here. In some examples, the weight coefficient can be used as an index to participate in the calculation of target similarity. For example, the name-related similarity includes character similarity, semantic similarity, and store type similarity. The target similarity can be obtained according to the following formula (2): sim (target store , existing store) = sim (character) ^α × sim (semantic) ^β × sim (type) ^γ × sim (position) ^δ (2)

Among them, sim (target store , existing store) is the target similarity; sim (character) is the character similarity; sim (semantic) is the semantic similarity; sim (type) is the store type similarity; sim (location) is the location similarity; α is the weight coefficient of character similarity; β is the weight coefficient of semantic similarity; γ is the weight coefficient of store type similarity; δ is the weight coefficient of location similarity. In some examples, for the convenience of calculation, α=β=γ= δ =1 can be used.

For easier understanding, the following example is used to illustrate the store deduplication processing method. In this example, the name-related similarity includes character similarity, semantic similarity, and store type similarity.

Get the store name and store address of the target store, convert the store address into longitude and latitude coordinates, and the converted longitude and latitude coordinates are {30.193, 120.173}. Using the grid coding algorithm, calculate the grid code of the target grid area where the target store is located, that is, wtm7y8e. The first 6 bits of the grid code of the neighboring grid area are the same as the first 6 bits of the grid code of the target grid area. The grid codes of the 8 neighboring grid areas can be obtained using the coding table shown in Figure 4. The grid codes of the 8 neighboring grid areas are wtm7y82, wtm7y83, wtm7y84, wtm7y8W, wtm7y8R, wtm7y8S, wtm7y8D and wtm7y8F. By searching the existing store database, it is determined that there are 158 existing stores in the target grid area, 0 existing stores in the neighboring grid area with grid code wtm7y82, 4 existing stores in the neighboring grid area with grid code wtm7y83, 1 existing store in the neighboring grid area with grid code wtm7y84, and 1 existing store in the neighboring grid area with grid code wtm7y8W. There are 0 existing stores in the neighboring grid area, 18 existing stores in the neighboring grid area with grid code wtm7y8R, 1 existing store in the neighboring grid area with grid code wtm7y8S, 0 existing stores in the neighboring grid area with grid code wtm7y8D, and 0 existing stores in the neighboring grid area with grid code wtm7y8F. That is, there are 181 existing stores in the target grid area and the neighboring grid area. The target similarity between the target store and each existing store in the target grid area and the neighboring grid area needs to be calculated.

The following example uses the calculation of the target similarity between the target store and one of the existing stores as an example. The name of the target store is "X1X2 (Hangzhou Binjiang Baolong City Plaza Store)", and the name of the existing store is "Hangzhou Binjiang District X3X4 Convenience Store", where X1, X2, X3 and X4 are all Chinese characters, and they are different Chinese characters.

0.32。 You can use the word segmentation tool to segment the store names of target stores and existing stores to obtain the words corresponding to the target stores and the words corresponding to the existing stores. The words corresponding to the target stores include `X1X2`, `(`, `Hangzhou City`, `Binjiang`, `Baolong`, `City`, `Plaza`, `Store`, and `)`. The words corresponding to the existing stores include `Hangzhou City`, `Binjiang District`, `X3X4`, and `Convenience Store`. The word frequency and reverse document frequency of each word are calculated. The word frequency and reverse document frequency of the above words `(`, `Hangzhou City` and `)` do not meet the conditions that the word frequency is less than or equal to the redundant word frequency threshold and the reverse document frequency is greater than the redundant frequency index threshold. Therefore, the words `(`, `Hangzhou City` and `)` are discarded. After discarding the words `(`, `Hangzhou City` and `)`, the selected word combination corresponding to the target store is "X1X2 Binjiang Baolong City Plaza Store", and the selected word combination corresponding to the existing store is "Binjiang District X3X4 Convenience Store". Using the above BLEU algorithm to calculate character similarity, "X1X2 Binjiang Baolong City Plaza Store" contains 11 1-grams, and "Binjiang District X3X4 Convenience Store" contains 8 1-grams. The co-occurrence counts of the 1-grams of the two are calculated separately. It can be seen that the three 1-grams of "Bin", "Jiang" and "Store" each co-occur once. Therefore, the character similarity of "X1X2 Binjiang Baolong City Plaza Store" and "Binjiang District X3X4 Convenience Store" is (3/11+3/8)/2

0.32.

"X1X2 (Hangzhou Binjiang Baolong City Plaza Store)" can be converted into a digital sequence [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15], and "Hangzhou Binjiang District X3X4 Convenience Store" can be converted into a digital sequence [3,4,5,6,7,16,17,18,19,20,14]. The same Chinese characters correspond to the same digits. The above two digital sequences are concatenated with [CLS] and [SEP] to form a single vector, and then input into the first model to obtain the semantic similarity of the two output by the first model.

The two digital sequences converted from "X1X2 (Baolong City Plaza Store, Binjiang, Hangzhou)" and "X3X4 Convenience Store, Binjiang District, Hangzhou" can be input into the second model to obtain the store type probability vector of the target store and the store type probability vector of the existing store. The values of the elements of the target store and the existing store in the three store type dimensions of "shopping", "supermarket" and "convenience store" are relatively high, and the store types represented by the store type similarity based on the store type probability vector of the target store and the store type probability vector of the existing store are relatively close.

Based on the store location information of the target store and the store location information of the existing store, the geographical distance between the two is determined to be 285 meters. Based on the geographical distance and the location deviation threshold, the location similarity can be calculated to be 0.8585.

Assuming the deduplication similarity threshold is 0.6, for the target store and the existing store, the target similarity calculated using the above formula (2) is less than 0.6, which means that the target store and the existing store are not the same store.

It should be noted that the acquisition, storage, use, and processing of information and data in the embodiments of the present invention are authorized by users or relevant institutions and comply with relevant provisions of national laws and regulations.

The second aspect of the present invention provides a store deduplication processing device. FIG6 is a schematic diagram of the structure of the store deduplication processing device provided by an embodiment of the present invention. As shown in FIG6, the store deduplication processing device 300 may include a first acquisition module 301, a grid area determination module 302, a second acquisition module 303, a calculation module 304 and a deduplication module 305.

The first acquisition module 301 can be used to obtain the first store name and first store location information of the target store.

The grid area determination module 302 can be used to determine the target grid area where the target store is located based on the first store location information.

The second acquisition module 303 can be used to obtain the second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area from the pre-stored stock store database.

The neighbor grid area is adjacent to the target grid area.

The calculation module 304 can be used to obtain the target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name, the first store location information, the second store name and the second store location information.

The deduplication module 305 can be used to remove the target store as a duplicate store when the target similarity is greater than or equal to the preset deduplication similarity threshold.

In the embodiment of the present invention, the grid area where the target store is located can be determined based on the store location information of the target store. The grid area is the area divided in the map. Based on the existing stores in the target grid area where the target store is located, the existing stores in the grid area around the target grid area, and the store name and store location information of the target store in the database, the similarity between the target store and the existing stores is obtained, and the newly acquired store is judged based on the similarity whether it is the same store as the existing store. If the newly acquired store is the same store as the existing store, the newly acquired store is considered to be a duplicate store and is removed. The deduplication process does not require manual participation, and the location of the store can be used to reduce the range of the existing stores used for comparison, thereby improving the efficiency of store deduplication processing.

Moreover, in addition to comparing the target store with the existing stores in the target grid area, the target store is also compared with the existing stores in the neighboring grid area to avoid missing the existing stores that are located near the boundary of the target grid area and are the same store as the target store, further improving the comprehensiveness and accuracy of store deduplication processing.

In some embodiments, the grid area has a grid code. The store deduplication processing device 200 may also include a neighbor grid area determination module.

In some examples, the neighbor grid area determination module can be used to: obtain the grid code of the target grid area; obtain the location information of the vertices of the target grid area according to the grid code of the target grid area and the grid code inverse algorithm; determine the location information of the auxiliary points located in the neighbor grid area according to the location information of the vertices of the target grid area; calculate the grid code of each neighbor grid area based on the location information of the auxiliary points in each neighbor grid area and the grid coding algorithm to determine the neighbor grid area.

In some examples, the values of a portion of digits in the grid codes of adjacent grid regions are the same. The neighboring grid region determination module can be used to: obtain the grid code of the target grid region; obtain the grid code of the candidate grid region according to the grid code of the target grid region, the candidate grid region includes a grid region in which the characters of a portion of digits in the grid code are the same as the characters of a portion of digits in the grid code of the target grid region; determine the grid code of the neighboring grid region in the grid code of the candidate grid region according to the correspondence between the grid region arrangement in the grid coding algorithm and the characters of the coded digits, so as to determine the neighboring grid region.

In some embodiments, the store deduplication processing device 300 may also include a first pre-processing module. The first pre-processing module may be used to: divide the map into multiple grid areas, and use a grid coding algorithm to assign a grid code to each grid area; obtain the store location information of the existing stores, and determine the grid area where the existing stores are located according to the store location information of the existing stores; establish a first correspondence between the grid codes of the existing stores and the grid areas where the existing stores are located, and store the first correspondence in the existing store database.

In some embodiments, the calculation module 304 may be used to: obtain the similarities between the target store and the N names of the existing stores located in the target grid area and the neighboring grid area based on the first store name and the second store name, where N is an integer greater than or equal to 1; obtain the location similarities between the target store and the existing stores located in the target grid area and the neighboring grid area based on the first store location information and the second store location information; calculate the target similarity based on the N name-related similarities, the location similarity, and the corresponding weight coefficients.

In some examples, the name-related similarity includes character similarity. The calculation module 304 can be used to: perform word segmentation on the first store name and the second store name respectively to obtain the vocabulary corresponding to the first store name and the vocabulary corresponding to the second store name; calculate the word frequency and reverse document frequency of the vocabulary corresponding to the first store name and the vocabulary corresponding to the second store name; select the vocabulary whose word frequency is less than or equal to the redundant word frequency threshold and whose reverse document frequency is greater than the redundant frequency index threshold; based on the selected vocabulary corresponding to the first store name and the selected vocabulary corresponding to the second store name, obtain the character similarity of the target store and the stock stores located in the target grid area and the neighboring grid area.

In some examples, the name-related similarity includes semantic similarity. The calculation module 304 can be used to: convert the first store name and the second store name into a first name digital sequence and a second name digital sequence respectively; input the first name digital sequence and the second name digital sequence into the first model to obtain the semantic similarity between the target store output by the first model and the stock stores located in the target grid area and the neighboring grid area, and the first model is used to output the semantic similarity of the two store names according to the digital sequences converted from the two input store names.

In some examples, the name-related similarity includes store type similarity. The calculation module 304 can be used to: obtain the first store name information according to the first store name; input the first store name information into the second model to obtain the store type probability vector of the target store output by the second model, the second model is used to output the store type probability vector according to the input store name information, and the store type probability vector is used to characterize the probability that the store indicated by the store name belongs to each store type; find the store type probability vector corresponding to the second store name in the stock store database; calculate the similarity between the store type probability vector of the target store and the store type probability vector corresponding to the second store name, and determine the similarity as the store type similarity of the target store and the stock stores located in the target grid area and the neighboring grid area.

In some examples, the calculation module 304 can be used to: obtain the geographical distance between the target store and the existing store based on the first store location information and the second store location information; obtain the location similarity between the target store and the existing store located in the target grid area and the neighboring grid area based on the ratio of the geographical distance and the location deviation threshold.

In some embodiments, the store deduplication processing device may further include a second pre-processing module. The second pre-processing module may be used to: obtain the store name of the existing store, and obtain the store name information according to the store name; input the store name information of the existing store into the second model to obtain the store type probability vector of the existing store output by the second model; establish a second correspondence between the existing store and the store type probability vector of the existing store, and store the second correspondence in the existing store database.

The third aspect of the present invention provides a store deduplication processing device. FIG7 is a schematic diagram of the structure of the store deduplication processing device provided by an embodiment of the present invention. As shown in FIG7, the store deduplication processing device 400 includes a memory 401, a processor 402, and a computer program stored in the memory 401 and executable on the processor 402.

In some examples, the processor 402 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present invention.

Memory 401 may include read-only memory (ROM), random access memory (RAM), magnetic disk storage medium device, optical storage medium device, flash memory device, electrical, optical or other physical/tangible memory storage device. Therefore, generally, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it can be operated to perform the operations described with reference to the store deduplication processing method according to the embodiment of the present invention.

The processor 402 runs the computer program corresponding to the executable program code by reading the executable program code stored in the memory 401, so as to implement the store deduplication processing method in the above embodiment.

In some examples, the store deduplication processing device 400 may also include a communication interface 403 and a bus 404. As shown in FIG7 , the memory 401, the processor 402, and the communication interface 403 are connected through the bus 404 and communicate with each other.

The communication interface 403 is mainly used to realize the communication between the modules, devices, units and/or equipment in the embodiment of the present invention. The input device and/or output device can also be connected through the communication interface 403.

The bus 404 includes hardware, software, or both, coupling the components of the store deduplication processing device 400 to each other. By way of example and not limitation, bus 404 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a Peripheral Component Interconnect Express (PCI) bus, or a 10-bit x86 bus. Express (PCI-E) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local Bus (VLB) bus or other suitable bus or a combination of two or more of the above. Where appropriate, bus 404 may include one or more buses. Although the embodiments of the present invention describe and illustrate specific buses, the present invention contemplates any suitable bus or interconnect.

The fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program instruction is stored. When the computer program instruction is executed by a processor, the store deduplication processing method in the above embodiment can be implemented, and the same technical effect can be achieved. To avoid repetition, it is not repeated here. Among them, the above-mentioned computer-readable storage medium may include a non-transient computer-readable storage medium, such as a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., which is not limited here.

The embodiment of the present invention provides a computer program product. When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device can execute the store deduplication processing method in the above embodiment and achieve the same technical effect. To avoid repetition, it will not be repeated here.

It should be made clear that each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. For the device embodiment, equipment embodiment, computer-readable storage medium embodiment, and computer program product embodiment, the relevant parts can refer to the description part of the method embodiment. The present invention is not limited to the specific steps and structures described above and shown in the figure. After understanding the spirit of the present invention, technicians in this field can make various changes, modifications and additions, or change the order between the steps. In addition, for the sake of brevity, the detailed description of the known method technology is omitted here.

Aspects of the present invention are described above with reference to flowcharts and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It will be understood that each block in the flowchart and/or block diagram, and combinations of blocks in the flowchart and/or block diagram, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine such that these instructions executed by the processor of the computer or other programmable data processing device cause the computer or other programmable data processing device to perform the operation of the computer program. It can realize the functions/actions specified in one or more blocks of the flowchart and/or block diagram. Such a processor may be, but is not limited to, a general purpose processor, a dedicated processor, an application specific processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and/or flowchart and the combination of boxes in the block diagram and/or flowchart can also be implemented by dedicated hardware that performs the specified function or action, or can be implemented by This is accomplished through a combination of specialized hardware and computer instructions.

Those skilled in the art should understand that the above embodiments are exemplary rather than restrictive. Different technical features appearing in different embodiments can be combined to achieve beneficial effects. Based on studying the drawings, instructions and the scope of the patent application, those skilled in the art should be able to understand and implement other variations of the disclosed embodiments. In the scope of the patent application, the term "including" does not exclude other devices or steps; the quantifier "one" does not exclude multiple; the terms "first" and "second" are used to identify names rather than to indicate any specific order. Any diagrammatic mark in the claim should not be understood as limiting the scope of protection. The functions of multiple parts appearing in the claim can be implemented by a single hardware or software module. The fact that certain technical features appear in different subordinate claim items does not mean that these technical features cannot be combined to achieve beneficial results.

S101, S102, S103, S104, S105: Steps

Claims (12)

A store deduplication processing method is characterized in that it includes: a store deduplication device obtains a first store name and first store location information of a target store; the store deduplication device determines a target grid area where the target store is located according to the first store location information; the store deduplication device obtains a second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area from a pre-stored stock store database. information, the neighboring grid area is adjacent to the target grid area; the store deduplication device obtains the target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name, the first store location information, the second store name and the second store location information; when the target similarity is greater than or equal to a preset deduplication similarity threshold, the store deduplication device The target store is removed as a duplicate store; wherein the grid area has a grid code, and before the store deduplication device obtains the second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area in the pre-stored stock store database, it also includes: the store deduplication device obtains the grid code of the target grid area; the store deduplication device removes the second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area according to the grid code of the target grid area The grid coding inverse algorithm is used to obtain the location information of the vertex of the target grid area; the store deduplication device determines the location information of the auxiliary point in the neighboring grid area according to the location information of the vertex of the target grid area; the store deduplication device calculates the grid code of each neighboring grid area based on the location information of the auxiliary point in each neighboring grid area and the grid coding algorithm to determine the neighboring grid area. As described in claim 1, the grid area has a grid code, and the grid codes of adjacent grid areas have the same value for a portion of digits. Before the store deduplication device obtains the second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area in the pre-stored stock store database, the method further includes: the store deduplication device obtains the grid code of the target grid area; the store deduplication device selects the second store name and second store location information of the stock stores located in the target grid area and the neighboring grid area according to the grid code of the target grid area; The grid code of the target grid area is obtained by obtaining the grid code of the candidate grid area, wherein the candidate grid area includes a grid area in which a part of the digital characters in the grid code is the same as a part of the digital characters in the grid code of the target grid area; the store deduplication device determines the grid code of the neighboring grid area in the grid code of the candidate grid area according to the correspondence between the grid area arrangement and the coded digital characters in the grid coding algorithm, so as to determine the neighboring grid area. The method as claimed in claim 1, further comprising: the store deduplication device divides the map into a plurality of grid areas, and uses a grid coding algorithm to assign a grid code to each grid area; the store deduplication device obtains the store location information of the existing store, and determines the grid area where the existing store is located according to the store location information of the existing store; the store deduplication device establishes a first correspondence between the existing store and the grid code of the grid area where the existing store is located, and stores the first correspondence in the existing store database. The method of claim 1, wherein the store deduplication device obtains the target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name, the first store location information, the second store name and the second store location information, including: the store deduplication device obtains the target similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name and the second store name. The store deduplication device obtains the location similarity between the target store and the existing stores in the target grid area and the neighboring grid area based on the first store location information and the second store location information; the store deduplication device calculates the target similarity based on the N name-related similarities, the location similarities and the corresponding weight coefficients. A method as described in claim 4, wherein the name-related similarity includes character similarity, and the store deduplication device obtains N name-related similarities between the target store and the existing stores located in the target grid area and the neighboring grid area based on the first store name and the second store name, including: the store deduplication device performs word segmentation on the first store name and the second store name respectively to obtain the vocabulary corresponding to the first store name and the vocabulary corresponding to the second store name; the store deduplication device performs word segmentation on the first store name and the second store name respectively to obtain the vocabulary corresponding to the first store name and the vocabulary corresponding to the second store name; The re-deduplication device calculates the word frequency and reverse document frequency of the words corresponding to the first store name and the words corresponding to the second store name; the store de-duplication device selects words whose word frequency is less than or equal to the redundant word frequency threshold and whose reverse document frequency is greater than the redundant frequency index threshold; the store de-duplication device obtains the character similarity between the target store and the stock stores located in the target grid area and the neighboring grid area based on the words corresponding to the selected first store name and the words corresponding to the selected second store name. The method of claim 4, wherein the name-related similarity includes semantic similarity, and the store deduplication device obtains N name-related similarities between the target store and the stock stores located in the target grid area and the neighboring grid area based on the first store name and the second store name, including: the store deduplication device converts the first store name and the second store name into a first name digital sequence and a second name digital sequence respectively; the store deduplication device inputs the first name digital sequence and the second name digital sequence into a first model to obtain the semantic similarity between the target store and the stock stores located in the target grid area and the neighboring grid area output by the first model, and the first model is used to output the semantic similarity of the two store names according to the digital sequence converted from the two input store names. A method as described in claim 4, wherein the name-related similarity includes store type similarity, and the store deduplication device obtains N name-related similarities between the target store and the existing stores located in the target grid area and the neighboring grid area based on the first store name and the second store name, including: the store deduplication device obtains first store name information based on the first store name; the store deduplication device inputs the first store name information into a second model to obtain a store type probability vector of the target store output by the second model, and the second model is used to Output a store type probability vector according to the input store name information, the store type probability vector is used to characterize the probability that the store indicated by the store name belongs to each store type; the store deduplication device searches the store type probability vector corresponding to the second store name in the stock store database; the store deduplication device calculates the similarity between the store type probability vector of the target store and the store type probability vector corresponding to the second store name, and determines the similarity as the store type similarity between the target store and the stock stores located in the target grid area and the neighboring grid area. The method as claimed in claim 7, further comprising: the store deduplication device obtains the store name of the existing store, and obtains the store name information according to the store name; the store deduplication device inputs the store name information of the existing store into the second model, and obtains the store type probability vector of the existing store output by the second model; the store deduplication device establishes a second correspondence between the existing store and the store type probability vector of the existing store, and stores the second correspondence in the existing store database. The method of claim 4, wherein the store deduplication device obtains the location similarity between the target store and the existing stores located in the target grid area and the neighboring grid area based on the first store location information and the second store location information, including: the store deduplication device obtains the geographical distance between the target store and the existing stores based on the first store location information and the second store location information; the store deduplication device obtains the location similarity between the target store and the existing stores located in the target grid area and the neighboring grid area based on the ratio of the geographical distance and the location deviation threshold. A store deduplication processing device is characterized in that it includes: a first acquisition module, used to obtain a first store name and first store location information of a target store; a grid area determination module, used to determine a target grid area where the target store is located according to the first store location information; a second acquisition module, used to obtain a second store name and second store location information of stock stores located in the target grid area and a neighboring grid area from a pre-stored stock store database, wherein the neighboring grid area is adjacent to the target grid area; a calculation module, used to obtain a correlation between the target store and the target grid area and the neighboring grid area based on the first store name, the first store location information, the second store name and the second store location information. a target similarity of the stock stores in the target area; a deduplication module, used for removing the target store as a duplicate store when the target similarity is greater than or equal to a preset deduplication similarity threshold; wherein the grid area has a grid code, and the neighboring grid area determination module is used for obtaining the grid code of the target grid area; according to the grid code of the target grid area and the grid code inverse The algorithm is used to obtain the position information of the vertex of the target grid area; the position information of the auxiliary point located in the neighboring grid area is determined according to the position information of the vertex of the target grid area; the grid code of each neighboring grid area is calculated based on the position information of the auxiliary point in each neighboring grid area and the grid coding algorithm to determine the neighboring grid area. A store deduplication processing device, characterized in that the device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the store deduplication processing method as described in any one of claims 1 to 9 is implemented. A computer-readable storage medium, characterized in that a computer program instruction is stored on the computer-readable storage medium, and when the computer program instruction is executed by a processor, a store deduplication processing method as described in any one of claim items 1 to 9 is implemented.

Images