WO2023168909A1 - Pre-training method and model fine-tuning method for geographical pre-training model - Google Patents

Abstract

The present application provides a pre-training method and model fine-tuning method for a geographical pre-training model, and relates to technical fields of artificial intelligence such as deep learning and graph structures. The pre-training method comprises: obtaining a sample node sequence, wherein the sample node sequence is generated on the basis of a preset point of interest heterogeneous graph and a random walk algorithm, the point of interest heterogeneous graph comprises nodes serving as points of interest and an edge connecting the nodes, names of the nodes are place names of corresponding points of interest, and the edge represents an association relationship that exists between corresponding nodes in the real world; inputting the sample node sequence which acts as a training sample into an initial geographical pre-training model; and controlling the initial geographical pre-training model to be trained according to a preset training target, and outputting a current geographical pre-training model that reaches the training target as a target geographical pre-training model. By means of integrating heterogeneous and multi-modal geographical knowledge into a model pre-training process, the effect of downstream tasks related to a geographical location is improved.

Description

Pre-training method of geographical pre-training model and model fine-tuning method

Cross-references to related applications

This patent application claims priority to the Chinese patent application submitted on March 10, 2022, with the application number 202210230756. incorporated by reference into this disclosure.

Technical field

The present disclosure relates to the field of data processing technology, specifically to the field of artificial intelligence technology such as deep learning and graph structure, and in particular to a pre-training method for a geographical pre-training model and a model fine-tuning method for a geographical pre-training model, as well as corresponding devices and electronic equipment. , computer-readable storage media and computer program products.

Background technique

Different from other fields, the map field is special. The information processing process in the map field often needs to be related to the real world. For example, in a map search engine, when a user enters a search term (or query term), the location of the candidate point of interest (full English name: Point of Interest, English abbreviation: POI) and its relationship with the user's current location Distance is a very important ranking feature.

Contents of the invention

Embodiments of the present disclosure propose a pre-training method for a geographical pre-training model, a model fine-tuning method for a geographical pre-training model, and corresponding devices, electronic equipment, computer-readable storage media and computer program products.

In the first aspect, an embodiment of the present disclosure proposes a pre-training method for a geographical pre-training model, which includes: obtaining a sample node sequence; wherein the sample node sequence is generated based on a preset interest point heterogeneous graph and a random walk algorithm. The point heterogeneous graph includes each node acted by each interest point and the edges connecting each node. The node name is the place name of the corresponding interest point, and the edges represent the correlation between the corresponding nodes in the real world; the sample node sequence is used as a training sample Input the initial geographic pre-training model; control the initial geographic pre-training model to train according to the preset training goals, and output the current geographic pre-training model that reaches the training goal as the target geographic pre-training model; among which, the training goals include guiding the model from training The sub-goal of learning the mapping relationship between the place name of the interest point and the preset location code in the sample. The preset location code corresponds to the geographical block where the corresponding interest point is located in the real world.

In the second aspect, an embodiment of the present disclosure proposes a pre-training device for a geographical pre-training model, including: a sample node sequence acquisition unit configured to obtain a sample node sequence; wherein the sample node sequence is based on preset heterogeneous points of interest Graph and random walk algorithm are generated. The heterogeneous graph of interest points includes each node acted by each interest point and the edges connecting each node. The node names are the place names of the corresponding interest points, and the edges represent the relationships between the corresponding nodes in the real world. relationship; the training sample input unit is configured to input the sample node sequence as a training sample to the initial geographic pre-training model; the geographic pre-training model training unit is configured to control the initial geographic pre-training model to train according to the preset training goals, and The current geographic pre-training model that reaches the training goal is output as the target geographic pre-training model; among which, the training goal includes sub-goals that guide the model to learn the mapping relationship between the place names of points of interest and preset location codes from the training samples. Assume that the location code corresponds to the geographical block where the corresponding point of interest is located in the real world.

In a third aspect, an embodiment of the present disclosure proposes a model fine-tuning method for a geographical pre-training model, which includes: obtaining a target geographical pre-training model; wherein the target geographical pre-training model is based on the geographical pre-training of any one of the first aspects. The model training method is obtained; the new functional requirements of the map application are obtained, and new training samples corresponding to the new functional requirements are determined; based on the target geographical pre-training model, through model fine-tuning technology and new training samples, the new functional requirements are generated corresponding to new geographic model.

In the fourth aspect, an embodiment of the present disclosure proposes a model fine-tuning device for a geographical pre-training model, including: a target geographical pre-training model acquisition unit configured to obtain the target geographical pre-training model; wherein the target geographical pre-training model is based on the following: The geographical pre-training model training device of any one of the second aspects is obtained; a new training sample determination unit configured to obtain new functional requirements of the map application and determine new training samples corresponding to the new functional requirements; a new geographical model generation unit , is configured to generate a new geographical model corresponding to new functional requirements based on the target geographical pre-training model through model fine-tuning technology and new training samples.

In a fifth aspect, embodiments of the present disclosure provide an electronic device. The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor. , the instructions are executed by at least one processor, so that when executed by at least one processor, the pre-training method of the geographical pre-training model can be implemented as described in any implementation manner of the first aspect or as described in any implementation manner of the third aspect. Model fine-tuning method for geographic pre-trained models.

In a sixth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium storing computer instructions. The computer instructions are used to enable the computer to implement geographic pre-training as described in any implementation manner in the first aspect when executed. The pre-training method of the model or the model fine-tuning method of the geographical pre-trained model as described in any implementation of the third aspect.

In a seventh aspect, embodiments of the present disclosure provide a computer program product including a computer program. When executed by a processor, the computer program can implement the pre-training method of a geographical pre-training model as described in any implementation manner in the first aspect. Or the model fine-tuning method of the geographical pre-training model as described in any implementation of the third aspect.

It should be understood that what is described in this section is not intended to identify key or important features of the embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of the drawings

Other features, objects and advantages of the present disclosure will become more apparent upon reading the detailed description of the non-limiting embodiments with reference to the following drawings:

Figure 1 is an exemplary system architecture in which the present disclosure may be applied;

Figure 2 is a flow chart of a pre-training method for a geographical pre-training model provided by an embodiment of the present disclosure;

Figure 3 is a schematic diagram of spatial knowledge of points of interest provided by an embodiment of the present disclosure;

Figure 4 is a flow chart of a method for generating a sample node sequence provided by an embodiment of the present disclosure;

Figure 5 is a schematic diagram of a process of generating a heterogeneous graph of points of interest provided by an embodiment of the present disclosure;

Figure 6 is a schematic diagram of the process of processing sample node sequences at different functional layers in a geographical pre-training model provided by an embodiment of the present disclosure;

Figure 7 is a schematic diagram of a training target for learning the mapping relationship between text and preset position coding provided by an embodiment of the present disclosure;

Figure 8 is a flow chart of a model fine-tuning method for a geographical pre-training model provided by an embodiment of the present disclosure;

Figure 9 is a structural block diagram of a pre-training device for a geographical pre-training model provided by an embodiment of the present disclosure;

Figure 10 is a structural block diagram of a model fine-tuning device for a geographical pre-training model provided by an embodiment of the present disclosure;

FIG. 11 is a schematic structural diagram of an electronic device suitable for executing a pre-training method for a geographical pre-training model and/or a model fine-tuning method for a geographical pre-training model provided by an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, in which various details of the embodiments of the present disclosure are included to facilitate understanding and should be considered to be exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness. It should be noted that, as long as there is no conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

In the technical solution of this disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are in compliance with relevant laws and regulations and do not violate public order and good customs.

FIG. 1 shows an exemplary system architecture 100 in which the pre-training and fine-tuning method of the geographical pre-training model of the present disclosure can be applied, as well as the corresponding apparatus, electronic device and computer-readable storage medium.

As shown in Figure 1, the system architecture 100 may include terminal devices 101, 102, 103, a network 104 and a server 105. The network 104 is a medium used to provide communication links between the terminal devices 101, 102, 103 and the server 105. Network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.

Users can use terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various applications for information communication between the terminal devices 101, 102, 103 and the server 105 may be installed, such as model training applications, model fine-tuning applications, map-related data processing applications, etc.

The terminal devices 101, 102, 103 and the server 105 may be hardware or software. When the terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, laptop computers, desktop computers, etc.; when the terminal devices 101, 102 When 103 is software, it can be installed in the electronic equipment listed above. It can be implemented as multiple software or software modules, or as a single software or software module, and is not specifically limited here. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or it can be implemented as a single server; when the server 105 is software, it can be implemented as multiple software or software modules, or it can be implemented as a single piece of software or software. Modules are not specifically limited here.

The server 105 can provide various services through various built-in applications. Taking a model training application that can provide pre-training services for geographical pre-training models as an example, the server 105 can achieve the following effects when running the model training application: First, Obtain the sample node sequence, which is pre-generated based on the preset interest point heterogeneous graph and the random walk algorithm. The interest point heterogeneous graph includes each node served by each interest point and the edges connecting each node. The node name is the place name of the corresponding point of interest, and the edges represent the correlation between corresponding nodes in the real world; then, the sample node sequence is input as a training sample to the initial geographical pre-training model; finally, the initial geographical pre-training model is controlled to follow the preset The training target is trained, and the current geographical pre-training model that reaches the training target is output as the target geographical pre-training model. The training target includes guiding the model to learn from the training samples the relationship between the place name of the point of interest and the preset location code. A sub-goal of the mapping relationship, the preset location code corresponds to the geographical block where the corresponding point of interest is located in the real world.

The target geographical pre-training model obtained through the above model training process can be applied in practical technical fields related to geographical location (such as the map field), thereby better meeting the growing new geographical knowledge-related needs in these technical fields. Functional requirements, through model fine-tuning technology, based on the geographical pre-training model, quickly obtain a new geographical model corresponding to the new functional requirements.

This process can be implemented through a model fine-tuning application. The server 105 can achieve the following effects when running the model fine-tuning application: first, obtain the target geographical pre-training model; then, obtain the new functional requirements of the map application, and determine the requirements related to the new New training samples corresponding to functional requirements; finally, based on the target geographical pre-training model, a new geographical model corresponding to the new functional requirements is generated through model fine-tuning technology and new training samples.

Both model training and model fine-tuning require more computing resources and stronger computing capabilities. Therefore, subsequent embodiments of the present disclosure provide a pre-training method for a geographical pre-training model or a model fine-tuning method for a geographical pre-training model. It is generally executed by a server 105 with stronger computing power and more computing resources. Correspondingly, the pre-training device of the geographical pre-training model or the model fine-tuning device of the geographical pre-training model is also generally provided in the server 105 . But at the same time, it should be pointed out that when the terminal devices 101, 102, and 103 also have computing capabilities and computing resources that meet the requirements, the terminal devices 101, 102, and 103 can also use the model training application or the model fine-tuning class installed on them. The application completes the above-mentioned various operations performed by the server 105, and then outputs the same results as the server 105. Correspondingly, the pre-training device of the geographical pre-training model or the model fine-tuning device of the geographical pre-training model can also be provided in the terminal equipment 101, 102, 103. In this case, the exemplary system architecture 100 may not include the server 105 and the network 104.

In addition, the server (or terminal device) used to train the target geographical pre-training model may be different from the server (or terminal device) used to perform model fine-tuning operations based on the target geographical pre-training model, so as to separate different model operations. Specially, the target geographic pre-training model or new geographic model trained by the server 105 can also be obtained through model distillation to obtain a lightweight model suitable for placement in the terminal devices 101, 102, 103, that is, it can be identified according to actual needs. The accuracy allows flexible selection of whether to use lightweight models in the terminal devices 101, 102, and 103 or to use a more complex model in the server 105.

It should be understood that the number of terminal devices, networks and servers in Figure 1 is only illustrative. Depending on implementation needs, there can be any number of end devices, networks, and servers.

In order to facilitate understanding of the technical solution provided by the present disclosure, please first refer to the flow chart of a pre-training method for a geographical pre-training model provided in Figure 2, in which the process 200 includes the following steps:

Step 201: Obtain the sample node sequence;

This step is intended for the execution subject of the pre-training method of the geographical pre-training model (such as the server 105 shown in Figure 1) to obtain the sample node sequence generated based on the preset interest point heterogeneous graph and the random walk algorithm. Among them, the interest point heterogeneous graph includes each node served by each interest point and the edges connecting each node. The node names are the place names of the corresponding interest points, and the edges represent the relationships between the corresponding nodes in the real world.

It can be seen that place names of points of interest are usually expressed in text form, and in order to reflect the location association between different nodes, it is necessary to combine spatial knowledge, which is usually expressed in numerical form. Among them, toponym mainly refers to the name of geographical location entities (such as POI, streets and regions). Spatial knowledge mainly includes the specific location of a geographical entity (usually expressed in the form of geographical coordinates), the spatial relationship between different geographical entities (usually expressed in the form of triples) and human movement trajectories (usually in the form of ID sequences). ), please refer to the schematic diagram shown in Figure 3.

According to the above description of place name knowledge and spatial knowledge, we need to overcome two problems to utilize them: 1) Heterogeneous data integration, that is, how to integrate text (including place name knowledge) with different modalities, and numbers and triples , sequence (including spatial knowledge) and other inputs are organically combined as a unified input for the pre-training model; 2) Modal difference, that is, how to represent data of different modalities in the same implicit space so that the model can fully learn different modalities. The knowledge contained in the state can be fully applied in downstream tasks.

This embodiment organically combines two different modalities of knowledge (i.e. heterogeneous), namely place name knowledge represented by text and spatial knowledge represented by numbers, in a graph manner, thereby obtaining a unified input for the pre-training model, that is to say The node names reflect place name knowledge, and the edges between nodes reflect spatial knowledge, thus making the heterogeneous graph itself contain knowledge of two modalities.

Random walk algorithm is also called random walk algorithm. The original meaning of random walk is that it is impossible to predict future development steps and directions based on past performance. In this embodiment, it is used to predict the future development steps and directions based on past trajectories. , generate other possible node sequences to obtain a large number of training samples.

As the name suggests, the sample node sequence generated by the random walk algorithm is a node sequence arranged in time sequence. In the case of 10 different points of interest named using numbers 01-10, an exemplary sample node sequence can be expressed as: 01- 03-08-04, that is, this is a sample node sequence with a walking length of 4. In chronological order, it first passed the interest point numbered 01, then passed the interest point numbered 03, and then passed the interest point numbered 08. points of interest, and finally passed the point of interest numbered 04. It should be noted that algorithm parameters such as walking length, walking direction, and walking weight of each edge can be set based on the actual situation and actual needs, and there are no specific restrictions here.

Step 202: Input the sample node sequence as a training sample into the initial geographical pre-training model;

On the basis of step 201, this step aims to have the above execution subject input each sample node sequence as a training sample into the initial geographical pre-training model. Specifically, according to the model characteristics of the initial geographical pre-training model, when inputting training samples into the initial geographical pre-training model, you can check whether batch input or parallel input is supported to improve the input efficiency and training efficiency of training samples.

Step 203: Control the initial geographical pre-training model to train according to the preset training target, and output the current geographical pre-training model that reaches the training target as the target geographical pre-training model.

On the basis of step 202, this step aims to have the above-mentioned execution subject perform knowledge learning from the input training samples according to the preset training goals, and then finally output the current geographic pre-training model that meets the training goals as the target geographic pre-training model. . Among them, the training goal is a goal used to guide the model on how to learn knowledge from training samples and what kind of knowledge it should learn, so that it can learn the required knowledge more accurately and better.

Since the sample node sequence contains both place name knowledge expressed in text and spatial knowledge expressed in numbers, the training objectives can be divided into two corresponding ones, for example, into the first one used to guide the learning of place name knowledge expressed in text form. The sub-training goal, and the second sub-training goal used to guide the model to learn the mapping relationship between text and the entity it represents in the real world coordinates, so as to effectively learn spatial knowledge. Considering that it is difficult to find the mapping relationship between text and real-world coordinates, it can also be converted into a mapping relationship that seeks to learn the mapping relationship between the text and the real-world geographical block to which the corresponding interest point belongs, and the geographical block can be based on real-world The preset position coding determined by world coordinates can reduce the difficulty of finding mapping relationships through position coding.

The training objectives include sub-goals that guide the model to learn from the training samples the mapping relationship between the place name of the interest point and the preset location code. The preset location code corresponds to the geographical block where the corresponding interest point is located in the real world.

The geographical pre-training model training method provided by the embodiment of the present disclosure can overcome multi-modal geographical knowledge by organically integrating place name knowledge expressed in text form and spatial knowledge expressed in digital form with a heterogeneous graph structure. Due to the existing modal differences, with the help of the initial geographical pre-training model that can process graph data, the geographical knowledge of different modes can be better learned in the same implicit space, thereby providing a better geographical knowledge for downstream tasks related to geographical location. Pre-train the model to improve the task implementation effect of downstream tasks.

On the basis of the above embodiment, a front-end search node attached to each node can also be added to the interest point heterogeneous graph, where the front-end search node records the search words received before the corresponding interest point is selected. , by pre-searching the pre-relationship between nodes, the generated sample node sequence contains search terms for points of interest, and then the geographical pre-training model also combines search terms during training to improve the search for different points of interest. accuracy and comprehensiveness of the correlation.

In the above embodiment, the edges in the interest point heterogeneous graph are used to represent the real-world associations between corresponding nodes. The edges can also be divided into solid edges and dotted edges according to different types of associations, where , the solid edge is determined based on the time series of points of interest recorded in the user's historical travel trajectory. The solid edge represents the travel logical association between different nodes; the dotted edge represents the same block between different nodes in the same geographical block. association. Furthermore, by adding the same-block association represented by the dotted edge, when the sample node sequence is subsequently generated based on the random walk algorithm, more possible node sequences can be obtained due to the node replacement method or the walk length improvement method provided by the same-block association. , and ultimately improve the model training effect by increasing the order of magnitude of training samples.

Please refer to Figure 4. Figure 4 is a flow chart of a method for generating a sample node sequence provided by an embodiment of the present disclosure, which provides a specific implementation method for how to obtain the sample node sequence required in step 201. The process 400 includes the following steps:

Step 401: Obtain user search logs and points of interest database from the map application;

This step is intended for the execution subject (which may still be the server 105 shown in Figure 1, or may be another server different from the service 105 or other device with computing capabilities) to obtain the user search log and interest point database from the map application.

The point of interest database records place name knowledge and spatial knowledge of each point of interest. Geographical name knowledge mainly refers to place names, and geographical names mainly refer to the names of geographical location entities (such as POIs, streets, and regions); spatial knowledge mainly includes the specific location of a geographical location entity (usually expressed in the form of geographical coordinates), and the relationship between different geographical entities. Spatial relationships (usually expressed in the form of triples) and human movement trajectories (usually expressed in the form of ID sequences) can be seen in the schematic diagram shown in Figure 3.

Step 402: Extract the search terms corresponding to each user search and the points of interest actually selected from the user search log, as well as the time series of points of interest corresponding to the user's travel trajectory;

Among them, the point of interest time series is obtained by arranging multiple points of interest involved in the user's travel trajectory in order of arrival time.

Step 403: Use each point of interest as a node, and establish a pre-search node attached to the corresponding node according to the corresponding search term;

Step 404: Establish solid edge connections between corresponding nodes with travel logical associations based on the time series of points of interest;

Step 405: Based on the boundaries of each geographical block in the spatial knowledge and the real-world coordinates of each interest point, establish dotted edge connections between corresponding nodes associated with the same block to obtain a heterogeneous graph of interest points;

The actual constructed interest point heterogeneous graph can be seen in the schematic diagram shown in Figure 5. The points of interest shown in Figure 5 include two types of nodes: POI nodes and pre-search nodes (search words entered when the user selects POI); three types of edges: click edges (search-click-POI in the figure, that is, the user uses this search word search POI), plot co-occurrence edges (POI-co-occurrence-POI in the figure, that is, two POIs appear in the same block, the block is pre-divided through the division method provided by the S2 geometry library), and movement trajectories Edge (the starting point-to-end point in the figure, that is, the two POIs that the user has reached one after another).

Step 406: Perform a random walk operation on the heterogeneous graph of interest points through a random walk algorithm to obtain a sample node sequence.

Specifically, by setting the parameters of the random walk algorithm according to the actual situation, a large number of sample node sequences can be obtained quickly and efficiently.

That is, the technical solution provided by this embodiment starts with user search logs and interest point databases, and then not only introduces search terms into the sample node sequence by setting pre-search nodes, but also sets solid edges and reflections that reflect the logical association of travel. The dotted edges associated with the block expand the possible trajectories that are not recorded in the user's travel trajectory, which not only makes the subsequent sample stage sequence contain more valuable knowledge, but also increases the order of magnitude of the sample node sequence, and ultimately Together, they can improve the comprehensiveness and accuracy of the geographical pre-training model in learning relevant geographical knowledge.

In order to improve the training effect of the target geographical pre-training model as much as possible, and considering that the training sample is a node sequence, the initial geographical pre-training model can also be set to include the first transformation (Tranformer) layer, the aggregation (TranSAGE) layer, the second For the Transformer layer, please refer to the schematic diagram shown in Figure 6. As shown in Figure 6, the first transformation layer (i.e., Transformer (L12) shown in Figure 6) is used to perform first feature coding on the node information of each node that constitutes the sample node sequence, and obtain the node classification code (i.e., Figure shown in 6

) and node context encoding (i.e. as shown in Figure 6

), the aggregation layer (i.e., the TranSAGE layer shown in Figure 6) is used to combine the node classification code of each node with the node classification codes of other nodes to perform feature aggregation, and obtain the aggregated node classification code (i.e., the node classification code shown in Figure 6

), the second transformation layer (i.e. Transformer (L1) shown in Figure 6) is used to perform second feature coding on the aggregated node classification coding and node context coding of each node, and the result of each second feature coding will be The corresponding pre-training target is trained according to the knowledge representation contained in the node information.

In order to facilitate the understanding of the above technical solution, the above data processing process is further explained in detail through a specific calculation method:

After randomly walking on the heterogeneous graph of interest points to obtain the input document D = {v ₁ , v ₂ ,..., v _n }, first use the sentence-piece algorithm to convert the text representation of each node _vi into subword sequence

Then we use the transformer layer to encode _Si :

Subsequently, a transformer-based aggregation layer is used to model the graph structure in the input sequence. For efficient operation, only the sequence aggregation representation of each node is used.

Make the following calculation:

in,

and

It is two linear layers based on different node types and adapting different parameters.

Subsequently, the aggregated representation

its original context

Indicates that they are connected end to end and modeled with another transform layer.

Will be used for pre-training tasks.

The above example is only a specific implementation of the above ideas in a certain application scenario. Those skilled in the art can combine different actual situations based on the data processing ideas reflected in the above first conversion layer, aggregation layer, and second conversion layer. There are many variations and adaptations available, too many to list here.

Based on any of the above embodiments, considering that it is difficult to find the mapping relationship between the text and the real-world coordinates, it can also be converted into seeking the mapping relationship between the learning text and the real-world geographical block to which the corresponding interest point belongs, and The geographical block can use a preset location code determined based on real-world coordinates to reduce the difficulty of finding mapping relationships through location coding.

An encoding rule for preset position encoding can be:

Divide the real world into multiple geographical blocks according to the preset block division method (for example, you can use the division standard provided by the S2geometry library);

Each geographical block is controlled to correspond to a coding token (which can be called a Token); among them, the length of the coding token corresponds to the granularity level of the block division it represents. Every time the granularity level of the block division increases by two levels, the length of the coding token Add one, and the encoding tokens of adjacent geographical block granularity levels (for example, levels 2n-1 and 2n) only differ in the last bit of encoding.

In order to predict multiple levels of coding tokens as efficiently as possible, the prediction task can be converted into position prediction for each bit of coding that constitutes the coding token. For example, predict the corresponding labels for the following three contents: 1) Coding order The character of the card at level 2n-1; 2) the character of the encoding token at level 2n; 3) the penultimate character shared by the encoding token at levels 2n-1 and 2n.

As shown in Figure 7, the training goal is for the geographical pre-training model to learn the mapping relationship between the text-represented point of interest place names and their geographical blocks in the real world. For example, the input is Road A, District C, City B The output of Park No.

It should be noted that this embodiment only provides an exemplary coding rule for preset position coding. The rules and specific details of the number of digits for preset position coding can be adjusted based on the actual situation, as long as the preset position coding can be achieved. Just set the location code to reduce the difficulty of finding the mapping relationship between text and geographical block code.

The above embodiments explain how to obtain a geographical pre-training model through pre-training from various aspects. The following will describe how to use the geographical pre-training model as an available "middleware" or " "Semi-finished products" to provide assistance for other downstream tasks in geography-related technical fields (such as the map field), so that downstream tasks can obtain a higher accuracy and better effect based on the "semi-finished products" through a small amount of training with a small number of samples. new geographic model.

Figure 8 provides a model fine-tuning method for a geographical pre-training model through process 800, which includes the following steps:

Step 801: Obtain the target geographical pre-training model;

Step 802: Obtain the new functional requirements of the map application and determine new training samples corresponding to the new functional requirements;

Step 803: Based on the target geographical pre-training model, generate a new geographical model corresponding to the new functional requirements through model fine-tuning technology and new training samples.

The principle of model fine-tuning technology is equivalent to using the model parameters of the previously trained target geographical pre-training model as the initial model parameters of the new geographical model, so as to directly have a better model structure through parameter inheritance, while using model fine-tuning The premise of the technology should be that the new functional requirements are strongly related to the capabilities of the target geographical pre-training model. Therefore, a new geographical model with better effect and used to realize the new functional requirements can be quickly obtained through only a small number of new training samples. .

Specifically, since the target geographical pre-training model integrates place name knowledge and spatial knowledge of points of interest, various associations between different points of interest are found through learning, so as long as the new functional requirements are related to the learned knowledge, that is The effect can be improved this way.

The following two new functional requirements are introduced respectively:

First, when the new functional requirement is to recommend similar points of interest, determine the user questionnaire corresponding to the recommendation of similar points of interest, and then generate a new training sample based on the content recorded in the user questionnaire;

Subsequently, based on the target geographical pre-training model, through model fine-tuning technology and new training samples, a new geographical model for recommending similar points of interest based on the current points of interest can be generated. That is, the new geographical model at this time can be used for users based on The current point of interest recommends similar points of interest. This new functional requirement can use the target geographical pre-training model to learn the logical association of the same block from the dotted edges that constitute the heterogeneous graph of points of interest (from the fact that points of interest of the same type usually appear "clustered" , has the characteristics of "aggregation"), and then recommends other points of interest of the same type as the current point of interest to the user based on the same block association.

Secondly, when new functional requirements require casual browsing, a small number of new training samples can also be obtained through questionnaires or other forms. Subsequently, based on the target geographical pre-training model, through model fine-tuning technology and new training samples, a new geographical model for recommending other points of interest based on the current point of interest can be generated. That is, the new geographical model at this time can recommend some other points of interest to the user based on the current point of interest. This new functional requirement can use the travel logical association learned by the target geographical pre-training model from the solid edges that constitute the heterogeneous graph of interest points. , and then recommend to the user other points of interest that have a travel logical relationship with the current point of interest based on the travel logical association, so as to satisfy the user's need for casual shopping through this association.

With further reference to Figures 9 and 10, as an implementation of the methods shown in the above figures, the present disclosure respectively provides embodiments of a pre-training device for a geographical pre-training model and a model fine-tuning device for a geographical pre-training model. The embodiment of the pre-training device for the pre-training model corresponds to the embodiment of the pre-training method for the geographical pre-training model shown in Figure 2, and the embodiment of the model fine-tuning device for the geographical pre-training model corresponds to the embodiment of the geographical pre-training method shown in Figure 8 The model corresponds to the embodiment of the model fine-tuning method. The above device can be applied in various electronic devices.

As shown in Figure 9, the pre-training device 900 of the geographical pre-training model in this embodiment may include: a sample node sequence acquisition unit 901, a training sample input unit 902, and a pre-training unit 903. Among them, the sample node sequence acquisition unit 901 is configured to obtain a sample node sequence; wherein the sample node sequence is generated based on a preset interest point heterogeneous graph and a random walk algorithm, and the interest point heterogeneous graph includes the interest points acting as Each node and the edge connecting each node, the node name is the place name of the corresponding point of interest, and the edge represents the association between the corresponding nodes in the real world; the training sample input unit 902 is configured to input the sample node sequence as the initial training sample Geographic pre-training model; the pre-training unit 903 is configured to control the initial geographic pre-training model to be trained according to the preset training goal, and output the current geographic pre-training model that reaches the training goal as the target geographic pre-training model; wherein, training The goal includes sub-goals that guide the model to learn from the training samples the mapping relationship between the place name of the interest point and the preset location code. The preset location code corresponds to the geographical block where the corresponding interest point is located in the real world.

In this embodiment, in the pre-training device 900 of the geographical pre-training model: the specific processing of the sample node sequence acquisition unit 901, the training sample input unit 902, the pre-training unit 903 and the technical effects thereof can be referred to Figure 2 respectively. The relevant descriptions of steps 201-203 in the corresponding embodiment will not be described again here.

In some optional implementations of this embodiment, the interest point heterogeneous graph may also include: a front-end search node attached to each node, and the front-end search node records the search received before the corresponding interest point is selected. word.

In some optional implementations of this embodiment, the edges include solid edges and dotted edges. The solid edges are determined based on the time series of points of interest recorded in the user's historical travel trajectory. The solid edges represent the travel between different nodes. Logical association, dotted edges represent the same-block association between different nodes in the same geographical block.

In some optional implementations of this embodiment, the pre-training device 900 of the geographical pre-training model may also include: a sample node sequence generation unit configured to generate a sample node sequence based on the interest point heterogeneous graph and a random walk algorithm. , the sample node sequence generation unit can be further configured as:

Obtain user search logs and point-of-interest database from the map application; among them, the point-of-interest database records place name knowledge and spatial knowledge of each point of interest;

Extract the search terms corresponding to each user search and the points of interest actually selected from the user search log, as well as the time series of points of interest corresponding to the user's travel trajectory;

Treat each point of interest as a node, and establish a pre-search node attached to the corresponding node based on the corresponding search term;

Establish solid edge connections between corresponding nodes with travel logical associations based on the time series of points of interest;

According to the boundaries of each geographical block in the spatial knowledge and the real-world coordinates of each interest point, a dotted edge connection between corresponding nodes associated with the same block is established to obtain a heterogeneous graph of interest points;

A random walk operation is performed on the heterogeneous graph of interest points through a random walk algorithm to obtain a sample node sequence.

In some optional implementations of this embodiment, the initial geographic pre-training model includes a first conversion layer, an aggregation layer, and a second conversion layer. The first conversion layer is used to convert the node information of each node that constitutes the sample node sequence. The first feature coding is performed separately to obtain node classification coding and node context coding. The aggregation layer is used to combine the node classification coding of each node with the node classification coding of other nodes for feature aggregation to obtain the aggregated node classification coding. The second conversion layer Used to separately perform second feature encoding on the aggregated node classification encoding and node context encoding of each node.

In some optional implementations of this embodiment, the encoding rules of the preset location encoding include: dividing the real world into multiple geographical blocks according to the preset block division method; controlling each geographical block to correspond to a Encoding token; among them, the length of the encoding token corresponds to the block division granularity level it represents. For every two levels of block division granularity level, the length of the encoding token increases by one, and the encoding token of the adjacent geographical block division granularity level increases by one. The cards only differ in the last digit of the code.

As a device embodiment corresponding to the method embodiment of the geographical pre-training model training method, the geographical pre-training model training device provided in this embodiment uses place name knowledge expressed in text form and spatial knowledge expressed in digital form to The graph structures of heterogeneous graphs are organically integrated to overcome the modal differences in multi-modal geographical knowledge. With the help of the initial geographical pre-training model that can process graph data, different modalities can be better learned in the same implicit space. geographical knowledge, thereby providing a better geographical pre-training model for downstream tasks related to geographical location, and improving the task implementation effect of downstream tasks.

As shown in Figure 10, the model fine-tuning device 1000 of the geographic pre-training model in this embodiment may include: a target geographic pre-training model acquisition unit 1001, a new training sample determination unit 1002, and a new geographic model generation unit 1003. Among them, the target geographical pre-training model acquisition unit 1001 is configured to acquire the target geographical pre-training model; wherein the target geographical pre-training model is obtained according to the geographical pre-training model training device as shown in Figure 9; the new training sample determination unit 1002 is configured To obtain new functional requirements for map applications and determine new training samples corresponding to the new functional requirements; the new geographical model generation unit 1003 is configured to use model fine-tuning technology and new training samples based on the target geographical pre-training model. Generate new geographic models corresponding to new functional requirements.

In this embodiment, in the model fine-tuning device 1000 of the geographical pre-training model: the specific processing of the target geographical pre-training model acquisition unit 1001, the new training sample determination unit 1002, the new geographical model generation unit 1003 and the technical effects thereof The relevant descriptions recorded in the embodiment of the model fine-tuning method of the geographical pre-training model as shown in Figure 8 will not be repeated here.

In some optional implementations of this embodiment, the new training sample determination unit 1002 may include a new training sample determination subunit configured to determine new training samples corresponding to new functional requirements. The new training sample determination subunit may be further configured to;

Recommend similar points of interest in response to new functional requirements, and determine user questionnaires corresponding to the recommendations of similar points of interest;

Generate new training samples based on user questionnaires;

Correspondingly, the new geographical model generating unit 1003 can be further configured to:

Based on the target geographical pre-training model, through model fine-tuning technology and new training samples, a new geographical model for recommending similar points of interest based on current points of interest is generated.

In some optional implementations of this embodiment, the new geographic model generating unit 1003 may be further configured to:

In response to new functional requirements for casual shopping, based on the target geographical pre-training model, through model fine-tuning technology and new training samples, a new geographical model is generated for recommending other points of interest in the same block based on the current point of interest.

As a device embodiment corresponding to the method embodiment of the model fine-tuning method for the geographical pre-training model, the model fine-tuning device for the geographical pre-training model provided in this embodiment is based on the target geographical pre-training model and combines new functional requirements and Model fine-tuning technology can quickly obtain a new geographic model that is actually used to meet new functional requirements based on a target geographic pre-trained model that contains more geographic knowledge.

According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores information that can be executed by the at least one processor. The instructions are executed by at least one processor, so that when executed by at least one processor, the pre-training method of the geographical pre-training model and/or the model fine-tuning method of the geographical pre-training model described in any of the above embodiments can be implemented.

According to an embodiment of the present disclosure, the present disclosure also provides a readable storage medium that stores computer instructions. The computer instructions are used to enable the computer to implement the geographical pre-training described in any of the above embodiments when executed. Pre-training methods for models and/or model fine-tuning methods for geographic pre-trained models.

Embodiments of the present disclosure provide a computer program product that, when executed by a processor, can implement the pre-training method of a geographical pre-training model and/or the model fine-tuning method of a geographical pre-training model described in any of the above embodiments.

11 illustrates a schematic block diagram of an example electronic device 1100 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to refer to various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are examples only and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 11 , the device 1100 includes a computing unit 1101 that can execute according to a computer program stored in a read-only memory (ROM) 1102 or loaded from a storage unit 1108 into a random access memory (RAM) 1103 Various appropriate actions and treatments. In the RAM 1103, various programs and data required for the operation of the device 1100 can also be stored. Computing unit 1101, ROM 1102 and RAM 1103 are connected to each other via bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.

Multiple components in the device 1100 are connected to the I/O interface 1105, including: input unit 1106, such as a keyboard, mouse, etc.; output unit 1107, such as various types of displays, speakers, etc.; storage unit 1108, such as a magnetic disk, optical disk, etc. ; and communication unit 1109, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1109 allows the device 1100 to exchange information/data with other devices through computer networks such as the Internet and/or various telecommunications networks.

Computing unit 1101 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing units 1101 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs various methods and processes described above, such as a pre-training method for a geographical pre-training model and/or a model fine-tuning method for a geographical pre-training model. For example, in some embodiments, the pre-training method of the geographical pre-training model and/or the model fine-tuning method of the geographical pre-training model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as the storage unit 1108 . In some embodiments, part or all of the computer program may be loaded and/or installed onto device 1100 via ROM 1102 and/or communication unit 1109 . When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the above-described pre-training method for the geographical pre-training model and/or the model fine-tuning method for the geographical pre-training model may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to perform the pre-training method of the geographical pre-training model and/or the model fine-tuning of the geographical pre-training model in any other suitable manner (eg, by means of firmware). method.

Various implementations of the systems and techniques described above may be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip implemented in a system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or a combination thereof. These various embodiments may include implementation in one or more computer programs executable and/or interpreted on a programmable system including at least one programmable processor, the programmable processor The processor, which may be a special purpose or general purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device. An output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that the program codes, when executed by the processor or controller, cause the functions specified in the flowcharts and/or block diagrams/ The operation is implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, laptop disks, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or a trackball) through which a user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and may be provided in any form, including Acoustic input, voice input or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., A user's computer having a graphical user interface or web browser through which the user can interact with implementations of the systems and technologies described herein), or including such backend components, middleware components, or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communications network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

Computer systems may include clients and servers. Clients and servers are generally remote from each other and typically interact over a communications network. The relationship of client and server is created by computer programs running on corresponding computers and having a client-server relationship with each other. The server can be a cloud server, also known as cloud computing server or cloud host. It is a host product in the cloud computing service system to solve the management difficulties existing in traditional physical host and virtual private server (VPS, Virtual Private Server) services. Large, weak business scalability.

It should be understood that various forms of the process shown above may be used, with steps reordered, added or deleted. For example, each step described in the present disclosure may be executed in parallel, sequentially, or in a different order. As long as the desired results of the technical solution disclosed in the present disclosure can be achieved, there is no limitation here.

The above-mentioned specific embodiments do not constitute a limitation on the scope of the present disclosure. It will be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions are possible depending on design requirements and other factors. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this disclosure shall be included in the protection scope of this disclosure.

Claims (21)

1. A pre-training method for geographical pre-training models, including:

   Obtain a sample node sequence; wherein the sample node sequence is generated based on a preset interest point heterogeneous graph and a random walk algorithm. The interest point heterogeneous graph includes each node acted by each interest point and connecting each of the nodes. The edge of the node is named the place name of the corresponding point of interest, and the edge represents the association between the corresponding nodes in the real world;

   Enter the sample node sequence as a training sample into the initial geographical pre-training model;

   Control the initial geographic pre-training model to train according to a preset training goal, and output the current geographic pre-training model that reaches the training goal as a target geographic pre-training model; wherein the training goal includes guiding the model from the The sub-goal of learning the mapping relationship between the place name of the point of interest and the preset location code in the training sample, where the preset location code corresponds to the geographical block where the corresponding point of interest is located in the real world.
2. The method according to claim 1, wherein the interest point heterogeneous graph further includes: a front-end search node attached to each of the nodes, and the front-end search node records the reception before the corresponding interest point is selected. search terms.
3. The method according to claim 1, wherein the edge includes a solid edge and a dotted edge, the solid edge is determined based on the time series of points of interest recorded in the user's historical travel trajectory, and the solid edge represents different nodes. The dotted edges represent the same-block association between different nodes in the same geographical block.
4. The method according to claim 1, further comprising: generating the sample node sequence based on the heterogeneous graph of interest points and the random walk algorithm, which is based on the heterogeneous graph of interest points and the random walk. The algorithm generates the sample node sequence, including:

   Obtain user search logs and point-of-interest database from the map application; wherein, the point-of-interest database records place name knowledge and spatial knowledge of each point of interest;

   Extract from the user search log the search terms corresponding to each user search and the points of interest actually selected, as well as the time series of points of interest corresponding to the user's travel trajectory;

   Treat each point of interest as a node, and establish a pre-search node attached to the corresponding node based on the corresponding search term;

   Establish solid edge connections between corresponding nodes with travel logical associations based on the interest point time series;

   According to the boundaries of each geographical block and the real-world coordinates of each point of interest in the spatial knowledge, establish a dotted edge connection between corresponding nodes associated with the same block to obtain the heterogeneous graph of the point of interest;

   The sample node sequence is obtained by performing a random walk operation on the interest point heterogeneous graph through the random walk algorithm.
5. The method according to claim 1, wherein the initial geographical pre-training model includes a first conversion layer, an aggregation layer, and a second conversion layer, and the first conversion layer is used to convert each of the sample node sequences that constitute the sample node sequence. The node information of the node is separately encoded with the first feature to obtain the node classification code and the node context code. The aggregation layer is used to combine the node classification code of each node with the node classification codes of other nodes to perform feature aggregation to obtain the aggregated node classification. Encoding, the second conversion layer is used to perform second feature encoding on the aggregated node classification encoding and node context encoding of each node.
6. The method according to any one of claims 1 to 5, wherein the encoding rules of the preset location encoding include: dividing the real world into multiple geographical blocks in a preset block division manner; controlling each geographical area Each block corresponds to a coding token; wherein, the length of the coding token corresponds to the block division granularity level it represents. Every time the block division granularity level increases by two levels, the length of the coding token increases by one, and the length of the coding token increases by one. The coded tokens at the geographic block granularity level differ only in the last digit of the code.
7. A model fine-tuning method for geographical pre-training models, including:

   Obtain a target geographical pre-training model; wherein the target geographical pre-training model is obtained according to the geographical pre-training model training method according to any one of claims 1-6;

   Obtain new functional requirements for map applications and determine new training samples corresponding to the new functional requirements;

   Based on the target geographical pre-training model, a new geographical model corresponding to the new functional requirements is generated through model fine-tuning technology and the new training samples.
8. The method according to claim 7, wherein determining new training samples corresponding to new functional requirements includes;

   Recommending similar points of interest in response to the new functional requirements, and determining a user questionnaire corresponding to the recommendation of similar points of interest;

   Generate the new training sample according to the user questionnaire;

   Correspondingly, on the basis of the target geographical pre-training model, through model fine-tuning technology and the new training samples, a new geographical model corresponding to the new functional requirements is generated, including:

   Based on the target geographical pre-training model, a new geographical model for recommending similar points of interest based on the current points of interest is generated through model fine-tuning technology and the new training samples.
9. The method according to claim 7, wherein the new geographical model corresponding to the new functional requirements is generated based on the target geographical pre-training model through model fine-tuning technology and the new training samples, including :

   In response to the new functional requirement for casual shopping, based on the target geographical pre-training model, through model fine-tuning technology and the new training samples, a new geographical model for recommending other points of interest based on the current point of interest is generated .
10. A pre-training device for a geographical pre-training model, including:

    A sample node sequence acquisition unit is configured to obtain a sample node sequence; wherein the sample node sequence is generated based on a preset interest point heterogeneous graph and a random walk algorithm, and the interest point heterogeneous graph includes each interest point acting as a Each node and the edge connecting each of the nodes, the node name is the place name of the corresponding point of interest, and the edge represents the association relationship between the corresponding nodes that exists in the real world;

    a training sample input unit configured to input the sample node sequence as a training sample into the initial geographic pre-training model;

    A pre-training unit configured to control the initial geographic pre-training model to train according to a preset training goal, and output the current geographic pre-training model that reaches the training goal as a target geographic pre-training model; wherein, the training The goal includes a sub-goal that guides the model to learn from the training samples a mapping relationship between the place name of the point of interest and a preset location code. The preset location code corresponds to the geographical block where the corresponding point of interest is located in the real world. .
11. The device according to claim 10, wherein the interest point heterogeneous graph further includes: a front-end search node attached to each of the nodes, and the front-end search node records the reception before the corresponding interest point is selected. search terms.
12. The device according to claim 10, wherein the edge includes a solid edge and a dotted edge, the solid edge is determined based on the time series of points of interest recorded in the user's historical travel trajectory, and the solid edge represents different nodes. The dotted edges represent the same-block association between different nodes in the same geographical block.
13. The apparatus according to claim 10, further comprising: a sample node sequence generating unit configured to generate the sample node sequence based on the interest point heterogeneous graph and the random walk algorithm, the sample node sequence generating unit is further configured to:

    Obtain user search logs and point-of-interest database from the map application; wherein, the point-of-interest database records place name knowledge and spatial knowledge of each point of interest;

    Extract from the user search log the search terms corresponding to each user search and the points of interest actually selected, as well as the time series of points of interest corresponding to the user's travel trajectory;

    Treat each point of interest as a node, and establish a pre-search node attached to the corresponding node based on the corresponding search term;

    Establish solid edge connections between corresponding nodes with travel logical associations based on the interest point time series;

    According to the boundaries of each geographical block and the real-world coordinates of each point of interest in the spatial knowledge, establish a dotted edge connection between corresponding nodes associated with the same block to obtain the heterogeneous graph of the point of interest;

    The sample node sequence is obtained by performing a random walk operation on the interest point heterogeneous graph through the random walk algorithm.
14. The device according to claim 10, wherein the initial geographic pre-training model includes a first conversion layer, an aggregation layer, and a second conversion layer, and the first conversion layer is used to convert each of the sample node sequences that constitute the sample node sequence. The node information of the node is separately encoded with the first feature to obtain the node classification code and the node context code. The aggregation layer is used to combine the node classification code of each node with the node classification codes of other nodes to perform feature aggregation to obtain the aggregated node classification. Encoding, the second conversion layer is used to perform second feature encoding on the aggregated node classification encoding and node context encoding of each node.
15. The device according to any one of claims 10 to 14, wherein the encoding rules of the preset location encoding include: dividing the real world into multiple geographical blocks according to a preset block division method; controlling each geographical area Each block corresponds to a coding token; wherein, the length of the coding token corresponds to the block division granularity level it represents. Every time the block division granularity level increases by two levels, the length of the coding token increases by one, and the length of the coding token increases by one. The coded tokens at the geographic block granularity level differ only in the last digit of the code.
16. A model fine-tuning device for geographical pre-training models, including:

    The target geographical pre-training model acquisition unit is configured to acquire the target geographical pre-training model; wherein the target geographical pre-training model is obtained according to the geographical pre-training model training device according to any one of claims 10 to 15;

    The new training sample determination unit is configured to obtain new functional requirements of the map application and determine new training samples corresponding to the new functional requirements;

    The new geographic model generation unit is configured to generate a new geographic model corresponding to the new functional requirements based on the target geographic pre-training model through model fine-tuning technology and the new training samples.
17. The apparatus according to claim 16, wherein the new training sample determining unit includes a new training sample determining subunit configured to determine new training samples corresponding to new functional requirements, the new training sample determining subunit is further configured to ;

    Recommending similar points of interest in response to the new functional requirements, and determining a user questionnaire corresponding to the recommendation of similar points of interest;

    Generate the new training sample according to the user questionnaire;

    Correspondingly, the new geographical model generation unit is further configured to:

    Based on the target geographical pre-training model, a new geographical model for recommending similar points of interest based on the current points of interest is generated through model fine-tuning technology and the new training samples.
18. The device according to claim 16, wherein the new geographical model generating unit is further configured to:

    In response to the new functional requirement for casual shopping, based on the target geographical pre-training model, through model fine-tuning technology and the new training samples, a new geographical model for recommending other points of interest based on the current point of interest is generated .
19. An electronic device including:

    at least one processor; and

    a memory communicatively connected to the at least one processor; wherein,

    The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can perform any one of claims 1-6. The pre-training method of the geographical pre-training model and/or the model fine-tuning method of the geographical pre-training model according to any one of claims 7-9.
20. A non-transient computer-readable storage medium storing computer instructions, the computer instructions being used to cause the computer to execute the pre-training method and/or rights of the geographical pre-training model according to any one of claims 1-6 The model fine-tuning method of the geographical pre-training model described in any one of requirements 7-9.
21. A computer program product, including a computer program, which when executed by a processor implements the pre-training method of the geographical pre-training model according to any one of claims 1-6 and/or any of claims 7-9. A model fine-tuning method for a geographical pre-training model.

Images