CN111600958B - Service discovery system, service data management method, server, and storage medium - Google Patents

Service discovery system, service data management method, server, and storage medium Download PDF

Info

Publication number
CN111600958B
CN111600958B CN202010437872.XA CN202010437872A CN111600958B CN 111600958 B CN111600958 B CN 111600958B CN 202010437872 A CN202010437872 A CN 202010437872A CN 111600958 B CN111600958 B CN 111600958B
Authority
CN
China
Prior art keywords
service
data
sequence number
layer
consistency
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010437872.XA
Other languages
Chinese (zh)
Other versions
CN111600958A (en
Inventor
刘凤鸣
陈旭升
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Bigo Technology Singapore Pte Ltd
Original Assignee
Guangzhou Baiguoyuan Information Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Guangzhou Baiguoyuan Information Technology Co Ltd filed Critical Guangzhou Baiguoyuan Information Technology Co Ltd
Priority to CN202010437872.XA priority Critical patent/CN111600958B/en
Publication of CN111600958A publication Critical patent/CN111600958A/en
Application granted granted Critical
Publication of CN111600958B publication Critical patent/CN111600958B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/50Network services
    • H04L67/51Discovery or management thereof, e.g. service location protocol [SLP] or web services
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • G06F16/278Data partitioning, e.g. horizontal or vertical partitioning
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L67/00Network arrangements or protocols for supporting network services or applications
    • H04L67/01Protocols
    • H04L67/10Protocols in which an application is distributed across nodes in the network
    • H04L67/1097Protocols in which an application is distributed across nodes in the network for distributed storage of data in networks, e.g. transport arrangements for network file system [NFS], storage area networks [SAN] or network attached storage [NAS]

Abstract

The invention discloses a service discovery system, a service data management method, a server and a storage medium, wherein the system comprises a consistency storage service layer and a service access layer, and the service access layer comprises at least two service access points; the service access point is used for receiving a registration request of the created first service to access the first service, and writing service online data of the first service into the consistency storage service layer; the consistency storage service layer is used for generating a corresponding target global sequence number after receiving service online data and storing the target global sequence number and the service online data in an associated mode. In the service discovery system, a consistency storage service layer and a service access layer containing consistency storage data are arranged, service access points in the service access layer are deployed in all network partitions of the world, and the contained data do not carry out consistency limitation; the layers in the service discovery system are mutually matched, so that the limitation of strong consistency of the global network partition is weakened, and the high availability of service discovery under the global network partition is ensured.

Description

Service discovery system, service data management method, server, and storage medium
Technical Field
The present invention relates to the field of computer technologies, and in particular, to a service discovery system, a service data management method, a server, and a storage medium.
Background
With the development of internet technology, a distributed system has become a currently mainstream data analysis and processing system, and a service discovery system under the distributed system is a key component for the distributed system to perform data analysis and processing.
Currently, there are a zookeeper (an application program coordinating service software) and etcd (service discovery supporting component) in a common service discovery system, which have strong consistency requirements, for example, when a service writes data to the service discovery system, the write operation needs to be allowed by more than half of the nodes under the service discovery system, so that network delay between the nodes is required to be very small, otherwise, the system performance of the whole service discovery system is greatly reduced.
In addition, in practical use, enterprises related to global network applications need to perform global network deployment according to regional partitions, and the service discovery system in the existing mode is not suitable for maintaining strong consistency under network partitions, for example, when a strong consistency service discovery system has nodes in the world (divided by regions such as asia, europe, america and the like) to provide services, if a huge number of service nodes exist in each network partition, a great deal of computing resources are consumed only by using the strong consistency service discovery nodes to perform strong consistency data synchronization, so that the usability of the system is seriously affected.
Disclosure of Invention
The embodiment of the invention provides a service discovery system and a service data management method, which improve the usability of the service discovery system in the global network dimension.
In a first aspect, an embodiment of the present invention provides a service discovery system, including: a consistency storage service layer and a service access layer, the service access layer comprising at least two service access points;
the service access point is used for receiving a registration request of the created first service to access the first service and writing service online data of the first service into the consistency storage service layer;
the consistency storage service layer is used for generating a corresponding target global sequence number after receiving the service online data and storing the target global sequence number and the service online data in a correlated mode.
In a second aspect, an embodiment of the present invention provides a service data management method, which is executed by the service discovery system provided in the first aspect of the present embodiment, including:
the service access point receives a registration request of the created first service to access the first service, and writes service online data of the first service into a consistency storage service layer;
And the consistency storage service layer generates a corresponding target global sequence number after receiving the service online data, and stores the target global sequence number and the service online data in a correlated manner.
In a third aspect, an embodiment of the present invention provides a server, configured to serve as a service access point in a service discovery system provided in an embodiment of the first aspect of the present invention, including:
one or more processors;
a storage means for storing one or more programs;
the one or more programs are executed by the one or more processors, so that the one or more processors are configured to perform the operation steps corresponding to the service access point in the service data management method according to the embodiment of the second aspect of the present invention.
In a fourth aspect, an embodiment of the present invention provides a server, configured to compose a consistency storage service layer in a service discovery system provided by the embodiment of the first aspect of the present invention, including:
one or more processors;
a storage means for storing one or more programs;
the one or more programs are executed by the one or more processors, so that the one or more processors are configured to execute the operation steps corresponding to the consistency storage service layer in the service data management method according to the embodiment of the second aspect of the present invention.
In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the service data management method provided by the embodiment of the second aspect of the present invention.
In the service discovery system and the service data management method provided above, the service discovery system includes: a consistency storage service layer and a service access layer, the service access layer comprising at least two service access points; the service access point is used for receiving a registration request of the created first service to access the first service and writing service online data of the first service into the consistency storage service layer; the consistency storage service layer is used for generating a corresponding target global sequence number after receiving the service online data and storing the target global sequence number and the service online data in a correlated mode. In the service discovery system, a consistency storage service layer and a service access layer containing consistency storage data are arranged, service access points in the service access layer are deployed in all network partitions of the world, and the contained data do not carry out consistency limitation; the scheme is matched with each service access point in the consistency storage data layer and the service access layer, so that the limitation of strong consistency of the global network partition is weakened, the high availability of service discovery under the global network partition is ensured, and the system performance of the distributed system is improved.
Drawings
Fig. 1 is a schematic diagram of a service discovery system according to a first embodiment of the present invention;
FIG. 2 is a flowchart illustrating an exemplary implementation of a service data management method performed by a service discovery system in accordance with a second embodiment of the present invention;
fig. 3 is a schematic hardware structure of a server according to a third embodiment of the present invention;
fig. 4 is a schematic hardware structure of a server according to a fourth embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the following detailed description of the embodiments of the present invention will be given with reference to the accompanying drawings. It should be understood that the described embodiments are merely some, but not all, embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention. In addition, the embodiments of the present invention and features of the embodiments may be combined with each other without conflict, and the embodiments may be referred to and cited with each other.
Example 1
Fig. 1 is a schematic diagram of a service discovery system according to a first embodiment of the present invention, where, as shown in fig. 1, the service discovery system includes: the consistency store service layer 11 and the service access layer 12, the service access layer 12 comprising at least two service access points 121.
The service access point 121 is configured to receive a registration request of the created first service, to access the first service, and write service online data of the first service into the consistency storage service layer 11;
and the consistency storage service layer 11 is used for generating a corresponding target global sequence number after receiving the service online data and storing the target global sequence number and the service online data in a correlated mode.
In this embodiment, the consistency storage service layer 11 may be specifically understood as a storage service layer that globally stores and manages service data associated with services deployed in a distributed system in a global internet dimension. The consistency storage service layer 11 may be formed based on selecting nodes within a small physical range from among nodes constituting the distributed system, wherein the present embodiment may select a part of nodes constituting the consistency storage service according to a geographical distribution of network nodes, and may select the network nodes constituting the consistency storage service layer 11 from among network partitions geographically asia, for example.
Meanwhile, compared with the service discovery system in the existing mode, the service discovery system provided in this embodiment further includes a service access layer 12, and the service access layer 12 specifically includes at least two service access points 121, where the service access points 121 can be specifically understood as a transit node, and specifically, a service created on a server can be accessed to the service discovery system through registration, where the server can be considered as node devices existing in each network partition in the distributed system.
It should be noted that, each service access point 121 does not have a consistency limitation, and one data copy is cached on the service access point, where the data copy is equivalent to the copy data of each global service data stored on the consistency storage service layer 11, in this embodiment, each service access point 121 may be deployed globally, and each service access point 121 establishes a communication connection with the consistency storage service layer 11, so as to implement interconnection between services created in the global internet dimension. For each service access point 121, in theory, the service built on the server in each network partition is allowed to perform access operation, in practical application, preferably, considering the principle of proximity, by way of example, the service created on the server may select one service access point 121 belonging to the same network partition to perform registration access of the service according to the regional information of the network partition to which the server belongs.
In the service discovery system provided in this embodiment, the application scenario may be described as follows: under the dimension of the global Internet, the distributed system forms a plurality of network partitions according to regions, a machine room forming each network partition of the distributed system comprises a certain number of servers, each server is equivalent to a cluster node of the distributed system, required services can be created on each node due to different application requirements, and meanwhile, the synchronization of data among the services needs to be ensured in consideration of business association among different services under the distributed system, so that the created service registration needs to be accessed into a service discovery system to realize management of service data associated with each service.
The existing service discovery system has a working mode equivalent to that when a new service is accessed, stored consistency service data is required to be copied to the new accessed service, and service data of the new access service itself is required to be notified to other accessed services. The use of existing models to manage large-scale service data consumes significant computing resources.
In this embodiment, after a new service is successfully created on the associated server, a service access point 121 may be selected from the service access layer 12 of the provided service discovery system to perform service access through registration, and in this embodiment, the service accessed to the service access point 121 is recorded as a first service, and after the first service is accessed, the service access point 121 may acquire service online data of the first service and write the service online data into the consistency storage service layer 11. Meanwhile, it may be understood that, after the service access point 121 accesses the first service, a registration reply that the access is successful may also be fed back to the first service. Any service access point 121 in the service access layer 12 has the functions described in the foregoing embodiments, and a plurality of service access points 121 may access different first services simultaneously and write service online data of the corresponding first service into the consistency service storage data simultaneously.
In addition, the first service in this embodiment may also be a service that is previously accessed to another service access point 121 in the service access layer 12, and when the service access point 121 accessed by the first service is down, the service may be switched to another service access point 121, so that the service is re-accessed to the service discovery system.
After writing the service online data of the first service into the consistency storage service layer 11, the consistency storage service layer 11 may generate a target global sequence number of the service online data, where the target global sequence number may be regarded as a unique identifier of one service data (such as the service online data) of the associated service in the whole global service data. In this embodiment, through the target global sequence number, service online data uniquely associated with the target global sequence number can be determined, and the service online data and the target global sequence number can be stored in an associated manner. The service access point 121 writes service online data of the first service into the consistency storage service layer 11, and ensures consistency storage of the service data on the local node by generating a corresponding target global sequence number.
The service discovery system provided in the first embodiment includes a consistency storage service layer 11 and a service access layer 12, where the service access point 121 in the service access layer 12 does not limit data consistency of the accessed service, the consistency storage service layer 11 includes only a node in a small physical range, and can implement consistency storage of service data associated with the service in the small physical range by generating a global sequence number for written service online data. Compared with the existing service discovery system, the global consistency of service data on each node is not required to be limited, the resource consumption in service data management is reduced, and the global sequence number setting mode can enable the system to effectively resist downtime of any node (service access point 121) and loss of any service data, so that high availability of the service discovery system is ensured.
Further, the consistency storage service layer 11 includes:
and the sequence number determining unit is used for determining the current maximum target global sequence number stored locally after receiving the service online data, and adding a determined value to the current maximum target global sequence number to serve as the target global sequence number of the first service.
In this embodiment, the consistency storage service layer 11 of the service discovery system may include a sequence number determining unit, through which target global data corresponding to service online data written by the service access point 121 can be effectively determined.
The local storage may be specifically understood as storage on the coherent storage service layer 11, and the current maximum target global sequence number may be specifically understood as a maximum target global sequence number of the target global sequence numbers already stored locally at the current time when the service access point 121 receives the write service online data is taken as the current time. It may be understood that, in this embodiment, the target global sequence number is an integer value that is greater than 0 and sequentially increases, and the sequence number determining unit may directly determine, from among locally stored sequence numbers, a sequence number value that is the largest at the current time, and then may add a determined value to the determined current maximum target global sequence number as the target global sequence number of the first service.
It can be understood that the service discovery system includes a larger number of service access points 121, and the service access points 121 are independent of each other, so that different service access points 121 may write service online data to the consistency storage service layer 11 at the same time, and at this time, the consistency storage service layer 11 may receive service online data written by the service access points 121 at the same time, and the generation operation of the corresponding target global sequence number needs to be performed on each service online data.
For the above determination value, for example, assuming that the consistency storage service layer 11 receives 3 service online data at the same time, after determining the current maximum target global sequence number, to ensure the uniqueness of the target global sequence numbers corresponding to different service online data, 3 determination values starting from 1 and sequentially increasing may be obtained for each service online data, for example, the determination values corresponding to the 3 service online data are respectively 1, 2 and 3, so that the sequence number determination unit may respectively determine a different target global sequence number corresponding to each service online data. In addition, it may be known that if the moments when the consistency storage service layer 11 receives each service online data are different, the corresponding current maximum target global sequence number is determined for each service online data, and then 1 is added as the target global sequence number of the corresponding service online data on the basis of the current maximum target global sequence number, where the determined value is equal to 1.
The above-mentioned optimization-defined sequence number determining unit of the present embodiment embodies an implementation process of generating the target global sequence number by the consistency storage service layer 11, and by this implementation process, consistency and global performance of service data of the accessed service in the service discovery system are ensured.
Further, the service access point 121 is further configured to broadcast an online data write notification of the first service to other service access points 121 in the service access layer 12;
each other service access point 121 is configured to determine a current to-be-compared sequence number of the local cache and feed back the current to-be-compared sequence number to the consistency storage service layer 11 after receiving the online data write notification;
the consistency storage service layer 11 is further configured to feed back service data to be synchronized to the corresponding service access point 121 according to the received current sequence numbers to be compared, so that the service online data of the first service is synchronized to each service access point 121 in the service access layer 12.
On the basis of the above embodiment, the service access point 121 in which the first service is accessed in the service access layer 12 is also optimized to have a function of broadcasting a message notification that the service online data of the first service has been written in the consistency storage service layer 11 to the other service access points 121 in the service access layer 12 after the service online data of the first service is written in the consistency storage service layer 11.
Specifically, after receiving the service online data writing notification, each other service access point 121 in the service access layer 12 may determine, from the locally stored data copies, the target global sequence number with the largest sequence number value corresponding to the current time, and feed back the target global sequence number as the current sequence number to be compared to the consistency storage service layer 11. The purpose of the other service access points 121 to perform the above operations is to update the data content of the locally stored copy of the data. The data copy may be considered as a data set formed by service data stored on the consistency storage service layer 11, and the update operation of the data set may be performed periodically or by triggering, for example, the above-mentioned receiving of the write notification of the service online data corresponds to a triggering condition.
In this embodiment, the consistency storage service layer 11 may compare the current to-be-compared sequence number fed back by each other service access point 121 with the current maximum target global sequence number of the consistency storage service layer 11 when receiving the current to-be-compared sequence number, so as to determine which new service online data are added to the consistency storage service layer 11 relative to each service access point 121, and thus, the newly added service online data are fed back to the corresponding service access point 121 as service data to be synchronized, so as to realize synchronization of the service online data of the first service on each service access point 121.
It can be known that, for each service access point 121, after receiving the service data to be synchronized fed back by the consistency storage service layer 11, the data updating of the locally stored data copy is achieved, and synchronization between the data copy on the service access point 121 and the service data stored on the consistency storage service layer 11 is ensured. In the above-mentioned execution operation of determining service data to be synchronized, only the newly added service data of the service access point 121, which is different from the consistency storage service layer 11, needs to be determined each time, and the newly added service data is used as the service data to be synchronized, so that the system only needs to complete the copying of the incremental data, and the occupation amount of computing resources of data consistency is greatly reduced.
The service discovery system provided in this embodiment realizes the consistency of global service data through the cooperation of the consistency storage service layer 11 and each service access point 121 in the service access layer 12, and the characteristics of message delay or network isolation of network partitions in the global internet dimension, and only causes delay delivery of service data at each service access point 121 through the support of the service discovery system, but does not lose service data and repeatedly send data, thereby improving the usability of the service discovery system.
Further, the consistency storage service layer 11 is specifically configured to:
for each other service access point 121, after receiving the current sequence numbers to be compared of the other service access points 121, obtaining the current maximum target global sequence number locally corresponding to the current time;
the sequence number lacking the current to-be-compared sequence number relative to the current maximum target global sequence number is marked as a to-be-synchronized global sequence number, and each to-be-synchronized global sequence number and corresponding service online data are fed back to the other service access points 121 as to-be-synchronized service data;
the global sequence number to be synchronized at least comprises a target global sequence number corresponding to the first service.
The above limitation of the embodiment gives a specific implementation manner of the consistency storage service layer 11 for determining the service data to be synchronized. The other service access point 121 is opposite to the service access point 121 sending the service online data in the service access layer 12, and when one service access point 121 broadcasts the service online data writing notification, the other service access point 121 is equivalent to the other service access point 121, and if the other service access point also receives the service online data writing notification, the other service access point itself is equivalent to the other service access point 121.
The above-described operations of the present embodiment are applicable to the execution of each other service access point 121 by the consistency store service layer 11. It can be seen that the consistency storage service layer 11 can determine which service online data is newly added to the consistency storage service layer 11 relative to the other service access points 121 by comparing the received current sequence number to be compared with the current maximum target global sequence number and comparing the determined global sequence number to be synchronized (i.e. the sequence number determined relative to the consistency storage service layer 11 in the other service access points 121), so that the incremental sequence number and the corresponding service online data can be determined as the service data to be synchronized.
Further, each of the other service access points 121 is further configured to:
and receiving service data to be synchronized, which are fed back by the consistency storage service layer 11, and feeding back each piece of service data to be synchronized to each second service which is accessed, so that the service online data of the first service is synchronized in each second service which is accessed.
The function of each service access point 121 in the service access layer 12 is also limited in this embodiment, and it can be known that the limitation is to store all service access points 121 of the service layer 11 that feed back the service data to be synchronized for consistency.
For the service access point 121 that receives the service data to be synchronized, a plurality of second services created on the server may be already accessed in a relatively current historical time, so that the service access point 121 can actively feed back the service data to be synchronized to each second service accessed on the service access point after receiving the service data to be synchronized, besides performing local storage, so that the second service associated with the service access point 121 can realize synchronization with service data corresponding to other services directly through the updated data copy of the service access point 121.
Further, each of the other service access points 121 is further configured to:
the information receiving unit is used for receiving the service data to be synchronized, which is fed back by the consistency storage service layer 11 correspondingly, and storing the service data locally;
and the data feedback unit is used for determining the current reference global sequence number of the third service after receiving the data synchronization request sent by the accessed third service, and sending service data to be fed back to the third service according to the maximum global sequence number so as to synchronize the service online data of the first service in the accessed third service.
The above functional description of the present embodiment also provides another functional limitation of each service access point 121 in the service access layer 12, specifically, the service access point 121 in the service access layer 12 may include an information receiving unit and a data feedback unit, and after receiving the service data to be synchronized, the information receiving unit may store the service data to be synchronized first, so as to implement updating of the data copy, where the service access point 121 does not actively push to a service (herein referred to as a third service) that accesses the service discovery system through the service access point 121. And the data feedback unit waits for the accessed third service to send a data synchronization request to the data feedback unit, and performs synchronous feedback of service data to the third service based on the data synchronization request.
In this embodiment, the third service may be a service that has just been accessed to a certain service access point 121, or a service that has been accessed to the service access point 121 but cannot passively receive service data to be synchronized when the service access point 121 has no long connection with the service access point 121. Thus, when the third service needs to be consistent with the service data on the other services, the data synchronization request may be actively sent to the service access point 121, where the data synchronization request includes the reference global sequence number currently provided on the third service, and the reference global sequence number may be understood as the target global sequence number that is the maximum sequence number value in the service data currently existing on the third service. The data feedback unit can determine service data to be fed back to the third service through the reference global sequence number.
Further, the data feedback unit is specifically configured to:
receiving a data synchronization request sent by an accessed third service, and acquiring a reference global sequence number currently possessed by the third service contained in the data synchronization request;
extracting a current maximum global sequence number stored locally at a current time, and determining an increment sequence number determined by the reference global sequence number relative to the current maximum global sequence number;
and extracting service online data corresponding to each increment sequence number from the local storage to form service data to be fed back and sending the service data to the third service.
The specific operation manner of the data feedback unit is given above in this embodiment, it may be known that, after receiving the data synchronization request of the third service, the data feedback unit in the service access point 121 may first obtain the reference global sequence number from the data synchronization request, then select the current maximum global sequence number at the current moment from the service data stored in itself, and also compare the reference global sequence number with the current maximum global sequence number, thereby determining the increment sequence number of the reference global sequence number relative to the current maximum global sequence number, and finally determine the service online data corresponding to each increment sequence number in the local storage and feed back the service online data to the third service as the data to be fed back.
Through the above operation of the embodiment, when the third service needs to synchronize with the data of other services, the service data corresponding to each target global sequence number in the sequence number difference existing at present can be pulled from the service access point, so that the pulling operation of all the service data is avoided, and the resource occupation of the system in the service discovery operation is reduced. And each service access point in the service discovery system provided by the embodiment also realizes the synchronization with the service data of the consistency storage service layer through the functions of the service access points, even if certain down service access points exist, the service accessed to the service access points can be accessed to other arbitrary service access points again, and the service availability is continuously ensured.
Example two
Fig. 2 shows a flowchart of an example implementation of a method for performing service data management by a service discovery system in a second embodiment of the present invention, where the method is applicable to managing service data in a distributed system with multiple network partitions in a global internet dimension. The method may be performed by the service discovery system provided in the first embodiment of the present invention.
In this regard, according to the description of the service discovery system in the above embodiment, it is known that the service discovery system specifically includes: the consistency stores the service layer and the service access layer, and the service access layer comprises at least two service access points.
The service data management method provided in the second embodiment can be specifically regarded as being implemented by cooperation of each layer in the service discovery system, so as to ensure that services created on servers provided in each network partition can implement service data synchronization in the global internet dimension.
It should be noted that, each service access point in this embodiment may perform the following operations in the corresponding steps, and in fig. 2, only one first service is described as an example, and the rest of service access points in the service access layer, which are opposite to the service access point accessed by the first service, are described as other service access points.
As shown in fig. 2, a service data management method provided in the second embodiment of the present invention specifically includes the following operations:
s201, a service access point receives a registration request of the created first service to access the first service, and writes service online data of the first service into a consistency storage service layer.
In this embodiment, the first service may be a service newly created on a server under any network partition to implement a certain service function, or may be a service that has been created and accessed to the service discovery system through a certain service access point, but may be a service that is accessed by selecting a new service access point again due to downtime of a dependent service access point.
In this embodiment, the service access point may be any one of the service access layers selected by the first service, and in addition, the first service in this embodiment is only one name for distinguishing the service in terms of expression, because at least two service access points are included in the service access layer, and thus, each service access point may perform the operation of this step when there is the first service access.
For example, the service access point as the main body of the step may allow the first service to access after receiving the registration request sent by the service on the server, and may write the service online data of the first service into the consistency storage service layer after the first service is successfully accessed.
S202, the consistency storage service layer generates a corresponding target global sequence number after receiving the service online data, and stores the target global sequence number and the service online data in a correlated mode.
In this embodiment, the consistency storage service layer may be formed based on selecting nodes within a small physical range from the nodes forming the distributed system, and is mainly used to globally store and manage service data of services in each network partition, where in this embodiment, part of the nodes forming the consistency storage service may be selected according to the regional distribution of the network nodes.
In this embodiment, the target global sequence number may be regarded as a unique identifier of one service data (such as service online data) of the associated service in the entire global service data. The consistency storage service layer as the execution main body of the step is to generate a corresponding target global sequence number with uniqueness for each service data written by the service access point, so that the consistency storage of the service online data of each service in the service discovery coefficient is realized, and a necessary premise is provided for the synchronization of the service data in the global internet dimension.
In the service data management method provided by the second embodiment of the invention, the service access point in the service access layer does not limit the data consistency of the accessed service, the consistency storage service layer only comprises a node in a small physical range, and the consistency storage of the service data associated with the service in the small physical range can be realized by generating the global sequence number for the written service online data.
On the basis of the above embodiment, as shown in fig. 2, the further optimization of this embodiment includes:
and S203, the service access point broadcasts the online data writing notification of the first service to other service access points in the service access layer.
For example, after the foregoing consistency storage service layer sets the global sequence number for the service online data of the first service and stores the service online data based on the global sequence number, the service access point may immediately perform message broadcasting to other service access points in the service layer, that is, notify the other service access points by sending an online data write notification to the other service access points: the service online data of the first service is already written into the consistency storage service layer, and the service online data which needs to be acquired can be acquired from the consistency storage service layer.
It can be understood that the service access point that performs the writing of the service online data corresponding to the first service may also be used as a member of other service access points to obtain the service online data of the first service accessed by the other service access points.
S204, after receiving the online data writing notification, each other service access point respectively determines the current to-be-compared sequence number of the local cache and feeds the current to-be-compared sequence number back to the consistency storage service layer.
For example, in order to synchronize the service online data of the first service to the local, each other service access point may determine a current sequence number to be compared according to the locally stored data copy, and feed back the sequence number to the executive storage service layer. The current to-be-compared sequence number is equal to a target global sequence number with the maximum sequence number value corresponding to the current moment determined from the locally stored data copy.
And S205, feeding back service data to be synchronized to the corresponding service access points by the consistency storage service layer according to the received current sequence numbers to be compared so as to synchronize the service online data of the first service to other service access points in the service access layer.
In this embodiment, the current to-be-compared sequence numbers sent by different other service access points to the consistency storage service layer are different, and the consistency storage service layer serving as the execution subject of this step may perform, for each other service access point, determination and feedback operations of to-be-synchronized service data based on this step.
For example, the service data to be synchronized may be understood as service data to be updated and added to the data copy, which is different from the data stored on the consistency storage service layer by the corresponding other service access point. The consistency storage service layer can compare the current target global sequence number with the current target global sequence number when receiving the current comparison sequence number according to the current comparison sequence number fed back by other service access points, so that it can be determined that new service online data are added on the consistency storage service layer relative to the service access points, and the newly added service online data are fed back to the corresponding service access points as service data to be synchronized, and synchronization of the service online data of the first service on the service access points is achieved.
The above-mentioned S203 to S205 are consistent with each service access point in the storage service layer and the service access layer, so as to implement consistency of global service data, and in the above-mentioned method provided in this embodiment, loss of service data and occurrence of repeated transmission of data are avoided, thereby also embodying high availability of the service discovery system.
Meanwhile, as a further limitation of the method provided in this embodiment, as shown in fig. 2, the service data management method provided in this embodiment further includes:
s206, the other service access points also respectively receive the service data to be synchronized fed back by the consistency storage service layer, and feed back the service data to be synchronized to the accessed second services, so that the service online data of the first service is synchronized in the accessed second services.
The execution subject of the step is a service access point in the service access layer, and for the service access point that has executed the operation of S204, the step may receive the service data to be synchronized fed back by the consistency storage service layer, and then may actively push the service data to be synchronized to a second service that is accessed through the service access point, so that the second service that is associated with the service access point may directly implement synchronization with service data corresponding to other services through a data copy updated by the service access point. The second service is specifically understood to be a service that is created in a server included in the network partition and has been accessed at the service access point, and there may be a plurality of second services.
In addition, as a further limitation of the method provided in this embodiment, as shown in fig. 2, the service access point in the service data management method provided in this embodiment may further perform the following steps:
s207, each other service access point receives the service data to be synchronized fed back by the consistency storage service layer and stores the service data locally.
For example, this step and S208 described below may be regarded as two different operations performed by the service access point on the received service data to be synchronized, and the two operations may be performed alternatively or both, which mainly relate to the connection state of the accessed service, for example, the present embodiment may perform the operations of S206 after S205, or may perform the operations of S207 and S208 after S205.
Meanwhile, in this step, the service access point may directly store the service data to be synchronized locally after receiving the service data to be synchronized, and does not actively push the service data to be synchronized to the service (herein referred to as the third service) accessed to the service access point, and only updates the data copy of the service access point.
And S208, after receiving the data synchronization request sent by the accessed third service, each other service access point determines the current reference global sequence number of the third service, and feeds back the service data to be synchronized to the third service according to the maximum global sequence number, so that the service online data of the first service is synchronized in the accessed third service.
Based on the above steps, the service access point in this step is equivalent to performing synchronous feedback of service data to the third service based on the data synchronization request after receiving the data synchronization request sent to it by the accessed third service. The third service may be a service that has just been accessed to a certain service access point, or may be a service that has been accessed to a service access point but that has no long connection with the service access point, and that cannot passively receive service data to be synchronized when the service access point data copy is updated, and may actively send a data synchronization request to the service access point when the service data needs to be consistent with service data on other services.
Specifically, the implementation of the above steps may be described as: the service access point firstly acquires a reference global sequence number from the data synchronization request, then selects a current maximum global sequence number at the current moment from service data stored in the service access point, and also compares the reference global sequence number with the current maximum global sequence number, thereby determining an increment sequence number of the reference global sequence number relative to the current maximum global sequence number, and finally determining service online data corresponding to each increment sequence number in the local storage and feeding back the service online data to a third service as data to be fed back.
In the second preferred embodiment of the present invention, the service access point performs service data synchronization on the service accessed to the service access point through the acquired data to be synchronized, so it can be seen that when the service accessed to the service access point needs data synchronization, only the incremental service data, which is different from the data copy in the service access point, needs to be pulled or received, and no full service data needs to be acquired, thereby reducing the resource occupation of the system in service discovery operation. By the service data management method of the service discovery system, the problem that consistency is difficult to achieve due to network isolation service data under network partition is solved, the problem that service data synchronization cannot be effectively performed on services connected with the service access point after the service access point is down, a large amount of computing resources are occupied during data synchronization is solved, incremental synchronization of the data is guaranteed, and operation cost of the system is effectively reduced.
Example III
Fig. 3 is a schematic hardware structure of a server according to a third embodiment of the present invention, where the server is used as a service access point in the service discovery system provided in the first embodiment, and may specifically include: a processor and a storage device. At least one instruction is stored in the storage device, and the instruction is executed by the processor, so that the server can execute the operation steps corresponding to the service access point in the service data management method provided by the second embodiment.
Referring to fig. 3, the server may specifically include: processor 30, storage device 31, display 32, input device 33, output device 34, and communication device 35. The number of processors 30 in the server may be one or more, one processor 30 being taken as an example in fig. 3. The number of storage means 31 in the server may be one or more, one storage means 31 being taken as an example in fig. 3. The processor 30, the storage means 31, the display 32, the input means 33, the output means 34 and the communication means 35 of the server may be connected by a bus or other means, in fig. 3 by way of example.
The storage device 31 is a computer-readable storage medium that can be used to store software programs, computer-executable programs, and modules. The storage device 31 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating device, at least one application program required for a function; the storage data area may store data created according to the use of the server, etc. In addition, the storage 31 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, the storage device 31 may further include memory remotely located with respect to the processor 30, which may be connected to a server via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Generally, the display screen 32 is used for displaying data according to instructions from the processor 30, and also for receiving touch operations on the display screen 32 and transmitting corresponding signals to the processor 30 or other devices. Optionally, when the display screen 32 is an infrared screen, it further includes an infrared touch frame disposed around the display screen 32, which may also be used to receive infrared signals and send the infrared signals to the processor 30 or other server.
The communication device 35 is configured to establish a communication connection with another server, which may be a wired communication device and/or a wireless communication device.
The input means 33 may be used for receiving input digital or character information and generating key signal inputs related to user settings and function control of the server, as well as cameras for capturing images and pick-up devices for capturing audio in video data. The output device 34 may include video equipment such as a display screen and audio equipment such as speakers. The specific composition of the input device 33 and the output device 34 may be set according to the actual situation.
The processor 30 executes various functional applications of the server and data processing, i.e. functional operations of the service access point in the service discovery system described above, by running software programs, instructions and modules stored in the storage means 31.
Specifically, in the embodiment, when the processor 30 executes one or more programs stored in the storage device 31, the following are specifically executed, for example: receiving a registration request of the created first service to access the first service, and writing service online data of the first service into a consistency storage service layer; or broadcasting an online data writing notification of the first service to other service access points in the service access layer; or after receiving the online data writing notification, determining the current sequence number to be compared of the local cache and feeding back the sequence number to the consistency storage service layer; or receiving service data to be synchronized, which is fed back by the consistency storage service layer, and feeding back each piece of service data to be synchronized to each second service which is accessed, so that service online data of the first service is synchronized to operate in each second service which is accessed.
Example IV
Fig. 4 is a schematic hardware structure of a server according to a fourth embodiment of the present invention, and in particular, the server is configured to form a consistency storage service layer in the service discovery system according to the first embodiment, and may specifically include: a processor and a storage device. At least one instruction is stored in the storage device, and the instruction is executed by the processor, so that the server can execute the operation steps corresponding to the consistency storage service layer in the service data management method provided by the second embodiment.
Referring to fig. 4, the server may specifically include: a processor 40, a storage device 41, a display 42, an input device 43, an output device 44, and a communication device 45. The number of processors 40 in the server may be one or more, one processor 40 being exemplified in fig. 4. The number of storage devices 41 in the server may be one or more, one storage device 41 being exemplified in fig. 4. The processor 40, the storage means 41, the display 42, the input means 43, the output means 44 and the communication means 45 of the server may be connected by a bus or by other means, in fig. 4 by way of example.
The storage device 41 is a computer-readable storage medium that can be used to store software programs, computer-executable programs, and modules. The storage device 41 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating device, at least one application program required for a function; the storage data area may store data created according to the use of the server, etc. In addition, the storage 41 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some examples, storage device 41 may further include memory remotely located relative to processor 40, which may be connected to a server via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
Generally, the display screen 42 is used for displaying data according to the instruction of the processor 40, and is also used for receiving a touch operation applied to the display screen 42 and transmitting a corresponding signal to the processor 40 or other devices. Optionally, when the display screen 42 is an infrared screen, it further includes an infrared touch frame disposed around the display screen 42, which may also be used to receive infrared signals and send the infrared signals to the processor 40 or other server.
Communication device 45, which may be a wired communication device and/or a wireless communication device, is used to establish a communication connection with other servers.
The input means 43 may be used for receiving input digital or character information and generating key signal inputs related to user settings and function control of the server, as well as cameras for capturing images and pick-up devices for capturing audio in video data. The output device 44 may include video equipment such as a display screen and audio equipment such as speakers. The specific composition of the input device 43 and the output device 44 may be set according to the actual situation.
The processor 40 executes various functional applications of the server and data processing, i.e., functional operations of the consistency storage service layer in the service discovery system described above, by running software programs, instructions, and modules stored in the storage 41.
Specifically, in the embodiment, when the processor 40 executes one or more programs stored in the storage device 41, the following operations are specifically implemented: generating a corresponding target global sequence number after receiving the service online data, and storing the target global sequence number and the service online data in a correlated manner; or feeding back service data to be synchronized to the corresponding service access point according to the received current sequence numbers to be compared, so that the service online data of the first service is synchronized to other service access points in the service access layer.
Embodiments of the present invention also provide a storage medium containing computer-executable instructions for performing the methods provided by embodiments of the present invention when executed by a computer processor. Illustratively, the foregoing embodiments include: the service access point receives a registration request of the created first service to access the first service, and writes service online data of the first service into a consistency storage service layer; and the consistency storage service layer generates a corresponding target global sequence number after receiving the service online data, and stores the target global sequence number and the service online data in a correlated manner.
It should be noted that, for the server and the storage medium embodiments, since they are substantially similar to the method embodiments, the description is relatively simple, and the relevant points refer to part of the description of the method embodiments.
Note that the above is only a preferred embodiment of the present invention and the technical principle applied. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, while the invention has been described in connection with the above embodiments, the invention is not limited to the embodiments, but may be embodied in many other equivalent forms without departing from the spirit or scope of the invention, which is set forth in the following claims.

Claims (13)

1. A service discovery system, comprising: a consistency storage service layer and a service access layer, the service access layer comprising at least two service access points;
the service access point is used for receiving a registration request of the created first service to access the first service and writing service online data of the first service into the consistency storage service layer;
The consistency storage service layer is used for generating a corresponding target global sequence number after receiving the service online data and storing the target global sequence number and the service online data in a correlated mode;
the consistency storage service layer comprises:
the serial number determining unit is used for determining a current maximum target global serial number stored locally after receiving the service online data, and adding a determined value to the current maximum target global serial number to serve as a target global serial number of the first service;
the plurality of service access points have the possibility of simultaneously accessing different first services and simultaneously writing service online data of the corresponding first services into consistency service storage data.
2. The system of claim 1, wherein the system further comprises a controller configured to control the controller,
the service access point is further configured to broadcast an online data write notification of the first service to other service access points in the service access layer;
each other service access point is respectively used for determining the current sequence number to be compared of the local cache and feeding back the sequence number to the consistency storage service layer after receiving the online data writing notification;
and the consistency storage service layer is further used for feeding back service data to be synchronized to the corresponding service access points according to the received current sequence numbers to be compared so as to synchronize the service online data of the first service to other service access points in the service access layer.
3. The system according to claim 2, wherein the consistency storage service layer is specifically configured to:
for each other service access point, after receiving the current to-be-compared sequence numbers of the other service access points, acquiring the current maximum target global sequence number locally corresponding to the current time;
the sequence number lacking the current to-be-compared sequence number relative to the current maximum target global sequence number is marked as a to-be-synchronized global sequence number, and each to-be-synchronized global sequence number and corresponding service online data are fed back to the other service access points as to-be-synchronized service data;
the global sequence number to be synchronized at least comprises a target global sequence number corresponding to the first service.
4. The system of claim 2, wherein each of the other service access points is further configured to:
and receiving service data to be synchronized, which are fed back by the consistency storage service layer, and feeding back each piece of service data to be synchronized to each second service which is accessed, so that the service online data of the first service is synchronized in each second service which is accessed.
5. The system of claim 2, wherein each of the other service access points is further configured to:
The information receiving unit is used for receiving the service data to be synchronized, which is fed back by the consistency storage service layer, and locally storing the service data;
and the data feedback unit is used for determining the current reference global sequence number of the third service after receiving the data synchronization request sent by the accessed third service, and sending service data to be fed back to the third service according to the maximum global sequence number so as to synchronize the service online data of the first service in the accessed third service.
6. The system according to claim 5, wherein the data feedback unit is specifically configured to:
receiving a data synchronization request sent by an accessed third service, and acquiring a reference global sequence number currently possessed by the third service contained in the data synchronization request;
extracting a current maximum global sequence number stored locally at a current time, and determining an increment sequence number determined by the reference global sequence number relative to the current maximum global sequence number;
and extracting service online data corresponding to each increment sequence number from the local storage to form service data to be fed back and sending the service data to the third service.
7. A service data management method, characterized by being executed by the service discovery system of any one of claims 1 to 6, comprising:
The service access point receives a registration request of the created first service to access the first service, and writes service online data of the first service into a consistency storage service layer;
and the consistency storage service layer generates a corresponding target global sequence number after receiving the service online data, and stores the target global sequence number and the service online data in a correlated manner.
8. The method as recited in claim 7, further comprising:
the service access point broadcasts an online data write notification of the first service to other service access points in a service access layer;
after receiving the online data writing notification, each other service access point respectively determines the current sequence number to be compared of the local cache and feeds the sequence number to the consistency storage service layer;
and the consistency storage service layer feeds back service data to be synchronized to the corresponding service access points according to the received current sequence numbers to be compared so as to synchronize the service online data of the first service to other service access points in the service access layer.
9. The method as recited in claim 8, further comprising:
and the other service access points also respectively receive the service data to be synchronized, which are fed back by the consistency storage service layer, and feed back the service data to be synchronized to the accessed second services, so that the service online data of the first service are synchronized in the accessed second services.
10. The method as recited in claim 8, further comprising:
each other service access point receives the service data to be synchronized, which is fed back by the consistency storage service layer, and stores the service data locally;
after receiving a data synchronization request sent by the accessed third service, each other service access point determines a reference global sequence number currently possessed by the third service, and feeds back service data to be synchronized to the third service according to the maximum global sequence number, so that service online data of the first service are synchronized in the accessed third service.
11. A server for use as a service access point in the service discovery system of any of claims 1-6, comprising:
one or more processors;
a storage means for storing one or more programs;
the one or more programs are executed by the one or more processors, so that the one or more processors are configured to perform the operation steps corresponding to the service access point in the service data management method according to any one of claims 7 to 9.
12. A server for constructing a consistency store service layer in the service discovery system of any of claims 1-6, comprising:
One or more processors;
a storage means for storing one or more programs;
the one or more programs are executed by the one or more processors, so that the one or more processors are configured to perform the operation steps corresponding to the consistency service layer in the service data management method according to any one of claims 7 to 9.
13. A computer-readable storage medium, on which a computer program is stored, characterized in that the program, when being executed by a processor, implements the service data management method according to any one of claims 7-9.
CN202010437872.XA 2020-05-21 2020-05-21 Service discovery system, service data management method, server, and storage medium Active CN111600958B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010437872.XA CN111600958B (en) 2020-05-21 2020-05-21 Service discovery system, service data management method, server, and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010437872.XA CN111600958B (en) 2020-05-21 2020-05-21 Service discovery system, service data management method, server, and storage medium

Publications (2)

Publication Number Publication Date
CN111600958A CN111600958A (en) 2020-08-28
CN111600958B true CN111600958B (en) 2023-06-02

Family

ID=72190625

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010437872.XA Active CN111600958B (en) 2020-05-21 2020-05-21 Service discovery system, service data management method, server, and storage medium

Country Status (1)

Country Link
CN (1) CN111600958B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114531394B (en) * 2020-11-20 2023-11-17 华为技术有限公司 Data synchronization method and device

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103647833A (en) * 2013-12-16 2014-03-19 百度在线网络技术(北京)有限公司 Continuous sequence number generation system and continuous sequence number generation method

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7945639B2 (en) * 2007-06-27 2011-05-17 Microsoft Corporation Processing write requests with server having global knowledge
CN100574326C (en) * 2007-08-08 2009-12-23 南京大学 Based on carrying out service discovery and replacement method under the behavior uniform condition
US10592475B1 (en) * 2013-12-27 2020-03-17 Amazon Technologies, Inc. Consistent data storage in distributed computing systems
CN104281506B (en) * 2014-07-10 2017-02-15 中国科学院计算技术研究所 Data maintenance method and system for file system
CN106126583A (en) * 2016-06-20 2016-11-16 环球大数据科技有限公司 The collection group strong compatibility processing method of a kind of distributed chart database and system
US10892950B2 (en) * 2017-11-24 2021-01-12 Microsoft Technology Licensing, Llc Distributed global discovery servers in operational technology infrastructure
CN108234641B (en) * 2017-12-29 2021-01-29 北京奇元科技有限公司 Data reading and writing method and device based on distributed consistency protocol

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103647833A (en) * 2013-12-16 2014-03-19 百度在线网络技术(北京)有限公司 Continuous sequence number generation system and continuous sequence number generation method

Also Published As

Publication number Publication date
CN111600958A (en) 2020-08-28

Similar Documents

Publication Publication Date Title
WO2020186909A1 (en) Virtual network service processing method, apparatus and system, and controller and storage medium
CN104142896B (en) A kind of buffer control method and system
CN109376197B (en) Data synchronization method, server and computer storage medium
CN111131399B (en) Method and device for dynamically increasing consensus nodes in block chain
US20120179791A1 (en) Consistency domains for replication in distributed computing
CN111338806B (en) Service control method and device
US11445013B2 (en) Method for changing member in distributed system and distributed system
CN111600958B (en) Service discovery system, service data management method, server, and storage medium
JP2016144169A (en) Communication system, queue management server, and communication method
US8725856B2 (en) Discovery of network services
CN114500257A (en) Network configuration distribution method, device, control node and storage medium
JP2020527881A (en) Message transmission method and device
CN114756385B (en) Elastic distributed training method under deep learning scene
CN114900449B (en) Resource information management method, system and device
CN115189931A (en) Distributed key management method, device, equipment and storage medium
CN112000885B (en) List service system, list generation method, server and storage medium
CN114205354A (en) Event management system, event management method, server, and storage medium
CN112052104A (en) Message queue management method based on multi-computer-room realization and electronic equipment
CN112333027A (en) Method for realizing distributed virtual network monitor based on software defined network
CN114531322B (en) Method, system, device and medium for synchronizing multi-gateway data of Internet of things
CN114422280B (en) Network deployment method, device, node and storage medium
CN113271229B (en) Equipment control method and device, storage equipment, safety equipment, switch, router and server
CN110457392B (en) Copy reading and writing method and device
CN116455897A (en) Method for realizing cooperative operation among nodes, cooperative service engine and cooperative system
CN114064619A (en) Data maintenance method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20231016

Address after: 31a, 15 / F, building 30, maple mall, bangrang Road, Brazil, Singapore

Patentee after: Baiguoyuan Technology (Singapore) Co.,Ltd.

Address before: 5-13 / F, West Tower, building C, 274 Xingtai Road, Shiqiao street, Panyu District, Guangzhou, Guangdong 510000

Patentee before: GUANGZHOU BAIGUOYUAN INFORMATION TECHNOLOGY Co.,Ltd.