CA3144046A1 - System and method of synchronizing data in neo4j to search server - Google Patents
System and method of synchronizing data in neo4j to search server Download PDFInfo
- Publication number
- CA3144046A1 CA3144046A1 CA3144046A CA3144046A CA3144046A1 CA 3144046 A1 CA3144046 A1 CA 3144046A1 CA 3144046 A CA3144046 A CA 3144046A CA 3144046 A CA3144046 A CA 3144046A CA 3144046 A1 CA3144046 A1 CA 3144046A1
- Authority
- CA
- Canada
- Prior art keywords
- search server
- neo4j
- image data
- request
- document
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 68
- 230000001360 synchronised effect Effects 0.000 claims description 32
- 238000013507 mapping Methods 0.000 claims description 9
- 230000008569 process Effects 0.000 description 7
- 238000012800 visualization Methods 0.000 description 6
- 238000010586 diagram Methods 0.000 description 4
- 230000009286 beneficial effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/901—Indexing; Data structures therefor; Storage structures
- G06F16/9024—Graphs; Linked lists
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Computational Linguistics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention discloses to a method and system of synchronizing data in Neo4j to search server. The method comprises: starting Neo4j image database, creating search sever client, constructing transaction event processor, and registering transaction event processor to Neo4j service; using transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map; using transaction event processor to construct a batch of document request objects of search server based on the document request object, using search server client to synchronize a batch of document request objects to search server cluster. The system of synchronizing data in Neo4j to search server provided by the present invention adopts the above-mentioned method of synchronizing data in Neo4j to search server and solves problem of synchronizing image data in Neo4j to search server in real time.
Description
Field [0001] The present disclosure relates to technical field of image database field, particularly to method and system of synchronizing data in Neo4j to search server.
Background
Background
[0002] Neo4j image database is a very popular image database that provides powerful and stable image storage and image query functions, which is widely used in knowledge graph application, and search server is often used to provide support for search service of application system.
[0003] Graph visualization is an application field of knowledge graph, which usually includes services such as search query and display of image data. In the visualization application scenario of the knowledge graph, the search server can be used to assist in the construction of query services for image data, and the image data can be synchronized to the search server cluster, and then the image data can be efficiently searched from the search server to avoid directly scanning the full amount of data from Neo4j library, thereby improving the performance of the search service for image data.
Therefore, it is a problem that must be solved to synchronize the image data in Neo4j to search server in real time.
Invention Content
Therefore, it is a problem that must be solved to synchronize the image data in Neo4j to search server in real time.
Invention Content
[0004] The purpose of the present invention is to provide a method and system of synchronizing data in Neo4j to a search server and solve the problem of how to synchronize image data in Neo4j to search server in real time.
[0005] To achieve the above-mentioned purpose, the present invention provides following solutions:
Date Recue/Date Received 2021-12-24
Date Recue/Date Received 2021-12-24
[0006] A method of synchronizing data in Neo4j to search server, wherein, the method comprises:
[0007] Starting Neo4j image database, creating a search sever client, constructing a transaction event processor, and registering the transaction event processor to Neo4j service;
[0008] Using the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map;
[0009] Using the transaction event processor to construct a batch of document request objects of the search server based on the document request object, using search server client to synchronize a batch of document request objects to search server cluster.
[0010] Preferably, the search server adopts Elasticsearch search server.
[0011] Preferably, creating a search server client, constructing a transaction event handler, and registering the transaction event handler to the Neo4j service, the method comprises:
[0012] Starting Neo4j image database, reading search server cluster address and search server index name in Neo4j image database configuration file;
[0013] Creating a search server client according to the search server cluster address;
[0014] Using the search server client to check whether the search server index name in the Neo4j image database configuration file exists, if not exists, creating search server index name;
[0015] Constructing a transaction event processor according to the Neo4j image database configuration Date Recue/Date Received 2021-12-24 file and registering the transaction event processor to Neo4j service.
[0016] Preferably, using the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map, the method comprises:
[0017] Creating a Map object in the transaction event processor, and the Map object adopts a key-value set mapping method;
[0018] Respectively constructing image data node and image data relationship in Neo4j image database into document request objects of search server;
[0019] Storing object constructed by entity id and index type of the image data node as the key to Map object, and/or
[0020] Storing the object constructed by entity id and index type of the image data relationship as the key to Map object;
[0021] Storing the document request object as value corresponding to Map object.
[0022] Specifically, the method of separately constructing image data nodes in Neo4j image database into document request objects of search server, comprising:
[0023] Determining whether image data nodes need to be synchronized according to Neo4j image database configuration file, if image data nodes need to be synchronized, perform one or more of following operations to synchronize image data nodes:
Date Recue/Date Received 2021-12-24
Date Recue/Date Received 2021-12-24
[0024] Constructing the image data nodes created in the transaction object of Neo4j image database into index request document objects of search server;
[0025] Constructing the image data nodes deleted from the transaction object of Neo4j image database into delete request document object of search server;
[0026] Constructing newly-labeled image data nodes in the transaction object of Neo4j image database into index request document object of search server;
[0027] Constructing the image data nodes with delete label in the transaction object of Neo4j image database into delete request document object of search server;
[0028] Constructing the image data nodes with newly added attribute in the transaction object of Neo4j image database into index request document object of search server;
[0029] Constructing the image data nodes with delete attribute in the transaction object of Neo4j image database into the delete request document object of search server.
[0030] Specifically, the method of separately constructing image data relationships, wherein the image data relationships need to be synchronized in Neo4j image database into document request objects of search server, comprising:
[0031] Determining whether the image data relationships need to be synchronized according to Neo4j image database the configuration file, if image relationship nodes need to be synchronized, perform one or more of the following operations to synchronize image data relationships:
[0032] Constructing the image data relationships created in the transaction object of Neo4j image database Date Recue/Date Received 2021-12-24 into index request document objects of search server;
[0033] Constructing the image data relationships deleted from the transaction object of Neo4j image database into delete request document objects of search server;
[0034] Constructing newly-labeled image data relationships in the transaction object of Neo4j image database into index request document objects of search server;
[0035] Constructing the image data relationships with delete attribute in the transaction object of Neo4j image database into the delete request document objects of search server.
[0036] Furthermore, using the transaction event processor to construct a batch of document request objects of search server based on the document request object, the method of using the search server client to synchronize a batch of document request objects to search server, comprising:
[0037] Using the transaction event processor to construct a batch if document request objects of search server based on the document request object;
[0038] Search server client uses an asynchronous request mode or a synchronous request mode to perform image data synchronization operations and synchronizes a batch of document request objects to search server cluster.
[0039] Preferably, according to Neo4j's configuration file, determining of search server client adopts asynchronous request mode or synchronous request mode to perform image data synchronization operations.
[0040] Preferably, a batch of document request objects are a Bulk request object, the asynchronous request Date Recue/Date Received 2021-12-24 mode is an asynchronous Bulk request mode, and the synchronous request mode is synchronous Bulk request method.
[0041] A system of synchronizing data in Neo4j to a search server, the system comprises an initialization module, a mapping module and a synchronizing module, wherein,
[0042] The initialization module configured to start Neo4j image database, create a search sever client, constructing a transaction event processor, and register the transaction event processor to Neo4j service;
[0043] The mapping module configured to use the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then store in Map;
[0044] The synchronizing module configured to use the transaction event processor to construct a batch of document request objects of the search server based on the document request object and use search server client to synchronize a batch of document request objects to search server cluster.
[0045] Compared with the prior art, the method and system for synchronizing data in Neo4j to search server provided by the present invention has the following beneficial effects:
[0046] The method of synchronizing data in Neo4j to a search server provided by the present invention, the search server client and transaction event processor are first created in Neo4j image database, then using the transaction event processor to construct the image data node and the image data relationship into the document request object of the search server and storing in Map, finally, using the transaction event processor to construct a batch of document request objects of search server, and using the search server client to synchronize a batch of document request objects to search server cluster. This allows Neo4j to automatically synchronize changed image data after submitted data is written into the transaction to search Date Recue/Date Received 2021-12-24 server cluster when performing writing operation, thereby helping developers to use Neo4j image database and search server to build high-quality graph visualization application.
[0047] The system of synchronizing data in Neo4j to a search server provided by the present invention adopts the above-mentioned method of synchronizing data in Neo4j to a search server and solves the problem of how to synchronize image data in Neo4j to a search server in real time.
Drawing Description
Drawing Description
[0048] The drawings described here are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary implementations of the present invention and their descriptions are used to explain the present invention, and do not constitute an improper limitation of the present invention. In the attached figures:
[0049] Figure 1 is a process diagram of a method of synchronizing data in Neo4j to search server in implementation of the present application;
[0050] Figure 2 is a process diagram of starting Neo4j image database in implementation of the present application.
[0051] Figure 3 is a process diagram of image data synchronizing in implementation of the present application;
[0052] Figure 4 is a system module diagram of a method of synchronizing data in Neo4j to search server in implementation of the present application.
Specific implementation methods Date Recue/Date Received 2021-12-24
Specific implementation methods Date Recue/Date Received 2021-12-24
[0053] To make clearer purpose, characteristic and benefit of the above-mentioned invention, the following will describe the technical solutions of the implementations in the present application with accompanying drawings, obviously the described implementations are only a part of the implementations in the present application. Based on the implementations in the present application, all other implementations obtained by those of ordinary skilled in the art will fall in the protection scope of the present application.
[0054] Implementation one
[0055] Please refer to Figure 1, a method of synchronizing data in Neo4j to a search server, comprising:
[0056] Starting Neo4j image database, creating a search sever client, constructing a transaction event processor, and registering the transaction event processor to Neo4j service;
[0057] Using the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map;
[0058] Using the transaction event processor to construct a batch of document request objects of the search server based on the document request object, using search server client to synchronize a batch of document request objects to search server cluster.
[0059] The method of synchronizing data in Neo4j to a search server provided in this implementation enables Neo4j to automatically convert changed image data into a batch document request objects after submitted data is written into the transaction when performing a write operation, furthermore, using search server client to synchronize a batch of document request objects to search server cluster, thereby helping developers to use the Neo4j image database and search server to build high-quality graph visualization applications.
Date Recue/Date Received 2021-12-24
Date Recue/Date Received 2021-12-24
[0060] In specific implementation, the search server can use the Elasticsearch search server. The Elasticsearch search server provides an open-source distributed document storage and analysis search engine, which can provide efficient index query capabilities, and has a wide range of applications in the field of big data search, which is often used to provide support for the search service of the application system.
[0061] In addition, the method in this implementation can be developed based on Neo4j image database community edition (hereinafter referred to as Neo4j) and implemented using the Java programming language. Neo4j has a core extension factory interface KernelExtensionFactory, this interface will create a Neo4j life cycle object, this Neo4j life cycle object can perform custom extension operations when Neo4j is started and closed. In addition, Neo4j will trigger a data writing transaction when performing a writing operation, the transaction event processor executes corresponding methods in the three phases before the start of the data writing transaction, after the transaction is committed, and after the transaction is rolled back, to synchronize the Neo4j middle map data to the search server. Neo4j saves transaction data in a transaction object (TransactionData) which is passed as a parameter to the method of the transaction event handler interface.
[0062] The method of synchronizing data in Neo4j to a search server provided by an implementation of the present invention uses these two mechanisms of Neo4j to read the search server cluster and search in the neo4j configuration file (neo4j.conf file) during server index configuration, initialize the search server client (for example: Elasticsearch client) and create index. When Neo4j performs writing operation, in the method of before data writing transaction is executed (beforeCommit() method), extracting image data in transaction object, and in the method after the data write transaction is committed (afterCommit() method) to synchronize extracted image data to search server cluster. Among them, the method of creating a search server client, constructing a transaction event handler, and registering transaction event handler to Neo4j service comprises:
Date Recue/Date Received 2021-12-24
Date Recue/Date Received 2021-12-24
[0063] Starting Neo4j image database, reading search server cluster address and search server index name in the configuration file of Neo4j image database;
[0064] Creating search server client according to the search server cluster address;
[0065] Using search server client to check whether search server index name in the configuration file of Neo4j image database exists, and if not exists, create index name of search server;
[0066]
Constructing a transaction event handler (ElasticSearchEventHandler) according to the configuration file of Neo4j image database and registering the transaction event handler in Neo4j service.
Constructing a transaction event handler (ElasticSearchEventHandler) according to the configuration file of Neo4j image database and registering the transaction event handler in Neo4j service.
[0067] The process of constructing a transaction event processor specifically includes: configuring a cluster of search servers (a plurality of nodes are separated by commas), configuring index name of search server (synchronized image data is saved in this index), configuring number of index shards, configuring number of index copies, configuring propagation mode of Elasticsearch, configuring whether to synchronize node data, whether to synchronize relational data, and whether to configure to synchronize data in an asynchronous manner, etc.
[0068] After search server client and transaction event processor are created, all data packaging and synchronization actions are performed in transaction event processor. The method of using transaction event processor to construct image data node and image data relationship into document request object of search server and storing in Map, comprising:
[0069] Creating a Map object in the transaction event processor, and the Map object adopts a key-value set mapping method;
[0070] Respectively constructing image data node and image data relationship in Neo4j image database Date Recue/Date Received 2021-12-24 into document request objects of search server;
[0071] Storing object constructed by entity id and index type of the image data node as the key to Map object, and/or
[0072] Storing the object constructed by entity id and index type of the image data relationship as the key to Map object;
[0073] Storing the document request object as value corresponding to Map object.
[0074] It is possible that a certain Neo4j transaction event processor only processes image data nodes or only image data relationships, accordingly, it is only necessary to store object constructed by entity id and index type of image data node as a Map object, or storing object constructed by entity id of image data relationship and index type as key of the Map object, and then storing the document request object as the value corresponding to the key, so as to avoid establishing too many key values, causing resource waste and improving index complexity, etc.
[0075] Since the data structure of image data includes image data nodes and image data relationships, it is necessary to construct image data nodes and image data relationships into document request objects of search server respectively. In the actual process of synchronizing image data, first constructing image data nodes in Neo4j image database into the document request objects of search server. The specific method includes:
[0076] Determining whether image data nodes need to be synchronized according to Neo4j image database configuration file, if image data nodes need to be synchronized, perform one or more of following operations to synchronize image data nodes:
Date Recue/Date Received 2021-12-24
Date Recue/Date Received 2021-12-24
[0077] Constructing the image data nodes created in the transaction object of Neo4j image database into index request document objects of search server;
[0078] Constructing the image data nodes deleted from the transaction object of Neo4j image database into delete request document object of search server;
[0079] Constructing newly-labeled image data nodes in the transaction object of Neo4j image database into index request document object of search server;
[0080] Constructing the image data nodes with delete label in the transaction object of Neo4j image database into delete request document object of search server;
[0081] Constructing the image data nodes with newly added attribute in the transaction object of Neo4j image database into index request document object of search server;
[0082] Constructing the image data nodes with delete attribute in the transaction object of Neo4j image database into the delete request document object of search server.
[0083] After image data node synchronization is completed, the image data relationships that need to be synchronized in the Neo4j image database are respectively constructed into document request objects of search server. The specific method includes:
[0084] Determining whether the image data relationships need to be synchronized according to Neo4j image database the configuration file, if image relationship nodes need to be synchronized, perform one or more of the following operations to synchronize image data relationships:
[0085] Constructing the image data relationships created in the transaction object of Neo4j image database Date Recue/Date Received 2021-12-24 into index request document objects of search server;
[0086] Constructing the image data relationships deleted from the transaction object of Neo4j image database into delete request document objects of search server;
[0087] Constructing newly-labeled image data relationships in the transaction object of Neo4j image database into index request document objects of search server;
[0088] Constructing the image data relationships with delete attribute in the transaction object of Neo4j image database into the delete request document objects of search server.
[0089] In specific implementation process, the image data relationships that need to be synchronized in Neo4j image database can also be constructed as the document request objects of search server, and then the image data nodes in Neo4j image database can be separately constructed as document request objects of search server, the requested document objects and indexes are saved in Map object in real time, and the corresponding value is returned according to key value index when it needs to be called, which is convenient, fast and not easy to make mistakes.
[0090] After obtaining document request objects, using the transaction event processor to construct a batch of document request objects of search server based on the document request object, the method of using the search server client to synchronize a batch of document request objects to search server, comprising:
[0091] Using the transaction event processor to construct a batch if document request objects of search server based on the document request object;
[0092] Search server client uses an asynchronous request mode or a synchronous request mode to perform image data synchronization operations and synchronizes a batch of document request objects to search Date Recue/Date Received 2021-12-24 server cluster.
[0093] Finally, according to Neo4j configuration file, it is determined that search server client adopts an asynchronous request mode or a synchronous request mode to perform image data synchronization operation. A batch of document request objects are Bulk request objects, the asynchronous request mode is an asynchronous Bulk request mode, and the synchronous request mode is synchronous Bulk request method. For example, according to Neo4j configuration file to determine whether to use asynchronous request method, if yes, the search server client uses asynchronous Bulk request method to synchronize a batch of document request objects to search server cluster, if not, search server client uses synchronous Bulk request to synchronize a batch of document request objects to search server cluster. In specific implementation, the Setting<T> class provided by Neo4j is used to read the required configuration values from neo4j.conf file according to configuration item name. The method for synchronizing data in Neo4j to a search server provided by the present invention solves the problem from the field of graph visualization applications, when Neo4j and search server are used to construct image search service, the image data cannot be automatically synchronized to the search server cluster in real time and help development people to use Neo4j and search server to build high-quality graph visualization applications.
[0094] Implementation two
[0095] Please refer to Figure 4, the present implementation provides a system of synchronizing data in Neo4j to a search server, the system comprises an initialization module, a mapping module and a synchronizing module, wherein, the initialization module configured to start Neo4j image database, create a search sever client, constructing a transaction event processor, and register the transaction event processor to Neo4j service; the mapping module configured to use the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then store in Map; the synchronizing module configured to use the transaction event processor to construct a batch of document request objects of the search server based on the document request object and use search Date Recue/Date Received 2021-12-24 server client to synchronize a batch of document request objects to search server cluster. The system of synchronizing data in Neo4j to search server can be integrated into the Neo4j image database in form of a plug-in, so that when Neo4j performs a writing operation, it can automatically convert the changed image data after the submitted data is written into the transaction into a batch of document request objects, and then use the search server client to synchronize a batch of document request objects to search server cluster.
[0096] The system of synchronizing data in Neo4j to the search server provided by the present invention adopts the method of synchronizing data in Neo4j to search server in the implementation one above, which solves the problem of how to synchronize the image data in Neo4j to search server in real time. Compared with the prior art, the beneficial effects of the system of synchronizing data in Neo4j to search server provided by the implementation of the present invention are the same as the beneficial effects of the method of synchronizing data in Neo4j to search server provided in the implementation one, in addition, other technical features in the system are the same as those disclosed in the method of the previous implementations and will not be repeated here.
[0097] In the description of the foregoing implementations, specific features, structures, materials, or characteristics can be combined in any one or more implementations or examples in a suitable manner.
[0098] The above-mentioned are only specific implementations of the present invention, but the protection scope of the present invention is not limited thereto. Any skilled in the art can easily think of changes or changes within the technical scope disclosed by the present invention, all replacements shall be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.
Date Recue/Date Received 2021-12-24
Date Recue/Date Received 2021-12-24
Claims (10)
1. A method of synchronizing data in Neo4j to search server, wherein, the method comprises:
starting Neo4j image database, creating a search sever client, constructing a transaction event processor, and registering the transaction event processor to Neo4j service;
using the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map; and using the transaction event processor to construct a batch of document request objects of the search server based on the document request object, using search server client to synchronize a batch of document request objects to search server cluster.
starting Neo4j image database, creating a search sever client, constructing a transaction event processor, and registering the transaction event processor to Neo4j service;
using the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map; and using the transaction event processor to construct a batch of document request objects of the search server based on the document request object, using search server client to synchronize a batch of document request objects to search server cluster.
2. The method of synchronizing data in Neo4j to search server according to claim 1, wherein, the search server adopts Elasticsearch search server.
3. The method of synchronizing data in Neo4j to a search server according to claim 1 or 2, wherein, creating a search server client, constructing a transaction event handler, and registering the transaction event handler to the Neo4j service, the method comprises:
starting Neo4j image database, reading search server cluster address and search server index name in Neo4j image database configuration file;
creating a search server client according to the search server cluster address;
using the search server client to check whether the search server index name in the Neo4j image database configuration file exists, if not exists, creating search server index name; and Date Recue/Date Received 2021-12-24 constructing a transaction event processor according to the Neo4j image database configuration file and registering the transaction event processor to Neo4j service.
starting Neo4j image database, reading search server cluster address and search server index name in Neo4j image database configuration file;
creating a search server client according to the search server cluster address;
using the search server client to check whether the search server index name in the Neo4j image database configuration file exists, if not exists, creating search server index name; and Date Recue/Date Received 2021-12-24 constructing a transaction event processor according to the Neo4j image database configuration file and registering the transaction event processor to Neo4j service.
4. The method of synchronizing data in Neo4j to a search server according to claim 3, using the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then storing in Map, the method comprises:
creating a Map object in the transaction event processor, and the Map object adopts a key-value set mapping method;
respectively constructing image data node and image data relationship in Neo4j image database into document request objects of search server;
storing object constructed by entity id and index type of the image data node as the key to Map object, and/or storing the object constructed by entity id and index type of the image data relationship as the key to Map object; and storing the document request object as value corresponding to Map object.
creating a Map object in the transaction event processor, and the Map object adopts a key-value set mapping method;
respectively constructing image data node and image data relationship in Neo4j image database into document request objects of search server;
storing object constructed by entity id and index type of the image data node as the key to Map object, and/or storing the object constructed by entity id and index type of the image data relationship as the key to Map object; and storing the document request object as value corresponding to Map object.
5. The method of synchronizing data in Neo4j to a search server according to claim 4, wherein the method of separately constructing image data nodes in Neo4j image database into document request objects of search server, comprising:
determining whether image data nodes need to be synchronized according to Neo4j image database Date Recue/Date Received 2021-12-24 configuration file, if image data nodes need to be synchronized, perform one or more of following operations to synchronize image data nodes:
constructing the image data nodes created in the transaction object of Neo4j image database into index request document objects of search server;
constructing the image data nodes deleted from the transaction object of Neo4j image database into delete request document object of search server;
constructing newly-labeled image data nodes in the transaction object of Neo4j image database into index request document object of search server;
constructing the image data nodes with delete label in the transaction object of Neo4j image database into delete request document object of search server;
constructing the image data nodes with newly added attribute in the transaction object of Neo4j image database into index request document object of search server; and constructing the image data nodes with delete attribute in the transaction object of Neo4j image database into the delete request document object of search server.
determining whether image data nodes need to be synchronized according to Neo4j image database Date Recue/Date Received 2021-12-24 configuration file, if image data nodes need to be synchronized, perform one or more of following operations to synchronize image data nodes:
constructing the image data nodes created in the transaction object of Neo4j image database into index request document objects of search server;
constructing the image data nodes deleted from the transaction object of Neo4j image database into delete request document object of search server;
constructing newly-labeled image data nodes in the transaction object of Neo4j image database into index request document object of search server;
constructing the image data nodes with delete label in the transaction object of Neo4j image database into delete request document object of search server;
constructing the image data nodes with newly added attribute in the transaction object of Neo4j image database into index request document object of search server; and constructing the image data nodes with delete attribute in the transaction object of Neo4j image database into the delete request document object of search server.
6. The method of synchronizing data in Neo4j to a search server according to claim 4, the method of separately constructing image data relationships, wherein the image data relationships need to be synchronized in Neo4j image database into document request objects of search server, comprising:
determining whether the image data relationships need to be synchronized according to Neo4j image database the configuration file, if image relationship nodes need to be synchronized, perform Date Recue/Date Received 2021-12-24 one or more of the following operations to synchronize image data relationships:
constructing the image data relationships created in the transaction object of Neo4j image database into index request document objects of search server;
constructing the image data relationships deleted from the transaction object of Neo4j image database into delete request document objects of search server;
constructing newly-labeled image data relationships in the transaction object of Neo4j image database into index request document objects of search server; and constructing the image data relationships with delete attribute in the transaction object of Neo4j image database into the delete request document objects of search server.
determining whether the image data relationships need to be synchronized according to Neo4j image database the configuration file, if image relationship nodes need to be synchronized, perform Date Recue/Date Received 2021-12-24 one or more of the following operations to synchronize image data relationships:
constructing the image data relationships created in the transaction object of Neo4j image database into index request document objects of search server;
constructing the image data relationships deleted from the transaction object of Neo4j image database into delete request document objects of search server;
constructing newly-labeled image data relationships in the transaction object of Neo4j image database into index request document objects of search server; and constructing the image data relationships with delete attribute in the transaction object of Neo4j image database into the delete request document objects of search server.
7. The method of synchronizing data in Neo4j to a search server according to claims 4-6, wherein using the transaction event processor to construct a batch of document request objects of search server based on the document request object, the method of using the search server client to synchronize a batch of document request objects to search server, comprising:
using the transaction event processor to construct a batch if document request objects of search server based on the document request object; and search server client uses an asynchronous request mode or a synchronous request mode to perform image data synchronization operations and synchronizes a batch of document request objects to search server cluster.
Date Recue/Date Received 2021-12-24
using the transaction event processor to construct a batch if document request objects of search server based on the document request object; and search server client uses an asynchronous request mode or a synchronous request mode to perform image data synchronization operations and synchronizes a batch of document request objects to search server cluster.
Date Recue/Date Received 2021-12-24
8. The method of synchronizing data in Neo4j to a search server according to claims 4-6, according to Neo4j's configuration file, determining of search server client adopts asynchronous request mode or synchronous request mode to perform image data synchronization operations.
9. The method of synchronizing data in Neo4j to a search server according to claim 8, wherein, a batch of document request objects are Bulk request objects, the asynchronous request mode is an asynchronous Bulk request mode, and the synchronous request mode is synchronous Bulk request method.
10. A system of synchronizing data in Neo4j to a search server, the system comprises an initialization module, a mapping module and a synchronizing module, wherein, The initialization module configured to start Neo4j image database, create a search sever client, constructing a transaction event processor, and register the transaction event processor to Neo4j service;
The mapping module configured to use the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then store in Map; and The synchronizing module configured to use the transaction event processor to construct a batch of document request objects of the search server based on the document request object and use search server client to synchronize a batch of document request objects to search server cluster.
Date Recue/Date Received 2021-12-24
The mapping module configured to use the transaction event processor to respectively construct image data node and image data relationship into document request object of search server, then store in Map; and The synchronizing module configured to use the transaction event processor to construct a batch of document request objects of the search server based on the document request object and use search server client to synchronize a batch of document request objects to search server cluster.
Date Recue/Date Received 2021-12-24
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011591917.5 | 2020-12-29 | ||
| CN202011591917.5A CN112632294B (en) | 2020-12-29 | 2020-12-29 | Method and system for synchronizing Neo4j data to search server |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA3144046A1 true CA3144046A1 (en) | 2022-06-29 |
Family
ID=75286150
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA3144046A Pending CA3144046A1 (en) | 2020-12-29 | 2021-12-24 | System and method of synchronizing data in neo4j to search server |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112632294B (en) |
| CA (1) | CA3144046A1 (en) |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20070021914A1 (en) * | 2005-07-20 | 2007-01-25 | Song Kyeong Sub | Navigation system with function of destination search using synchronization service of home telematics system, and method thereof |
| CN108170847B (en) * | 2018-01-18 | 2021-08-31 | 国网福建省电力有限公司 | Big data storage method based on Neo4j graph database |
| CN110990516B (en) * | 2019-12-13 | 2023-06-27 | 中科图新(苏州)科技有限公司 | Map data processing method, device and server |
-
2020
- 2020-12-29 CN CN202011591917.5A patent/CN112632294B/en active Active
-
2021
- 2021-12-24 CA CA3144046A patent/CA3144046A1/en active Pending
Also Published As
| Publication number | Publication date |
|---|---|
| CN112632294A (en) | 2021-04-09 |
| CN112632294B (en) | 2022-09-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109800222B (en) | HBase secondary index self-adaptive optimization method and system | |
| CN111427966B (en) | Database transaction processing method and device and server | |
| CN110196871B (en) | Data warehousing method and system | |
| CN112463311B (en) | Transaction processing method and device, computer equipment and storage medium | |
| US10191932B2 (en) | Dependency-aware transaction batching for data replication | |
| US8924365B2 (en) | System and method for range search over distributive storage systems | |
| EP2874077B1 (en) | Stateless database cache | |
| US20230161758A1 (en) | Distributed Database System and Data Processing Method | |
| US20220004542A1 (en) | Method and apparatus for updating database by using two-phase commit distributed transaction | |
| US11599514B1 (en) | Transactional version sets | |
| CN111797121A (en) | Strong consistency query method, device and system for read-write separation architecture service system | |
| CN111324610A (en) | Data synchronization method and device | |
| CN115668141A (en) | Distributed processing of transactions in a network using timestamps | |
| CN112416991A (en) | Data processing method and device and storage medium | |
| JP2022160666A (en) | Global secondary index method and device for distributed database | |
| CN110955719B (en) | Data access processing equipment, system and method | |
| US11709809B1 (en) | Tree-based approach for transactionally consistent version sets | |
| CN111414356A (en) | Data storage method and device, non-relational database system and storage medium | |
| US11522975B2 (en) | Double persistence layer using an in-memory map | |
| CN112800060A (en) | Data processing method and device, computer readable storage medium and electronic equipment | |
| WO2020192663A1 (en) | Data management method and related device | |
| CA3144046A1 (en) | System and method of synchronizing data in neo4j to search server | |
| US20110191549A1 (en) | Data Array Manipulation | |
| CN115098537B (en) | Transaction execution method and device, computing equipment and storage medium | |
| CN115113989B (en) | Transaction execution method, device, computing equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |
|
| EEER | Examination request |
Effective date: 20220916 |