CN113051441A

CN113051441A - Storage design and management method of entity object

Info

Publication number: CN113051441A
Application number: CN202110607738.4A
Authority: CN
Inventors: 赵鹏; 王海强; 杨晓冬; 姜青涛
Original assignee: Beijing Daoda Tianji Technology Co ltd
Current assignee: Beijing Daoda Tianji Technology Co ltd
Priority date: 2021-06-01
Filing date: 2021-06-01
Publication date: 2021-06-29

Abstract

The embodiment of the disclosure provides a storage design and management method, a storage design and management device, storage design and management equipment and a computer-readable storage medium for entity objects. The method comprises the steps of creating N entity objects according to business requirements; n is a positive integer greater than or equal to 2; storing the entity object into a graph database in a node form, and simultaneously storing the entity object into a full-text retrieval database in a document form; creating a related entity object according to the service requirement, and storing the related entity object in the graph database; the associated entity object is used for associating the entity object. In this way, the application requirements of most scenes in storage, management and analysis of mass data for enterprise system application can be met, and the system architecture of enterprise application can be simple, efficient and expandable.

Description

Storage design and management method of entity object

Technical Field

Embodiments of the present disclosure relate generally to the field of architecture design, and more particularly, to a method, an apparatus, a device, and a computer-readable storage medium for storage design and management of entity objects.

Background

At this stage, database systems commonly used by enterprises can be broadly divided into two broad categories, traditional relational databases (e.g., Mysql, Oracle, PostgreSQL, etc.) and non-relational databases (e.g., Mongodb, Elasticsearch, and Neo4 j).

For the traditional relational database, except for a single charging database (like oracle), most of the relational databases are poor in efficient reading and writing performance of mass data; a fixed table structure (schema mode) has poor flexibility, and especially for an enterprise with rapid business change, modification of the table structure of the system brings a large amount of workload and inconvenience; on high concurrent read and write, hard disk I/O is a great bottleneck; in complex association analysis and information query of data, the cartesian product of association table query can seriously reduce the query efficiency.

For the non-relational database, the transactional processing is relatively weak, a fixed SQL query language is not available, and the learning cost is relatively high.

In summary, common query requirements are currently developed for enterprise system applications, such as: in full-text retrieval, conditional query, spatial query based on geographic information and complex multi-level association query analysis, the conventional relational database and the non-relational database cannot meet the common query and analysis requirements at the same time.

Disclosure of Invention

According to an embodiment of the disclosure, a storage design and management scheme for entity objects is provided.

In a first aspect of the disclosure, a method for designing and managing storage of entity objects is provided. The method comprises the following steps:

creating N entity objects for the service requirement; n is a positive integer greater than or equal to 2;

storing the entity object into a graph database in a node form, and simultaneously storing the entity object into a full-text retrieval database in a document form;

creating a related entity object according to the service requirement, and storing the related entity object in the graph database; the associated entity object is used for associating the entity object.

Further, still include:

receiving retrieval information;

analyzing the retrieval information, and selecting a service query or analysis interface;

and inquiring corresponding retrieval information from the graph database or the full-text retrieval database through the service inquiry or analysis interface.

Further, the analyzing the retrieved information and selecting a service query or analysis interface includes:

analyzing the retrieval information through a route selector of query and analysis service, and selecting a query interface of a full-text retrieval database if the retrieval information is not complex incidence relation query;

and if the retrieval information is the query and analysis of the complex association relationship or the path, selecting a query and analysis interface of the graph database.

Further, the querying, through the service querying or analyzing interface, the corresponding retrieval information from the graph database or the full-text retrieval database includes:

if the service query or analysis interface is the query interface of the full-text retrieval database, calling a query API of the full-text retrieval database through a query service for querying;

if the business query or analysis interface is the query analysis interface of the graph database, the query analysis is carried out by calling a query API of the graph database through the analysis service.

Further, still include:

when the entity incidence relation is transactionally operated, the transactional operation is synchronously written into the graph database and the full-text retrieval database through SEATA distributed transaction service; the transactional operations include creation, addition, and/or deletion.

In a second aspect of the present disclosure, an apparatus for designing and managing storage of entity objects is provided. The device includes:

the creating module is used for creating N entity objects according to the service requirements; n is a positive integer greater than or equal to 2;

the storage module is used for storing the entity object into a graph database in a node form and storing the entity object into a full-text retrieval database in a document form;

the association module is used for creating an association entity object according to the service requirement and storing the association entity object into the graph database; the associated entity object is used for associating the entity object.

Further, still include:

the receiving module is used for receiving retrieval information;

the analysis module is used for analyzing the retrieval information and selecting a service query or analysis interface;

and the query module is used for querying corresponding retrieval information from the graph database or the full-text retrieval database through the service query or analysis interface.

In a third aspect of the disclosure, an electronic device is provided. The electronic device includes: a memory having a computer program stored thereon and a processor implementing the method as described above when executing the program.

In a fourth aspect of the present disclosure, a computer readable storage medium is provided, having stored thereon a computer program, which when executed by a processor, implements a method as in accordance with the first aspect of the present disclosure.

The method for storing and managing the entity objects provided by the embodiment of the application establishes N entity objects for the service requirements; n is a positive integer greater than or equal to 2; storing the entity object into a graph database in a node form, and simultaneously storing the entity object into a full-text retrieval database in a document form; creating a related entity object according to the service requirement, and storing the related entity object in the graph database; the associated entity object is used for associating the entity object, so that the application requirements of most scenes in storage, management and analysis of mass data for enterprise system application can be met, and the system architecture of the enterprise application is simple, efficient and extensible.

It should be understood that the statements herein reciting aspects are not intended to limit the critical or essential features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description.

Drawings

The above and other features, advantages and aspects of various embodiments of the present disclosure will become more apparent by referring to the following detailed description when taken in conjunction with the accompanying drawings. In the drawings, like or similar reference characters designate like or similar elements, and wherein:

FIG. 1 illustrates a flow diagram of a method for storage design and management of physical objects according to an embodiment of the present disclosure;

FIG. 2 illustrates a flow diagram of a method for storage design and management of physical objects according to yet another embodiment of the present disclosure;

FIG. 3 shows an architecture diagram according to an embodiment of the present disclosure;

FIG. 4 illustrates a block diagram of an apparatus for storage design and management of physical objects, in accordance with an embodiment of the present disclosure;

FIG. 5 illustrates a block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure.

Detailed Description

To make the objects, technical solutions and advantages of the embodiments of the present disclosure more clear, the technical solutions of the embodiments of the present disclosure will be described clearly and completely with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are some, but not all embodiments of the present disclosure. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

In addition, the term "and/or" herein is only one kind of association relationship describing an associated object, and means that there may be three kinds of relationships, for example, a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" herein generally indicates that the former and latter related objects are in an "or" relationship.

FIG. 1 shows a flow diagram of a method 100 for storage design and management of physical objects according to an embodiment of the disclosure. The method 100 comprises:

s110, creating N entity objects according to business requirements; n is a positive integer greater than or equal to 2.

In some embodiments, N entity objects are created according to the business needs of the user. For example, entity target users, test papers, scores and the like are created according to business requirements of the users, and generally N is a positive integer greater than or equal to 2.

And S120, storing the entity object into a graph database in a node form, and simultaneously storing the entity object into a full-text retrieval database in a document form.

In some embodiments, the graph database base is a graph-theory based database. Preferably, Neo4j is used as the graph database for the specific implementation.

In some embodiments, the solid objects are modeled and stored in the Neo4j database as nodes, i.e., the solid objects are stored in the Neo4j database as vertices (Vertex). The Neo4j database is mainly used for solving the query and analysis of multi-level association relations. Based on a graph traversal principle, quickly inquiring or analyzing the correlation of a certain node in a certain topological structure; and the associated path between a certain node and another node and the like, thereby greatly improving the query efficiency of querying the complex association relation and the path.

In some embodiments, the Elasticsearch database is selected as a full-text search database, and the entity object is stored in the Elasticsearch database in a document form, that is, the entity object is mapped to the Elasticsearch database. Based on the Elasticissearch database, services such as full-text retrieval, multi-condition query and geographic information-based spatial query can reach second-level query response; the form of the document may be set according to an application scene, such as XML, XHTML, and the like.

In some embodiments, both the Neo4j and the elastic search database can be used as storage databases for mass data, and since both are NoSQL databases, in order to ensure transaction consistency, for creating, updating, deleting, and the like, a SEATA distributed transaction framework is used to ensure synchronous writing of data into the Neo4j and the elastic search database. Transactional operations (create, modify, delete, etc.) and non-transactional operations are isolated by read-write separation.

It should be noted that the relevant data of any entity object in the Neo4j and elastic search database should be consistent all the time.

S130, creating a related entity object according to business requirements, and storing the related entity object in the graph database; the associated entity object is used for associating the entity object.

In some embodiments, an associated entity object is created according to the business requirements of the user, and the associated entity object is stored in the Neo4j database in the form of an Edge (Edge); the associated entity object is used for associating the entity object, namely associating a starting endpoint (entity object) and a terminating endpoint (entity object) involved by the associated entity object.

As shown in fig. 2, further, the method further includes:

s210, receiving the retrieval information.

And receiving retrieval information, such as full-text retrieval, multi-condition query, geographic space query, complex association relation query and/or path analysis query, input by a user.

S220, analyzing the retrieval information, and selecting a service query or analysis interface.

In some embodiments, the retrieved information is analyzed by a router of a query and analysis service,

if the retrieval information is a non-complex incidence relation query, selecting a query interface of a full-text retrieval database;

S220, corresponding retrieval information is queried from the graph database or the full-text retrieval database through the service query or analysis interface.

As shown in fig. 3, in some embodiments, if the service query or analysis interface is a query interface of the full-text search database, the query service calls a query API of the full-text search database to perform a query;

if the business query or analysis interface is the query analysis interface of the graph database, the query analysis is carried out by calling a query API of the graph database through an analysis service;

further, when transactional operation needs to be performed on the entity association relationship, the transactional operation is synchronously written into the graph database and the full-text retrieval database through SEATA distributed transaction service; the transactional operations include creation, addition, and/or deletion.

According to the embodiment of the disclosure, the following technical effects are achieved:

through the combination of Neo4j (graph database) and elastic search (full-text retrieval database), the application requirements of most scenes in storage, management and analysis of mass data are met, and the system architecture of enterprise application is simple, efficient and extensible.

Specifically, only entity objects are stored in the Elasticsearch, and the design of association relationship is not involved, so as to ensure that related entity document (entity object) data keeps sparse consistency of a data matrix in a bottom Lucene (full text search engine toolkit) full text search engine of the Elasticsearch, improve index search performance, reduce the calling of an association interface, and improve query efficiency;

the design of the association relation (the creation of the association entity object) is carried out in Neo4j, and the query result of the required association relation can be rapidly queried only through one-time query request through a Cypher query API (interface) of Neo4j, so that the query efficiency of complex association relation and path query is greatly improved.

It is noted that while for simplicity of explanation, the foregoing method embodiments have been described as a series of acts or combination of acts, it will be appreciated by those skilled in the art that the present disclosure is not limited by the order of acts, as some steps may, in accordance with the present disclosure, occur in other orders and concurrently. Further, those skilled in the art should also appreciate that the embodiments described in the specification are exemplary embodiments and that acts and modules referred to are not necessarily required by the disclosure.

The above is a description of embodiments of the method, and the embodiments of the apparatus are further described below.

Fig. 4 shows a block diagram of an apparatus 400 for storage design and management of physical objects according to an embodiment of the present disclosure. As shown in fig. 4, the apparatus 400 includes:

It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the described module may refer to the corresponding process in the foregoing method embodiment, and is not described herein again.

FIG. 5 shows a schematic block diagram of an electronic device 500 that may be used to implement embodiments of the present disclosure. As shown, device 500 includes a Central Processing Unit (CPU) 501 that may perform various appropriate actions and processes according to computer program instructions stored in a Read Only Memory (ROM) 502 or computer program instructions loaded from a storage unit 708 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The CPU 501, ROM 502, and RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.

A number of components in the device 500 are connected to the I/O interface 505, including: an input unit 506 such as a keyboard, a mouse, or the like; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508, such as a magnetic disk, optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices through a computer network such as the internet and/or various telecommunication networks.

The processing unit 501 performs the various methods and processes described above, such as the methods 100, 200. For example, in some embodiments, the methods 100, 200 may be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the CPU 501, one or more steps of the methods 100, 200 described above may be performed. Alternatively, in other embodiments, the CPU 501 may be configured to perform the methods 100, 200 by any other suitable means (e.g., by way of firmware).

The functions described herein above may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a load programmable logic device (CPLD), and the like.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions/operations specified in the flowchart and/or block diagram to be performed. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Further, while operations are depicted in a particular order, this should be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. Under certain circumstances, multitasking and parallel processing may be advantageous. Likewise, while several specific implementation details are included in the above discussion, these should not be construed as limitations on the scope of the disclosure. Certain features that are described in the context of separate embodiments can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable subcombination.

Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

Claims

1. A storage design and management method of entity objects is characterized by comprising the following steps:

creating N entity objects according to business requirements; n is a positive integer greater than or equal to 2;

2. The method of claim 1, further comprising:

receiving retrieval information;

3. The method of claim 2, wherein analyzing the retrieved information and selecting a service query or analysis interface comprises:

4. The method according to claim 3, wherein said querying, through said business query or analysis interface, said corresponding search information from said graph database or full text search database comprises:

5. The method of claim 4, further comprising:

6. An apparatus for designing and managing storage of physical objects, comprising:

7. The apparatus of claim 6, further comprising:

the receiving module is used for receiving retrieval information;

8. The apparatus of claim 7, wherein the analyzing the retrieved information and selecting a service query or analysis interface comprises:

9. An electronic device comprising a memory and a processor, the memory having stored thereon a computer program, wherein the processor, when executing the program, implements the method of any of claims 1-5.

10. A computer-readable storage medium, on which a computer program is stored, which program, when being executed by a processor, carries out the method according to any one of claims 1 to 5.