CN111274410A - Data storage method and device and data query method and device - Google Patents
Data storage method and device and data query method and device Download PDFInfo
- Publication number
- CN111274410A CN111274410A CN202010070748.4A CN202010070748A CN111274410A CN 111274410 A CN111274410 A CN 111274410A CN 202010070748 A CN202010070748 A CN 202010070748A CN 111274410 A CN111274410 A CN 111274410A
- Authority
- CN
- China
- Prior art keywords
- data
- stored
- attribute data
- source information
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000013500 data storage Methods 0.000 title claims abstract description 18
- 230000008030 elimination Effects 0.000 claims description 2
- 238000003379 elimination reaction Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000004590 computer program Methods 0.000 description 1
- 230000008676 import Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 125000006850 spacer group Chemical group 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/36—Creation of semantic tools, e.g. ontology or thesauri
- G06F16/367—Ontology
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/31—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/3331—Query processing
- G06F16/334—Query execution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/33—Querying
- G06F16/338—Presentation of query results
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/30—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
- G06F16/38—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computational Linguistics (AREA)
- Library & Information Science (AREA)
- Software Systems (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
A data storage method and device and a data query method and device comprise the following steps: acquiring attribute data to be stored and data source information of the attribute data to be stored; and correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in a knowledge graph. Due to the fact that the attribute data to be stored and the data source information of the attribute data to be stored are obtained and correspondingly stored in the knowledge graph, the attribute data in the knowledge graph can display the source information, a user can conduct operation on the data source information according to the attribute data, and user experience is improved.
Description
Technical Field
The present disclosure relates to data processing technologies, and in particular, to a data storage method and apparatus and a data query method and apparatus.
Background
Knowledge-graph is a common means to describe the relationship between things, where a more common model is called an attribute graph. In the attribute map, entities and relations in the knowledge graph are mainly included, or may be also referred to as points and edges, and are identified by a unique primary key field, and besides the primary key field, the entities and relations also have a plurality of attributes.
In the related art, the data storage of the knowledge graph only stores the attribute data in the knowledge graph.
However, the data storage method cannot reflect the source of the attribute data in the knowledge graph, so that a user cannot perform operations on the attribute data related to data source information, and the user experience is reduced.
Disclosure of Invention
The application provides a data storage method and device and a data query method and device, which can enable attribute data in a knowledge graph to display source information, so that a user can carry out operation on the source information of the data aiming at the attribute data, and user experience is improved.
The application provides a data storage method, which comprises the following steps:
acquiring attribute data to be stored and data source information of the attribute data to be stored;
and correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in a knowledge graph.
When the attributes of the attribute data to be stored are the same and are single values, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in a knowledge graph, including:
when the attribute data to be stored are the same and the data source information of the attribute data to be stored is the same, judging whether a primary key corresponding to the attribute of the same attribute data exists in the knowledge graph or not to obtain a first judgment result;
and correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first judgment result.
Correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first judgment result, wherein the method comprises the following steps:
when the first judgment result shows that the corresponding key of the attribute of the same attribute data to be stored exists in the knowledge graph, replacing the attribute data and the data source information of the attribute data to be stored with the data source information of the same attribute data to be stored in the knowledge graph, and storing the attribute data and the data source information of the attribute data to be stored correspondingly;
and when the judgment result shows that the corresponding primary key of the attribute of the same attribute data to be stored does not exist in the knowledge graph, correspondingly storing the data source information of the same attribute data to be stored and the same attribute data to be stored in the knowledge graph.
When the attribute of the attribute data to be stored is multiple-valued, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in a knowledge graph, including:
carrying out duplication removal on the attribute data to be stored and the data source information of the attribute data to be stored;
judging whether the primary key corresponding to the attribute of the attribute data to be stored after the duplication elimination processing exists in the knowledge graph or not to obtain a second judgment result;
and correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to the second judgment result.
Correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to the second judgment result, wherein the method comprises the following steps:
when the main key corresponding to the attribute of the attribute data to be stored exists in the knowledge graph, combining the attribute data to be stored and the data source information with the attribute data and the data source information of the attribute data to be stored in the knowledge graph, and correspondingly storing the attribute data and the data source information in the knowledge graph;
and when the primary key corresponding to the attribute of the attribute data to be stored does not exist in the knowledge graph, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
The data types of the attribute data to be stored comprise: bulk data and streaming data;
and when the data type of the attribute data to be stored is the streaming data, processing the attribute data to be stored as a plurality of micro-batches.
When the attribute data to be stored has data source information of a plurality of interdependence levels, the data source information of the attribute data to be stored is the data source information of the minimum level.
The application also provides a data query method, which comprises the following steps:
acquiring a data query request aiming at a knowledge graph; the data query request is used for querying target attribute data and data source information of the target attribute data;
inquiring the data source information of the target attribute data and the target data in the knowledge graph according to the data inquiry request; the target attribute data and the data source information of the target attribute data are stored in a knowledge graph in advance correspondingly;
and returning the inquired target attribute data and the data source information of the target attribute data to a requester sending the data inquiry request.
The present application also provides a data storage device comprising:
the device comprises a first acquisition module, a second acquisition module and a storage module, wherein the first acquisition module is used for acquiring attribute data to be stored and data source information of the attribute data to be stored;
and the first processing module is used for correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
The present application also provides a data query apparatus, including:
the second acquisition module is used for acquiring a data query request aiming at the knowledge graph; the data query request is used for querying target attribute data and data source information of the target attribute data;
the second processing module is used for inquiring the data source information of the target attribute data and the target data in the knowledge graph according to the data inquiry request; the target attribute data and the data source information of the target attribute data are stored in a knowledge graph in advance correspondingly;
the second processing module is further configured to return the queried target attribute data and data source information of the target attribute data to a requester that sends the data query request.
Compared with the related art, the method comprises the following steps: acquiring attribute data to be stored and data source information of the attribute data to be stored; and correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in a knowledge graph. Due to the fact that the attribute data to be stored and the data source information of the attribute data to be stored are obtained and correspondingly stored in the knowledge graph, the attribute data in the knowledge graph can display the source information, a user can conduct operation on the data source information according to the attribute data, and user experience is improved.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by the practice of the application. The objectives and other advantages of the application may be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.
Drawings
The accompanying drawings are included to provide an understanding of the present disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the examples serve to explain the principles of the disclosure and not to limit the disclosure.
Fig. 1 is a schematic flowchart of a data storage method according to an embodiment of the present application;
fig. 2 is a schematic diagram of attribute data to be stored and data source information of the attribute data to be stored according to an embodiment of the present application;
fig. 3 is a schematic diagram of another attribute data to be stored and data source information of the attribute data to be stored according to an embodiment of the present application;
fig. 4 is a schematic flowchart of a data query method according to an embodiment of the present application;
FIG. 5 is a schematic structural diagram of a data storage device according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of a data query device according to an embodiment of the present application.
Detailed Description
The present application describes embodiments, but the description is illustrative rather than limiting and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or instead of any other feature or element in any other embodiment, unless expressly limited otherwise.
The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements disclosed in this application may also be combined with any conventional features or elements to form a unique inventive concept as defined by the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive aspects to form yet another unique inventive aspect, as defined by the claims. Thus, it should be understood that any of the features shown and/or discussed in this application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not limited except as by the appended claims and their equivalents. Furthermore, various modifications and changes may be made within the scope of the appended claims.
Further, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other orders of steps are possible as will be understood by those of ordinary skill in the art. Therefore, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. Further, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.
An embodiment of the present application provides a data storage method, as shown in fig. 1, including:
And 102, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
In an exemplary example, a single attribute data to be stored and data source information of the attribute data to be stored may be as shown in fig. 2, and a plurality of attribute data to be stored and data source information of the attribute data to be stored may be as shown in fig. 3, wherein the attribute data to be stored and the data source information of the attribute data to be stored may be separated by a spacer.
In an exemplary embodiment, when a plurality of attribute data to be stored belong to the same attribute and are single values, correspondingly storing the attribute data to be stored and data source information of the attribute data to be stored in a knowledge graph, including:
firstly, when a plurality of attribute data to be stored are the same and data source information of the plurality of attribute data to be stored is the same, judging whether a primary key corresponding to the attribute of the same attribute data exists in a knowledge graph or not, and obtaining a first judgment result.
And secondly, correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first judgment result.
In an exemplary embodiment, correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first determination result includes:
firstly, when the first judgment result shows that the corresponding key of the attribute of the same attribute data to be stored exists in the knowledge graph, replacing the attribute data and the data source information of the attribute data to be stored with the data source information of the same attribute data to be stored in the knowledge graph, and storing the attribute data and the data source information correspondingly.
And secondly, when the judgment result shows that the main key corresponding to the attribute of the same attribute data to be stored does not exist in the knowledge graph, correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph.
In an exemplary embodiment, when attributes to which a plurality of attribute data to be stored belong are multiple values, correspondingly storing the attribute data to be stored and data source information of the attribute data to be stored in a knowledge graph, the storing includes:
firstly, carrying out duplicate removal on a plurality of attribute data to be stored and data source information of the attribute data to be stored.
And secondly, judging whether the primary key corresponding to the attribute of the attribute data to be stored after the deduplication processing exists in the knowledge graph or not, and obtaining a second judgment result.
And finally, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to a second judgment result.
In an exemplary embodiment, the correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to the second determination result includes:
firstly, when a main key corresponding to the attribute of attribute data to be stored exists in a knowledge graph, combining the attribute data to be stored and data source information with the attribute data and the data source information of the attribute data to be stored in the knowledge graph, and correspondingly storing the attribute data and the data source information in the knowledge graph.
And secondly, when the primary key corresponding to the attribute of the attribute data to be stored does not exist in the knowledge graph, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
In one illustrative example, the data types of attribute data to be stored include: batch data and streaming data.
In an exemplary embodiment, when the data type of the attribute data to be stored is streaming data, the attribute data to be stored is processed as a plurality of micro-batches.
Many times, it is necessary to store not only bulk data in the form of files, but also streaming data in the form of elementary units Topic of data write operations of the open source streaming platform Kafka. However, it should be noted that when streaming data is introduced, streams are handled as micro-batches one by one.
In an exemplary example, when there are multiple interdependence levels of data source information in the attribute data to be stored, the data source information of the attribute data to be stored is the minimum level of data source information.
In an exemplary example, if the attribute data to be stored contains data source information, the data source information is directly obtained, but if the attribute data to be stored does not contain data source information, only the data source information of the batch level is available, that is, the data source information of each attribute data in the attribute data of the batch is the same, and the data source information is injected into each attribute data.
In an exemplary embodiment, the attribute data to be stored is all attribute data in a folder, and each subfolder in the folder corresponds to different data source information. In this case, the data source corresponding to each subfolder is stored in the dictionary table by establishing the dictionary table in advance, and then the data source information corresponding to the subfolder is injected into each piece of attribute data in the subfolders when the attribute data is stored.
In an exemplary example, for streaming import, there are similar scenarios, and data from different data sources may be written into different Kafka topics, in this case, data source information corresponding to each Kafka Topic may also be stored in a dictionary table by building the dictionary table in advance, and then when storing attribute data, the data source corresponding to the Kafka Topic is injected into each attribute data in the Kafka Topic.
According to the data storage method provided by the embodiment of the application, the attribute data to be stored and the data source information of the attribute data to be stored are obtained, and the attribute data to be stored and the data source information of the attribute data to be stored are correspondingly stored in the knowledge graph, so that the attribute data in the knowledge graph realize the display of the source information, a user can carry out operation related to the data source information aiming at the attribute data, and the user experience is improved.
An embodiment of the present application provides a data query method, as shown in fig. 4, including:
In an exemplary embodiment, the data source information may be classified in advance, after the data source information of the target attribute data and the target attribute data is obtained by querying, the classification of the data source information may be obtained, and the data source information of the target attribute data and the target attribute data with high classification may be returned to the requester sending the data query request
According to the data query method provided by the embodiment of the application, the data source information of the attribute data and the target attribute data is stored in the knowledge graph correspondingly in advance, so that when the data query request is obtained, the attribute data can be queried, the data source information of the attribute data can also be queried, the display of the source information of the attribute data is realized, a user can perform operation related to the data source information aiming at the attribute data, and the user experience is improved.
The embodiment of the present application also provides a data storage device, as shown in fig. 5, the data storage device 3 includes:
the first obtaining module 31 is configured to obtain attribute data to be stored and data source information of the attribute data to be stored.
The first processing module 32 is configured to correspondingly store the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
In an exemplary embodiment, when the attributes of the attribute data to be stored are the same and are single values, the first processing module 32 is specifically configured to:
when the attribute data to be stored are the same and the data source information of the attribute data to be stored is the same, judging whether the primary key corresponding to the attribute to which the same attribute data belongs exists in the knowledge graph or not, and obtaining a first judgment result.
And correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first judgment result.
In an exemplary embodiment, the first processing module 32 is further specifically configured to:
and when the first judgment result shows that the corresponding primary key of the attribute of the same attribute data to be stored exists in the knowledge graph, replacing the attribute data and the data source information of the attribute data to be stored with the data source information of the same attribute data to be stored in the knowledge graph, and storing the attribute data and the data source information of the attribute data to be stored correspondingly.
And when the judgment result shows that the corresponding primary key of the attribute of the same attribute data to be stored does not exist in the knowledge graph, correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph.
In an exemplary embodiment, when the attribute to which the attribute data to be stored belongs is a multiple value, the first processing module 32 is further specifically configured to:
and carrying out duplication removal on the plurality of attribute data to be stored and the data source information of the attribute data to be stored.
And judging whether the primary key corresponding to the attribute of the attribute data to be stored after the deduplication processing exists in the knowledge graph or not to obtain a second judgment result.
And correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to the second judgment result.
In an exemplary embodiment, the first processing module 32 is further specifically configured to:
when the main key corresponding to the attribute of the attribute data to be stored exists in the knowledge graph, combining the attribute data to be stored and the data source information with the attribute data and the data source information of the attribute data to be stored in the knowledge graph, and correspondingly storing the attribute data and the data source information in the knowledge graph.
And when the primary key corresponding to the attribute of the attribute data to be stored does not exist in the knowledge graph, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
In one illustrative example, the data types of attribute data to be stored include: batch data and streaming data.
In an exemplary embodiment, when the data type of the attribute data to be stored is streaming data, the attribute data to be stored is processed as a plurality of micro-batches.
In an exemplary example, when there are multiple interdependence levels of data source information in the attribute data to be stored, the data source information of the attribute data to be stored is the minimum level of data source information.
According to the data storage device provided by the embodiment of the application, the attribute data to be stored and the data source information of the attribute data to be stored are obtained, and the attribute data to be stored and the data source information of the attribute data to be stored are correspondingly stored in the knowledge graph, so that the attribute data in the knowledge graph realize the display of the source information, a user can carry out operation related to the data source information aiming at the attribute data, and the user experience is improved.
In practical applications, the first obtaining module 31 and the first Processing module 32 are implemented by a Central Processing Unit (CPU), a microprocessor Unit (MPU), a Digital Signal Processor (DSP), a Field Programmable Gate Array (FPGA), or the like, which are located in the data storage device.
An embodiment of the present application further provides a data query apparatus, as shown in fig. 6, where the data query apparatus 4 includes:
a second obtaining module 41, configured to obtain a data query request for the knowledge-graph; the data query request is used for querying the target attribute data and the data source information of the target attribute data.
The second processing module 42 is configured to query the target attribute data and the data source information of the target data in the knowledge graph according to the data query request; the target attribute data and the data source information of the target attribute data are stored in the knowledge graph in advance correspondingly.
The second processing module 42 is further configured to return the queried target attribute data and data source information of the target attribute data to the requester sending the data query request.
According to the data query device provided by the embodiment of the application, the data source information of the attribute data and the target attribute data is stored in the knowledge graph correspondingly in advance, so that when the data query request is obtained, the attribute data can be queried, the data source information of the attribute data can be queried, the display of the source information of the attribute data is realized, a user can perform operation related to the data source information aiming at the attribute data, and the user experience is improved.
In practical applications, the second obtaining module 41 and the second processing module 42 are implemented by a CPU, an MPU, a DSP or an FPGA located in the data query device.
An embodiment of the present application further provides a data processing apparatus, including: a processor and a memory, wherein the memory has stored therein a computer program which, when executed by the processor, implements the processing of the method as set forth in any one of the above.
An embodiment of the present application further provides a storage medium, where the storage medium stores computer-executable commands, and the computer-executable commands are used for executing the processing of any one of the methods described above.
It will be understood by those of ordinary skill in the art that all or some of the steps of the methods, systems, functional modules/units in the devices disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed by several physical components in cooperation. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as is well known to those of ordinary skill in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, Digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can accessed by a computer. In addition, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media as known to those skilled in the art.
Claims (10)
1. A method of storing data, comprising:
acquiring attribute data to be stored and data source information of the attribute data to be stored;
and correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in a knowledge graph.
2. The method according to claim 1, wherein when the attributes of the plurality of attribute data to be stored are the same and are single values, the correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph comprises:
when the attribute data to be stored are the same and the data source information of the attribute data to be stored is the same, judging whether a primary key corresponding to the attribute of the same attribute data exists in the knowledge graph or not to obtain a first judgment result;
and correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first judgment result.
3. The method according to claim 2, wherein correspondingly storing the same attribute data to be stored and the data source information of the same attribute data to be stored in the knowledge graph according to the first determination result comprises:
when the first judgment result shows that the corresponding key of the attribute of the same attribute data to be stored exists in the knowledge graph, replacing the attribute data and the data source information of the attribute data to be stored with the data source information of the same attribute data to be stored in the knowledge graph, and storing the attribute data and the data source information of the attribute data to be stored correspondingly;
and when the judgment result shows that the corresponding primary key of the attribute of the same attribute data to be stored does not exist in the knowledge graph, correspondingly storing the data source information of the same attribute data to be stored and the same attribute data to be stored in the knowledge graph.
4. The method according to claim 1, wherein when the attributes to which the plurality of attribute data to be stored belong are multi-valued, the storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph correspondingly comprises:
carrying out duplication removal on the attribute data to be stored and the data source information of the attribute data to be stored;
judging whether the primary key corresponding to the attribute of the attribute data to be stored after the duplication elimination processing exists in the knowledge graph or not to obtain a second judgment result;
and correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to the second judgment result.
5. The method according to claim 4, wherein the correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph according to the second determination result comprises:
when the main key corresponding to the attribute of the attribute data to be stored exists in the knowledge graph, combining the attribute data to be stored and the data source information with the attribute data and the data source information of the attribute data to be stored in the knowledge graph, and correspondingly storing the attribute data and the data source information in the knowledge graph;
and when the primary key corresponding to the attribute of the attribute data to be stored does not exist in the knowledge graph, correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
6. The method according to any one of claims 1 to 5, wherein the data type of the attribute data to be stored comprises: bulk data and streaming data;
and when the data type of the attribute data to be stored is the streaming data, processing the attribute data to be stored as a plurality of micro-batches.
7. The method according to any one of claims 1 to 5, wherein when there are multiple interdependency levels of data source information in the attribute data to be stored, the data source information of the attribute data to be stored is the minimum level of data source information.
8. A method for querying data, comprising:
acquiring a data query request aiming at a knowledge graph; the data query request is used for querying target attribute data and data source information of the target attribute data;
inquiring the data source information of the target attribute data and the target data in the knowledge graph according to the data inquiry request; the target attribute data and the data source information of the target attribute data are stored in a knowledge graph in advance correspondingly;
and returning the inquired target attribute data and the data source information of the target attribute data to a requester sending the data inquiry request.
9. A data storage device, comprising:
the device comprises a first acquisition module, a second acquisition module and a storage module, wherein the first acquisition module is used for acquiring attribute data to be stored and data source information of the attribute data to be stored;
and the first processing module is used for correspondingly storing the attribute data to be stored and the data source information of the attribute data to be stored in the knowledge graph.
10. A data query apparatus, comprising:
the second acquisition module is used for acquiring a data query request aiming at the knowledge graph; the data query request is used for querying target attribute data and data source information of the target attribute data;
the second processing module is used for inquiring the data source information of the target attribute data and the target data in the knowledge graph according to the data inquiry request; the target attribute data and the data source information of the target attribute data are stored in a knowledge graph in advance correspondingly;
the second processing module is further configured to return the queried target attribute data and data source information of the target attribute data to a requester that sends the data query request.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010070748.4A CN111274410A (en) | 2020-01-21 | 2020-01-21 | Data storage method and device and data query method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010070748.4A CN111274410A (en) | 2020-01-21 | 2020-01-21 | Data storage method and device and data query method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111274410A true CN111274410A (en) | 2020-06-12 |
Family
ID=71003354
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010070748.4A Pending CN111274410A (en) | 2020-01-21 | 2020-01-21 | Data storage method and device and data query method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111274410A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112163485A (en) * | 2020-09-18 | 2021-01-01 | 杭州海康威视系统技术有限公司 | Data processing method and device, database system and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170337260A1 (en) * | 2015-02-13 | 2017-11-23 | Guangzhou Shenma Mobile Information Technology Co. Ltd. | Method and device for storing data |
| CN109902130A (en) * | 2019-01-31 | 2019-06-18 | 北京明略软件系统有限公司 | A kind of date storage method, data query method and apparatus, storage medium |
| CN110019694A (en) * | 2017-07-26 | 2019-07-16 | 凡普互金有限公司 | Method, apparatus and computer readable storage medium for knowledge mapping |
| CN110222127A (en) * | 2019-06-06 | 2019-09-10 | 中国电子科技集团公司第二十八研究所 | The converging information method, apparatus and equipment of knowledge based map |
| CN110457502A (en) * | 2019-08-21 | 2019-11-15 | 京东方科技集团股份有限公司 | Construct knowledge mapping method, man-machine interaction method, electronic equipment and storage medium |
-
2020
- 2020-01-21 CN CN202010070748.4A patent/CN111274410A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20170337260A1 (en) * | 2015-02-13 | 2017-11-23 | Guangzhou Shenma Mobile Information Technology Co. Ltd. | Method and device for storing data |
| CN110019694A (en) * | 2017-07-26 | 2019-07-16 | 凡普互金有限公司 | Method, apparatus and computer readable storage medium for knowledge mapping |
| CN109902130A (en) * | 2019-01-31 | 2019-06-18 | 北京明略软件系统有限公司 | A kind of date storage method, data query method and apparatus, storage medium |
| CN110222127A (en) * | 2019-06-06 | 2019-09-10 | 中国电子科技集团公司第二十八研究所 | The converging information method, apparatus and equipment of knowledge based map |
| CN110457502A (en) * | 2019-08-21 | 2019-11-15 | 京东方科技集团股份有限公司 | Construct knowledge mapping method, man-machine interaction method, electronic equipment and storage medium |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112163485A (en) * | 2020-09-18 | 2021-01-01 | 杭州海康威视系统技术有限公司 | Data processing method and device, database system and electronic equipment |
| CN112163485B (en) * | 2020-09-18 | 2023-11-24 | 杭州海康威视系统技术有限公司 | Data processing method and device, database system and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102725755B (en) | Method and system of file access | |
| US20170031948A1 (en) | File synchronization method, server, and terminal | |
| KR102564170B1 (en) | Method and device for storing data object, and computer readable storage medium having a computer program using the same | |
| CN107704202B (en) | Method and device for quickly reading and writing data | |
| US10678779B2 (en) | Generating sub-indexes from an index to compress the index | |
| CN108572789B (en) | Disk storage method and device, message pushing method and device and electronic equipment | |
| CN108614837B (en) | File storage and retrieval method and device | |
| CN102799598A (en) | Data recovery method for deleting repeated data | |
| CN109240607B (en) | File reading method and device | |
| US9734171B2 (en) | Intelligent redistribution of data in a database | |
| CN109388644B (en) | Data updating method and device | |
| CN110647423B (en) | Method, device and readable medium for creating storage volume mirror image based on application | |
| CN104346347A (en) | Data storage method, device, server and system | |
| CN105765570A (en) | Music identification | |
| CN104615459A (en) | MoCA equipment parameter configuration method and device | |
| CN110889424B (en) | Vector index establishing method and device and vector retrieving method and device | |
| CN112035413B (en) | Metadata information query method, device and storage medium | |
| CN111274410A (en) | Data storage method and device and data query method and device | |
| US11853229B2 (en) | Method and apparatus for updating cached information, device, and medium | |
| US10872103B2 (en) | Relevance optimized representative content associated with a data storage system | |
| CN111858609A (en) | Fuzzy query method and device for block chain | |
| CN111666347B (en) | Data processing method, device and equipment | |
| CN109960695B (en) | Management method and device for database in cloud computing system | |
| CN113051301A (en) | Object storage method, system and equipment | |
| CN113312414B (en) | Data processing method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200612 |
|
| RJ01 | Rejection of invention patent application after publication |