CN110275969A - A kind of data presentation technique in hundred billion grades of knowledge picture libraries - Google Patents
A kind of data presentation technique in hundred billion grades of knowledge picture libraries Download PDFInfo
- Publication number
- CN110275969A CN110275969A CN201910514448.8A CN201910514448A CN110275969A CN 110275969 A CN110275969 A CN 110275969A CN 201910514448 A CN201910514448 A CN 201910514448A CN 110275969 A CN110275969 A CN 110275969A
- Authority
- CN
- China
- Prior art keywords
- point
- data
- index
- custom attributes
- attribute
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 66
- 238000013507 mapping Methods 0.000 claims abstract description 56
- 238000006243 chemical reaction Methods 0.000 claims abstract description 22
- 238000013506 data mapping Methods 0.000 claims description 4
- 238000007689 inspection Methods 0.000 claims 1
- 230000009466 transformation Effects 0.000 abstract 1
- 238000010586 diagram Methods 0.000 description 7
- 238000013500 data storage Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 239000000284 extract Substances 0.000 description 2
- 239000000203 mixture Substances 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/583—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/587—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Library & Information Science (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses the data presentation techniques in a kind of hundred billion grades of knowledge picture libraries, including expressing for knowledge method in chart database, index representation method, Knowledge conversion representation method；Wherein expressing for knowledge method includes the data format representation of point and the data format representation on side in chart database, includes two parts: the storage format of the description explanation and data of data；Index representation method is made of association extension index, the full-text index of point and the full-text index on side；Knowledge conversion representation method describes initial data, and how Mapping and Converting is point, number of edges evidence.The present invention can express the knowledge of arbitrary structures, be capable of the retrieval of any dimension of flexibly customizing, and Knowledge conversion process does not have data dependence constraint, can promote the handling capacity of knowledge transformation calculating.
Description
Technical field
The present invention relates to the data expression sides in technical field of data storage, more particularly to a kind of hundred billion grades of knowledge picture libraries
Method.
Background technique
Chart database is the database based on figure relationship (Graph, non-image) model foundation, and data wherein included are often
There are point data and number of edges evidence.Such as the friend relation in social networks, wherein everyone to indicate, friend relation is with side table
Show.The point stored in picture library data and side generally require to support the extension of dynamic structure: the data structure of newly-increased type point
Extension, existing type point data structure adjustment；Newly-increased type while data structure extension, existing type while data
The adjustment of structure.
In chart database, in order to support the storage of hundred billion grades of knowledge, need to have a kind of data coding method, Neng Gouzhi
Support the Knowledge conversion for being associated with extension/attribute retrieval/full-text search, supporting high-throughput of flexible data definition, any dimension
Process, but there is presently no this technical solutions.
Summary of the invention
In view of the above drawbacks of the prior art, technical problem to be solved by the invention is to provide a kind of hundred billion grades of knowledge
Data presentation technique in picture library, so as to solve the deficiencies in the prior art.
To achieve the above object, the present invention provides the data presentation techniques in a kind of hundred billion grades of knowledge picture libraries, including figure
Expressing for knowledge method in database indexes representation method, Knowledge conversion representation method；Wherein expressing for knowledge in chart database
Method includes the data format representation of point and the data format representation on side, and include two parts: the description of data is said
Bright and data storage formats；Index representation method is made of association extension index, the full-text index of point and the full-text index on side；
Knowledge conversion representation method describes initial data, and how Mapping and Converting is point, number of edges evidence.
Further, include: in the data format representation of the point
The description of S11, point data illustrate, for the Custom Attributes of description point and the retrieval mode of Custom Attributes, include
In have: the type of point, the parent type of point, the Custom Attributes of point, the Custom Attributes type of point, point Custom Attributes
Retrieval mode；The Custom Attributes type at its midpoint include text, integer, lint-long integer, number, date-time, the date, when
Between, longitude and latitude, the retrieval mode of the Custom Attributes of point includes nothing, index in classification, attribute retrieval, full-text search；
The storage format of S12, point data put actual storage content in picture library, consist of two parts: build-in attribute and
Custom Attributes；Wherein build-in attribute includes: the unique number, the data source identification of point, the storage number of tracing to the source of point, point of point
The generation time of type, the label of point, point；Custom Attributes is the attribute flexibly defined according to different scenes；Wherein, point traces back
Source storage number is made of the data source identification of the unique number and point put.
Further, include: in the data format representation on the side
The description explanation of S21, number of edges evidence, describe the Custom Attributes on side and the retrieval mode of Custom Attributes, include
Inside have: while type, while parent type, while direction, the type of A endpoint, the type of B endpoint, while Custom Attributes, side
Custom Attributes type, side Custom Attributes retrieval mode；
The storage format of S22, number of edges evidence, while the actual storage content in picture library consists of two parts: build-in attribute and
Custom Attributes；Wherein build-in attribute include: while unique number, while data source identification, side trace to the source storage number, the end A
Point unique number, the unique number of B endpoint, while type, while label, side the generation time；Wherein, the storage of tracing to the source on side
Number by while unique number and while data source identification form.
Further, the association extension index, the full-text index of point and the full-text index on side specifically include:
S31, figure association extension index, to store the association expansion relation of figure；Consist of two parts: the expansion relation of point
With the aggregate list on side；Wherein:
Point expansion relation include: A endpoint unique number, while type, while date of occurrence, the unique number of B endpoint；
While aggregate list include 0 or multiple while connection identifier, list size be 0 indicate be a virtual presence
Side, list size be more than 0 expression physical presence side, it is each while connection identifier include: while time of origin, side resource mark
Know；
The full-text index of S32, point, to store search field index a little；It consists of three parts, inherently indexes, makes by oneself
Adopted field index and full-text index；Wherein:
Intrinsic index includes: the mark of type, point that the storage of tracing to the source of the unique number, resource identification, point of point is numbered, put
Label；
Custom Attributes field is according in the data description of point, the Custom Attributes type of point, the Custom Attributes of point
Retrieval mode setting；The retrieval mode of the Custom Attributes of point includes index in classification, attribute retrieval；
Full-text index, the retrieval mode setting of Custom Attributes type, the Custom Attributes of point according to point；Its midpoint
The retrieval mode of Custom Attributes is full-text search；
S33, side full-text index, to store side search field index；It consists of three parts, inherently indexes, makes by oneself
Adopted field index and full-text index；Wherein:
Intrinsic index includes: while unique number, resource identification, while storage number of tracing to the source, while type, while mark
Label, the unique number of A endpoint, the unique number of B endpoint；
Custom Attributes field according in the data description on side, while Custom Attributes type, while Custom Attributes
Retrieval mode setting；Wherein, the retrieval mode of the Custom Attributes on side includes index in classification, attribute retrieval；
Full-text index, according to while Custom Attributes type, while Custom Attributes retrieval mode setting；Wherein side
The retrieval mode of Custom Attributes is full-text search.
Further, the initial data Mapping and Converting is the method for point data specifically: includes one or more mappings
Unit, each map unit describe the mapping mode of a kind of point；Inside map unit, the type comprising point, point it is unique
Number mapping relations, the label mapping relationship of point, Custom Attributes mapping relations；The attribute that wherein mapping relations are given directions is by original
Those of data field composition, the attribute at midpoint are build-in attribute or Custom Attributes, and the field of initial data includes one
Or multiple fields.
Further, the initial data Mapping and Converting is the method for number of edges evidence specifically: includes one or more mappings
Unit, each map unit describe the mapping mode on a kind of side；Inside map unit, comprising while type, while it is unique
Number mapping relations, the mapping relations of the unique number of A endpoint, the unique number mapping relations of B endpoint, side label mapping close
System, Custom Attributes mapping relations；Wherein mapping relations refer to that the attribute on side is made of those of initial data field, midpoint
Attribute is build-in attribute or Custom Attributes, and the field of initial data includes one or more fields.
The beneficial effects of the present invention are:
1, using knowledge representation method, the knowledge of arbitrary structures can be expressed；
It 2, being capable of flexibly customizing times using association extension/attribute retrieval/full-text search method of any dimension of knowledge
The retrieval for dimension of anticipating, enables and is provided simultaneously with association extension/attribute retrieval/full-text search using the knowledge picture library of this method,
Even the association with attribute can also be supported to extend；
3, the method defined using Knowledge conversion, Knowledge conversion process do not have data dependence constraint, can promote knowledge and turn
Change the handling capacity calculated.
It is described further below with reference to technical effect of the attached drawing to design of the invention, specific structure and generation, with
It is fully understood from the purpose of the present invention, feature and effect.
Detailed description of the invention
Fig. 1 is data format schematic diagram a little；
Fig. 2 is the data format schematic diagram on side；
Fig. 3 is the data format schematic diagram of index；
Fig. 4 is that diagram data extracts mapping description schematic diagram.
Specific embodiment
As shown in Figs 1-4, the present invention provides the data presentation techniques in a kind of hundred billion grades of knowledge picture libraries, including diagram data
Expressing for knowledge method in library indexes representation method, Knowledge conversion representation method；Wherein expressing for knowledge method in chart database
The data format representation of data format representation and side including point, include two parts: data description explanation and
The storage format of data；Index representation method is made of association extension index, the full-text index of point and the full-text index on side；Knowledge
Representation method is converted to describe initial data how Mapping and Converting is point, number of edges evidence.
Expressing for knowledge method in chart database is made of the expression of point and side, comprising:
The data format of S1 point, the data representation format of point are divided into two parts, as shown in Figure 1, point data description explanation and
The storage format of point data.Wherein, include: in the data format of point
The description of S11, point data illustrate, for the Custom Attributes of description point and the retrieval mode of Custom Attributes, include
In have: the type of point, the parent type of point, the Custom Attributes of point, the Custom Attributes type of point, point Custom Attributes
Retrieval mode；The Custom Attributes type at its midpoint include text, integer, lint-long integer, number, date-time, the date, when
Between, longitude and latitude, the retrieval mode of the Custom Attributes of point includes nothing, index in classification, attribute retrieval, full-text search；
The storage format of S12, point data put actual storage content in picture library, consist of two parts: build-in attribute and
Custom Attributes；Wherein build-in attribute includes: the unique number, the data source identification of point, the storage number of tracing to the source of point, point of point
The generation time of type, the label of point, point；Custom Attributes is the attribute flexibly defined according to different scenes；Wherein, point traces back
Source storage number is made of the data source identification of the unique number and point put.
The data format on the side S2, the data representation format on side, is divided into two parts, as shown in Fig. 2, number of edges evidence description explanation and
The storage format of number of edges evidence.Wherein, include: in the data format on side
The description explanation of S21, number of edges evidence, describe the Custom Attributes on side and the retrieval mode of Custom Attributes, include
Inside have: while type, while parent type, while direction, the type of A endpoint, the type of B endpoint, while Custom Attributes, side
Custom Attributes type, side Custom Attributes retrieval mode；
The storage format of S22, number of edges evidence, while the actual storage content in picture library consists of two parts: build-in attribute and
Custom Attributes；Wherein build-in attribute include: while unique number, while data source identification, side trace to the source storage number, the end A
Point unique number, the unique number of B endpoint, while type, while label, side the generation time；Wherein, the storage of tracing to the source on side
Number by while unique number and while data source identification form.
A kind of method for being associated with extension, attribute retrieval, full-text search of any dimension of knowledge, by association extension index, point
Full-text index and side full-text index composition, as shown in Figure 3, comprising:
S31 figure association extension index, to store the association expansion relation of figure.Consist of two parts, the expansion relation of point
With the aggregate list on side.Wherein 1, the expansion relation of point includes, the unique number of A endpoint, while type, while date of occurrence (can
Choosing, if not the side of event mode, is not filled out), the unique number of B endpoint；2, while aggregate list include 0 or multiple while connection mark
Know, list size is 0 to indicate to be a virtual presence while (to two-way extension while virtual, such as there are the sides of A- > B, then
Virtual side is exactly A- > B), list size is more than the side of 0 expression physical presence.The connection identifier on each side includes the generation on side
Time (optional, if not the side of event mode, is not filled out), side resource identification；
The full-text index of S32 point, to store search field index a little.It consists of three parts, it is intrinsic to index, is customized
Field index and full-text index.Wherein 1, intrinsic index includes, the unique number of point, resource identification, point storage number of tracing to the source,
The label of the type, point put；Custom Attributes field according in the data description of point, the Custom Attributes type of point, point from
The retrieval mode (index in classification/attribute retrieval) of defined attribute is arranged；2, full-text index, the Custom Attributes type of foundation point,
Retrieval mode (full-text search) setting of the Custom Attributes of point；
The full-text index on the side S33, the search field to store side index.It consists of three parts, it is intrinsic to index, is customized
Field index and full-text index.Wherein 1, intrinsic index includes, while unique number, resource identification, while storage number of tracing to the source,
While type, while label, A endpoint unique number, the unique number of B endpoint；Data of the Custom Attributes field according to side
In description, while Custom Attributes type, while Custom Attributes retrieval mode (index in classification/attribute retrieval) setting；2,
Full-text index, according to while Custom Attributes type, while Custom Attributes retrieval mode (full-text search) setting.
The data presentation technique of high-throughput load is supported, how description initial data is converted to point, number of edges evidence, such as Fig. 4
It is shown, comprising:
How the mapping description of S41 point, description initial data are mapped as point data.Comprising one or more map units, often
A map unit describes the mapping mode of a kind of point.Inside map unit, the type comprising point, the mapping of the unique number of point
Relationship, the label mapping relationship of point, Custom Attributes mapping relations.Wherein mapping relations give directions attribute (build-in attribute, or from
Defined attribute) it is formed by which field of initial data and (may include one or more fields)；
How the mapping description of the side S42, description initial data are mapped as number of edges evidence.Comprising one or more map units, often
A map unit describes the mapping mode on a kind of side.Inside map unit, comprising while type, while unique number mapping
Relationship, the mapping relations of the unique number of A endpoint, the unique number mapping relations of B endpoint, the label mapping relationship on side, make by oneself
Adopted attribute mapping relations.Wherein mapping relations refer to the attribute (build-in attribute or Custom Attributes) on side by initial data which
Field forms (may include one or more fields).
To make simplified form, part related to the present invention is only schematically shown in each figure, they are not represented
Its practical structures as product.In addition, there is identical structure or function in some figures so that simplified form is easy to understand
Component only symbolically depicts one of those, or has only marked one of those.Herein, "/" indicates that " there are one
It is a ", " there are multiple " or " all exist "；The data referred in the text refer in particular to the data of structuring.
The data presentation technique in a kind of hundred billion grades of knowledge picture libraries proposed in this paper can be used to characterize number in knowledge picture library
According to, Knowledge conversion process and knowledge loading procedure are generally comprised, during two kinds, method specific implementation presented herein
It is as follows:
During Knowledge conversion, the diagram data based on the description in Fig. 4 extracts mapping descriptor format, uses distributed computing
Frame (MapReduce/Spark/Flink), converts raw data into side described in point data or Fig. 2 described in Fig. 1
Data, and by the knowledge store after conversion into HDFS file system.
During Knowledge conversion, need to read mapping definition.For mapping data, the initial data of every a kind of input,
There are multiple mapping definitions, each mapping definition can be converted to a kind of point data or a kind of number of edges evidence.The storage of mapping definition data
In database (MySQL/MongoDB).
During Knowledge conversion, the data after conversion are using the representation method characterization in Fig. 1/Fig. 2.The step of Knowledge conversion
Suddenly include: 1, reading the associated mapping definition data of initial data；2, initial data is read；3, it loops through and does not have a reading
Mapping；4, it maps one by one according to field mapping ruler using Distributed Architecture, converts raw data into point/number of edges evidence；5,
After completing single Mapping and Converting, based on point/number of edges evidence duplicate removal, store into HDFS, the file format of storage is
(SequenceFile/Parquet/Avro)；If 6, all reading mapping processing are completed, Knowledge conversion process terminates；If not complete
At continuation step 4.
In knowledge loading procedure, the knowledge after conversion is loaded into the storage engines inside picture library.It is deposited in storage engines
The data of storage include the data indicated with knowledge representation method (Fig. 1/Fig. 2) and association extension/attribute with any dimension of knowledge
The data that retrieval/full-text search method (Fig. 3) indicates.Knowledge store engine uses HBase and Solr/ElasticSearch,
Wherein HBase stored knowledge details data, association extension index data；The full-text search of Solr/ElasticSearch storage point,
Side full-text search data.
In knowledge loading procedure, the knowledge data after converting is read, is loaded into Solr/ElasticSearch, foundation
Solr/ElasticSearch constructs some full-text indexs and side full-text index automatically；In full-text index and side full-text index,
Only storage index data；
In knowledge loading procedure, read conversion after knowledge data, be loaded into HBase, save as a detailed data and
Side detailed data；
In knowledge loading procedure, the knowledge data after converting is read, distributed computing framework (MapReduce/ is passed through
Spark/Flink), according to association extension index in association extension/attribute retrieval/full-text search method of any dimension of knowledge
Data format, building index.The index is stored using HBase.Processing step includes: 1, reading the knowledge after conversion；2, make
With Distributed Architecture computation index；If the mark of 3 number of edges evidences includes the field of Time of Day dimension, date-time is just divided into day
The endpoint of phase part (being accurate to day) and time portion (being accurate to the second), day part and A indicate, the endpoint of B indicates, the class on side
Type collectively constitutes expansion relation part a little；Time portion and resource identification collectively constitute the aggregate list part on side；4, foundation
The expansion relation of point, using GroupBy (expansion relation of point), by the side under the data of the expansion relation of point having the same
Aggregate list aggregates into a big aggregate list；5, the association extension index data of generation is written in HDFS；6, it will give birth to
At association extension index data make generate HBase HFile file, imported into HBase in batches.
The preferred embodiment of the present invention has been described in detail above.It should be appreciated that those skilled in the art without
It needs creative work according to the present invention can conceive and makes many modifications and variations.Therefore, all technologies in the art
Personnel are available by logical analysis, reasoning, or a limited experiment on the basis of existing technology under this invention's idea
Technical solution, all should be within the scope of protection determined by the claims.
Claims (6)
1. the data presentation technique in a kind of hundred billion grades of knowledge picture libraries, which is characterized in that including expressing for knowledge in chart database
Method indexes representation method, Knowledge conversion representation method；Wherein expressing for knowledge method includes the data lattice put in chart database
The data format representation of formula representation and side includes two parts: the storage format of the description explanation and data of data；
Index representation method is made of association extension index, the full-text index of point and the full-text index on side；Knowledge conversion representation method is retouched
Stating initial data, how Mapping and Converting is point, number of edges evidence.
2. the data presentation technique in hundred billion grades of knowledge picture libraries of one kind as described in claim 1, it is characterised in that: the point
Include: in data format representation
The description explanation of S11, point data, for the Custom Attributes of description point and the retrieval mode of Custom Attributes, include is interior
Have: the type of point, the parent type of point, the Custom Attributes of point, the Custom Attributes type of point, point Custom Attributes inspection
Rope mode；The Custom Attributes type at its midpoint includes text, integer, lint-long integer, number, date-time, date, time, warp
Latitude, the retrieval mode of the Custom Attributes of point include nothing, index in classification, attribute retrieval, full-text search；
The storage format of S12, point data put actual storage content in picture library, consist of two parts: build-in attribute and making by oneself
Adopted attribute；Wherein build-in attribute includes: the unique number, the data source identification of point, the storage number of tracing to the source of point, the class of point of point
The generation time of type, the label of point, point；Custom Attributes is the attribute flexibly defined according to different scenes；Wherein, point is traced to the source
Storage number is made of the data source identification of the unique number and point put.
3. the data presentation technique in hundred billion grades of knowledge picture libraries of one kind as described in claim 1, it is characterised in that: the side
Include: in data format representation
The description explanation of S21, number of edges evidence, describe the Custom Attributes on side and the retrieval mode of Custom Attributes, the content for including
Have: while type, while parent type, while direction, the type of A endpoint, the type of B endpoint, while Custom Attributes, side from
Defined attribute type, side Custom Attributes retrieval mode；
The storage format of S22, number of edges evidence, while the actual storage content in picture library consists of two parts: build-in attribute and making by oneself
Adopted attribute；Wherein build-in attribute include: while unique number, while data source identification, side trace to the source storage number, A endpoint
Unique number, the unique number of B endpoint, while type, while label, side the generation time；Wherein, the storage number of tracing to the source on side
By while unique number and while data source identification form.
4. the data presentation technique in hundred billion grades of knowledge picture libraries of one kind as described in claim 1, which is characterized in that the association
The full-text index of extension index, the full-text index of point and side specifically includes:
S31, figure association extension index, to store the association expansion relation of figure；Consist of two parts: the expansion relation of point and side
Aggregate list；Wherein:
Point expansion relation include: A endpoint unique number, while type, while date of occurrence, the unique number of B endpoint；
While aggregate list include 0 or multiple while connection identifier, list size be 0 indicate be a virtual presence side, column
Table size be more than 0 expression physical presence side, it is each while connection identifier include: while time of origin, side resource identification；
The full-text index of S32, point, to store search field index a little；It consists of three parts, intrinsic index, customized word
Segment index and full-text index；Wherein:
Intrinsic index includes: the label of type, point that the storage of tracing to the source of the unique number, resource identification, point of point is numbered, put；
In the data description of Custom Attributes field foundation point, the retrieval of the Custom Attributes type, the Custom Attributes of point of point
Mode is arranged；The retrieval mode of the Custom Attributes of point includes index in classification, attribute retrieval；
Full-text index, the retrieval mode setting of Custom Attributes type, the Custom Attributes of point according to point；It makes by oneself at its midpoint
The retrieval mode of adopted attribute is full-text search；
S33, side full-text index, to store side search field index；It consists of three parts, intrinsic index, customized word
Segment index and full-text index；Wherein:
Intrinsic index includes: while unique number, resource identification, while storage number of tracing to the source, while type, while label, the end A
Unique number, the unique number of B endpoint of point；
Custom Attributes field according in the data description on side, while Custom Attributes type, while Custom Attributes retrieval
Mode is arranged；Wherein, the retrieval mode of the Custom Attributes on side includes index in classification, attribute retrieval；
Full-text index, according to while Custom Attributes type, while Custom Attributes retrieval mode setting；Wherein side is made by oneself
The retrieval mode of adopted attribute is full-text search.
5. the data presentation technique in hundred billion grades of knowledge picture libraries of one kind as described in claim 1, which is characterized in that described original
Data Mapping and Converting is the method for point data specifically: comprising one or more map units, each map unit describes one
The mapping mode of class point；Inside map unit, the type comprising point, the unique number mapping relations of point, point label mapping
Relationship, Custom Attributes mapping relations；The attribute that wherein mapping relations are given directions is made of those of initial data field, midpoint
Attribute be build-in attribute or Custom Attributes, the field of initial data includes one or more fields.
6. the data presentation technique in hundred billion grades of knowledge picture libraries of one kind as described in claim 1, which is characterized in that described original
Data Mapping and Converting is the method for number of edges evidence specifically: comprising one or more map units, each map unit describes one
The mapping mode on class side；Inside map unit, comprising while type, while unique number mapping relations, unique volume of A endpoint
Number mapping relations, the unique number mapping relations of B endpoint, the label mapping relationship on side, Custom Attributes mapping relations；Wherein
Mapping relations refer to that the attribute on side is made of those of initial data field, and the attribute at midpoint is build-in attribute or customized category
Property, the field of initial data includes one or more fields.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910514448.8A CN110275969A (en) | 2019-06-13 | 2019-06-13 | A kind of data presentation technique in hundred billion grades of knowledge picture libraries |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910514448.8A CN110275969A (en) | 2019-06-13 | 2019-06-13 | A kind of data presentation technique in hundred billion grades of knowledge picture libraries |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110275969A true CN110275969A (en) | 2019-09-24 |
Family
ID=67960809
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910514448.8A Pending CN110275969A (en) | 2019-06-13 | 2019-06-13 | A kind of data presentation technique in hundred billion grades of knowledge picture libraries |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110275969A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111538804A (en) * | 2020-04-20 | 2020-08-14 | 北京京安佳新技术有限公司 | HBase-based graph data processing method and equipment |
-
2019
- 2019-06-13 CN CN201910514448.8A patent/CN110275969A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111538804A (en) * | 2020-04-20 | 2020-08-14 | 北京京安佳新技术有限公司 | HBase-based graph data processing method and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9015118B2 (en) | Determining and presenting provenance and lineage for content in a content management system | |
| US11334549B2 (en) | Semantic, single-column identifiers for data entries | |
| KR20170019352A (en) | Data query method and apparatus | |
| Ye et al. | Development of a highly flexible mobile GIS-based system for collecting arable land quality data | |
| Jo et al. | High-performance geospatial big data processing system based on MapReduce | |
| US11726846B2 (en) | Interface for processing sensor data with hyperscale services | |
| US11645247B2 (en) | Ingestion of master data from multiple applications | |
| US8250533B2 (en) | Reflection over objects | |
| CN104216961A (en) | Method and device for data processing | |
| Jeong et al. | A data management infrastructure for bridge monitoring | |
| Wan et al. | An effective NoSQL-based vector map tile management approach | |
| Fadhel et al. | A comparison of time series databases for storing water quality data | |
| Park et al. | E-Navigation-supporting data management system for variant S-100-based data | |
| US20190197123A1 (en) | Metadata storage method, device and server | |
| Lu et al. | Study on urban expansion and population density changes based on the inverse S-shaped function | |
| CN110275969A (en) | A kind of data presentation technique in hundred billion grades of knowledge picture libraries | |
| CN104794567B (en) | A kind of Infectious Diseases Data management method based on HBase | |
| Evans et al. | A data quality strategy to enable fair, programmatic access across large, diverse data collections for high performance data analysis | |
| CN111125216A (en) | Method and device for importing data into Phoenix | |
| US20220083724A1 (en) | Methods and systems for assisting document editing | |
| CN105786478A (en) | Data processing method and device | |
| Zhu et al. | Algebraic operations on spatiotemporal data based on RDF | |
| Negru et al. | A unified approach to data modeling and management in big data era | |
| CN103150408B (en) | Real-time data base finds data base the System and method for accessed according to calling the roll | |
| Jeong et al. | Data management technologies for infrastructure monitoring |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190924 |