CN117251601A - Improved optimization algorithm for knowledge graph compression storage - Google Patents

Abstract

The invention relates to the field of knowledge graph correlation, in particular to an improved optimization algorithm for knowledge graph compression storage, which is characterized in that after secondary compression is carried out on a k2-tree, priority ranking of data block sizes is carried out on a sub-matrix, when data are searched, the data are searched according to the priority, so that the blocks ranked in front are both subjects and objects, the data reading efficiency can be improved, the method can also be applied to personalized ranking in the field of electronic commerce, and the most desired data of a user can be ranked in front for preferential search aiming at user preference.

Description

Improved optimization algorithm for knowledge graph compression storage

Technical Field

The invention relates to the field of knowledge graph correlation, in particular to an improved optimization algorithm for knowledge graph compression storage.

Background

Knowledge graph: the method combines the theory and method of subjects such as application mathematics, graphics, information visualization technology, information science and the like with the method of metering introduction analysis, co-occurrence analysis and the like, and utilizes the visualized map to vividly display the core structure, development history, leading edge field and overall knowledge architecture of the subjects to achieve the modern theory of multi-subject fusion.

Sparse matrix: in the matrix, if the number of elements with the value of 0 is far more than that of non-0 elements and the distribution of the non-0 elements is irregular, the matrix is called as a sparse matrix; in contrast, if the number of elements other than 0 is the majority, the matrix is referred to as a dense matrix. The sum of non-zero elements is defined as the thickness of the matrix over the sum of all elements of the matrix.

Brum filter: the bloom filter is a special hash table in which each slot stores only 0 or 1, so each 01 can be stored using the smallest unit bit in the computer to achieve a significant space saving. The lookup of a certain data object a under very large data sets can still have significant overhead even with hash tables, especially when looking up in multiple independently distributed data sets.

RLE compression technique: the basic idea of RLE compression algorithms is to divide data into two cases according to a linear sequence: one is a continuous repeated data block and the other is a continuous non-repeated data block.

The principle of RLE algorithm is to use an attribute representing the number of blocks plus a data block to represent several blocks of data that are originally continuous, so as to achieve the goal of saving storage space. The RLE algorithm generally selects a data block length of 1 byte, and the integrity of the number of the blocks is also represented by 1 byte, and for image files or text files with a color number of less than 256 colors, the block length of 1 byte is suitable.

A binary group ordering technique: the tuples are sorted from small to large by a first element and if the first is the same and the second is sorted from large to small, the tuples are used by math to describe mathematical objects that determine the composition.

k2-triples: k2-triples is a tuple compression storage specifically designed for tuples sharing attributes, the basic idea of which is based on k 2-tree. k2-triples store tuples with the same properties using a bit matrix, wherein the rows and columns of the bit matrix correspond to the subjects and objects of the tuple respectively, which are set to 1 when the subject and object to which any bit in the bit matrix corresponds are present, and otherwise to 0.

Disclosure of Invention

The invention aims to provide an improved optimization algorithm for compressing and storing a knowledge graph, which is characterized in that after knowledge graph data is compressed for the second time, the retrieval efficiency of the data after the data is compressed and stored is improved, bit blocks representing 1 in a bit string are extracted, a binary group is formed by the bit blocks and subscripts thereof, a large number of continuous bit blocks representing 1 are preferentially retrieved through the sequence of the binary group, and therefore the retrieval speed is greatly improved. At this time, only the digits with the binary digits of 1 and the corresponding lower marks are recorded, so that the effective data in the sparse matrix can be accurately found, and the larger the numerical value of the digits is, the more continuous 1 s are represented, and the continuous 1 search query is preferentially searched.

In order to achieve the above purpose, the present invention provides the following technical solutions: a knowledge graph compression storage improved optimization algorithm, comprising the steps of:

s1, performing secondary compression on a k2-tree to obtain a bit string, wherein each number in the bit string represents the frequency of binary bits 0 and 1;

step S2, judging which bit blocks contain binary bits of 1 according to the first bit identifier;

step S3, forming a binary array by the numerical value of the effective digit and the array subscript which are judged in the step S2, wherein a first element of the binary array is how many continuous '1' S exist in the bit block, and a second element of the binary array is the subscript of the bit block in the bit string after secondary compression;

step S4, ordering the first element of each binary group data in the step S3 from large to small, wherein the more the obtained element of the binary group is, the more 1 binary bits are continuously contained;

step S5, when data are read, firstly, reading the second number of each element in the binary array, wherein the second number is the subscript corresponding to the bit string after secondary compression, and then accessing the bit string by using the subscript;

and S6, when data is required to be queried, sequentially searching the ordered binary array, wherein the array element which is more front represents more '1' binary bits in the bit string after secondary compression, and the contained information amount is larger.

Preferably, the bit strings in the step S1 are in the following format: [ head identifier ] + frequency of binary bits, for example, [1]2 1 1 1 2 1 1 2 1 1 1 1 1 2 2 2 2 2 1 3 1 4 1 2 1 1], in this bit string, [1] represents a binary string starting with 1, 0 and 1 bit blocks are alternately arranged so that 2 on the 0 th bit represents 2 1 s, 1 on the first bit represents one 0, one on the second bit represents one 1, 1 on the third bit represents one 0, 2 on the fourth bit represents 2 1 s, and so on, 1 represents that subjects and objects corresponding to any bit in the matrix exist, and conversely, 0.

Preferably, in the step S2, determining which bit blocks contain "1" bits according to the first bit identifier is specifically:

firstly, according to the first bit identifier in the step S1, all bit blocks containing 1 are found, and as the first bit in the bit string after secondary compression is the identification bit, the first bit is judged, and if the identification bit is 1, the valid bit of the bit string is from 1; the identification bit is 0, which means that the bit string is started from 0, wherein even bits are read if the first bit is judged to be 1, because the subscript is started from 0, and odd bits are read if the first bit is judged to be 0.

Preferably, the format of the two-tuple in the step S3 is as follows: args { { { data, index }, wherein data is data in the bit string, index is a subscript of the bit string, and the binary group is used for storing data, so that a target object is stored, and the position of the object in the original character string is also stored; all bit block values containing 1 and subscripts in the bit string after secondary compression form a binary array, wherein the first element is a value and the second element is a subscript.

Preferably, in the step S4, the first element of each binary group data is arranged from large to small by using a bubble ordering, and the specific bubble ordering is as follows:

For(int i＝0；i<arr.length-1&&flag；i++)

IF args[i][1]<args[i+1][1]

Swap(args[i],args[i+1])

End。

compared with the prior art, the invention has the beneficial effects that: the method can be applied to personalized sequencing in the e-commerce field, and can be used for preferentially searching the most wanted block of the user in front according to the user preference.

Drawings

FIG. 1 is a schematic flow chart of an optimization algorithm of a storage query after secondary compression;

FIG. 2 is a schematic diagram of a data reading process according to the present invention;

fig. 3 is a schematic diagram of a practical application flow in the field of querying a re-user of the present invention.

Detailed Description

The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Referring to fig. 1-3, the present invention provides a technical solution: a knowledge graph compression storage improved optimization algorithm, comprising the steps of:

s1, performing secondary compression on a k2-tree to obtain a bit string, wherein each number in the bit string represents the frequency of binary bits 0 and 1;

step S2, judging which bit blocks contain binary bits of 1 according to the first bit identifier;

step S3, forming a binary array by the numerical value of the effective digit and the array subscript which are judged in the step S2, wherein a first element of the binary array is how many continuous '1' S exist in the bit block, and a second element of the binary array is the subscript of the bit block in the bit string after secondary compression;

step S4, ordering the first element of each binary group data in the step S3 from large to small, wherein the more the obtained element of the binary group is, the more 1 binary bits are continuously contained;

step S5, when data are read, firstly, reading the second number of each element in the binary array, wherein the second number is the subscript corresponding to the bit string after secondary compression, and then accessing the bit string by using the subscript;

and S6, when data is required to be queried, sequentially searching the ordered binary array, wherein the array element which is more front represents more '1' binary bits in the bit string after secondary compression, and the contained information amount is larger.

Further, the bit strings in step S1 are in the format of: [ head identifier ] + frequency of binary bits, for example, [1]2 1 1 1 2 1 1 2 1 1 1 1 1 2 2 2 2 2 1 3 1 4 1 2 1 1], in this bit string, [1] represents a binary string starting with 1, 0 and 1 bit blocks are alternately arranged so that 2 on the 0 th bit represents 2 1 s, 1 on the first bit represents one 0, one on the second bit represents one 1, 1 on the third bit represents one 0, 2 on the fourth bit represents 2 1 s, and so on, 1 represents that subjects and objects corresponding to any bit in the matrix exist, and conversely, 0.

Further, in step S2, determining which bit blocks contain "1" bits according to the first bit identifier is specifically:

firstly, according to the first bit identifier in the step S1, all bit blocks containing 1 are found, and as the first bit in the bit string after secondary compression is the identification bit, the first bit is judged, and if the identification bit is 1, the valid bit of the bit string is from 1; the identification bit is 0, which means that the bit string is started from 0, wherein even bits are read if the first bit is judged to be 1, because the subscript is started from 0, and odd bits are read if the first bit is judged to be 0

Further, the format of the two-tuple in step S3 is as follows: args { { { data, index }, wherein data is data in the bit string, index is a subscript of the bit string, and the binary group is used for storing data, so that a target object is stored, and the position of the object in the original character string is also stored; all bit block values containing 1 and subscripts in the bit string after secondary compression form a binary array, wherein the first element is a value and the second element is a subscript

Further, in the step S4, the first element of each binary group data is arranged from large to small by using a bubble ordering, and the specific bubble ordering is as follows:

For(int i＝0；i<arr.length-1&&flag；i++)

IF args[i][1]<args[i+1][1]

Swap(args[i],args[i+1])

End。

specifically, the knowledge graph compression storage improvement optimization algorithm can be used for user preference retrieval service in the field of electronic commerce, the search before personalization is identical to the same query, and the results seen by different users are identical, so that the requirements of all users may not be met. In the commodity searching, the problem is particularly remarkable, because the users of the commodity searching may particularly favor commodities of certain brands, prices and shops, in order to reduce the screening cost of the users, the searching results need to be displayed in a personalized way according to the users, the first step of individuation is to model the users and the commodities respectively, the second step is to server the models, and after the two steps are carried out, the steps are the prior art, so that redundant description is omitted; when a user inquires, a user model service and an online search service are simultaneously called by using a merge service model, the user model service returns user dimension characteristics, the online search service returns commodity information, a sorting module uses the two parts of data to reorder results, different types of commodities can be given different weights according to the extracted user dimension characteristics, the data are subjected to secondary compression, the more continuous 1 after the secondary compression is represented as the higher the preference degree of the commodity user is, the more the weight is occupied, if 0 is represented as the no interest of the user in the commodity, the data after the secondary compression can be sorted according to an optimization algorithm designed in the text, so that the data blocks containing more continuous 1 are arranged in front, and the commodity which is preferred by the user can be searched by the user is represented, wherein the improved optimization algorithm for the compression and storage of the knowledge graph is equivalent to an embedded module function in the commodity.

The method can be applied to personalized sequencing in the e-commerce field, and can be used for preferentially searching the most wanted block of the user in front according to the user preference.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made therein without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (5)

1. The improved optimization algorithm for the compressed storage of the knowledge graph is characterized by comprising the following steps of:

s1, performing secondary compression on a k2-tree to obtain a bit string, wherein each number in the bit string represents the frequency of binary bits 0 and 1;

step S2, judging which bit blocks contain binary bits of 1 according to the first bit identifier;

step S3, forming a binary array by the numerical value of the effective digit and the array subscript which are judged in the step S2, wherein a first element of the binary array is how many continuous '1' S exist in the bit block, and a second element of the binary array is the subscript of the bit block in the bit string after secondary compression;

step S4, ordering the first element of each binary group data in the step S3 from large to small, wherein the more the obtained element of the binary group is, the more 1 binary bits are continuously contained;

step S5, when data are read, firstly, reading the second number of each element in the binary array, wherein the second number is the subscript corresponding to the bit string after secondary compression, and then accessing the bit string by using the subscript;

and S6, when data is required to be queried, sequentially searching the ordered binary array, wherein the array element which is more front represents more '1' binary bits in the bit string after secondary compression, and the contained information amount is larger.

2. The knowledge-graph compressed storage improvement optimization algorithm according to claim 1, wherein: the bit string format in the step S1 is as follows: [ head identifier ] + frequency of binary bits, for example, [1]2 1 1 1 2 1 1 2 1 1 1 1 1 2 2 2 2 2 1 3 1 4 1 2 1 1], in this bit string, [1] represents a binary string starting with 1, 0 and 1 bit blocks are alternately arranged so that 2 on the 0 th bit represents 2 1 s, 1 on the first bit represents one 0, one on the second bit represents one 1, 1 on the third bit represents one 0, 2 on the fourth bit represents 2 1 s, and so on, 1 represents that subjects and objects corresponding to any bit in the matrix exist, and conversely, 0.

3. The knowledge-graph compressed storage improvement optimization algorithm according to claim 1, wherein: in the step S2, determining which bit blocks contain "1" binary bits according to the first bit identifier specifically includes:

firstly, according to the first bit identifier in the step S1, all bit blocks containing 1 are found, and as the first bit in the bit string after secondary compression is the identification bit, the first bit is judged, and if the identification bit is 1, the valid bit of the bit string is from 1; the identification bit is 0, which means that the bit string is started from 0, wherein even bits are read if the first bit is judged to be 1, because the subscript is started from 0, and odd bits are read if the first bit is judged to be 0.

4. The knowledge-graph compressed storage improvement optimization algorithm according to claim 1, wherein: the format of the two-tuple in the step S3 is as follows: args { { { data, index }, wherein data is data in the bit string, index is a subscript of the bit string, and the binary group is used for storing data, so that a target object is stored, and the position of the object in the original character string is also stored; all bit block values containing 1 and subscripts in the bit string after secondary compression form a binary array, wherein the first element is a value and the second element is a subscript.

5. The knowledge-graph compressed storage improvement optimization algorithm according to claim 1, wherein: in the step S4, the first element of each binary group data is arranged from large to small by using the bubble ordering, and the specific bubble ordering is as follows:

For(int i＝0；i<arr.length-1&&flag；i++)

IF args[i][1]<args[i+1][1]

Swap(args[i],args[i+1])

End。