CN107239571B

CN107239571B - Index construction method based on multidimensional data space technology

Info

Publication number: CN107239571B
Application number: CN201710506059.1A
Authority: CN
Inventors: 孙成通; 董毅; 付宪瑞; 王玉奎
Original assignee: Inspur Financial Information Technology Co Ltd
Current assignee: Inspur Financial Information Technology Co Ltd
Priority date: 2017-06-28
Filing date: 2017-06-28
Publication date: 2021-04-09
Anticipated expiration: 2037-06-28
Also published as: CN107239571A

Abstract

The invention relates to an index construction method based on a multidimensional data space technology, which comprises the steps of initializing a multidimensional data retrieval system, loading data files needing to be indexed, and setting multidimensional file index items, wherein the multidimensional file index items at least comprise a file type index dimension, a file attribute index dimension, a file characteristic index dimension and a file content index dimension. Based on the differentiation characteristics, the method can rapidly narrow the retrieval range in a multi-dimensional indexing mode step by step, thereby achieving the purpose of improving the retrieval speed. The method can create index data for various types of files such as pictures, audio, video, compression, encryption and the like, thereby greatly improving the retrieval speed of the contents of the files, having important application value in the fields such as fingerprint comparison, face recognition and the like, and having higher efficiency and better accuracy than the traditional method.

Description

Index construction method based on multidimensional data space technology

Technical Field

The invention relates to an index construction method based on a multidimensional data space technology, and belongs to the technical field of software.

Background

With the improvement and popularization of informatization construction of various traditional industries such as finance, security, government affairs and the like, more and more user data need to be stored and retrieved. Particularly, in some large-scale systems such as banks involving group users, the data of the relevant data carried by the large-scale systems is hundreds of millions, and how to quickly and accurately locate the valid data has become an important problem in these industries. When an index is created in a traditional way, the index is often created based on readable characters, and the maximum performance can be ensured. The index created by the program cannot lock the main features of the data, and the index is only regarded as a meaningless simple identifier, so that the index cannot effectively locate the attribute features of the data, and the improvement of the retrieval performance is not obvious.

Taking the search of fingerprint data as an example, the conventional fingerprint search technology is very competitive today when the user data is growing in a large scale, and the improvement of hardware performance alone is not enough to deal with the huge information search amount. On the one hand, the retrieval process of image type data such as fingerprints is complicated, and on the other hand, the traditional data retrieval mode is not suitable for the retrieval requirement of multimedia type data. How to improve the retrieval efficiency of a large amount of non-readable data becomes a technical problem which is urgently needed to be solved in adaptation to the rapid development of the market.

Disclosure of Invention

The technical problem to be solved by the invention is to overcome the defects of the prior art and provide an index construction method based on the index features of a multidimensional data space technology, high index efficiency, uniqueness, group isolation and the like.

In order to solve the above technical problems, the present invention provides an index construction method based on multidimensional data space technology, which includes initializing a multidimensional data retrieval system and loading data files requiring index establishment, and is characterized by further including the following steps:

firstly, setting a multidimensional file index item, wherein the multidimensional file index item at least comprises a file type index dimension, a file attribute index dimension, a file characteristic index dimension and a file content index dimension;

step two, combining all the dimensional feature indexes constructed in the step one to create a multi-dimensional index structure in a data space;

loading the multidimensional index structure into the current retrieval data space and fusing the multidimensional index structure with the current index content to form a multidimensional data space search tree;

and step four, completing the construction of the multi-dimensional index.

The invention is further defined by the following technical characteristics: the multidimensional file index item of the step one also comprises a spatial grammar index.

Furthermore, the construction of the file type index dimension is to establish a corresponding data space dimension according to the file type.

Further, the file types include a file type determined based on a file name suffix, a file type determined based on a file header, and a file type having a self-encoding characteristic.

Further, the file feature index dimension comprises a uniqueness index, an index for acquiring a file feature code by an auxiliary means, and a spatial geometry feature index.

Further, the spatial geometrical features comprise audio features and graphic texture features.

Further, the file content index dimension includes an index of visible characters and an index of non-visible characters.

Further, the non-visible characters are associated through file commonality characteristic content.

Further, the data space includes spatial coordinates, spatial geometry, and interactions between geometries.

Further, when the data file needing to establish the index is loaded into the data space, the binary data is converted into three-dimensional data, and then the index dimension is established and points to the specific coordinate in the space geometry.

The invention has the beneficial effects that: the main difference between the multidimensional fast retrieval technology and the traditional search technology is represented on the multidimensional index creation. In the process of hybrid retrieval, due to various factors such as different file types, different data contents, different data characteristics and the like, the indexing modes of various types of data have great difference. Based on the differentiation characteristics, the method can rapidly narrow the retrieval range in a multi-dimensional indexing mode step by step, thereby achieving the purpose of improving the retrieval speed. The method can create index data for various types of files such as pictures, audio, video, compression, encryption and the like, thereby greatly improving the retrieval speed of the contents of the files, having important application value in the fields such as fingerprint comparison, face recognition and the like, and having higher efficiency and better accuracy than the traditional method.

Drawings

FIG. 1 is a flow chart of index construction according to the present invention.

Detailed Description

Example 1

The index construction method based on the multidimensional data space technology provided by the embodiment is as shown in fig. 1: the method comprises initializing a multi-dimensional data retrieval system and loading data files needing to be indexed, and is characterized by further comprising the following steps:

step one, setting the index dimension of the file according to the difference of the loaded file type and content, comprising the following steps:

1) file type index dimension

Creating an index according to the file type characteristics, which mainly comprises the following steps: file type determined based on a filename suffix, file type determined based on a file header, and content characteristics specific to certain file types, among others. Taking fingerprint image files as an example, the file types can be rapidly classified based on file name suffix or file header definition, but files in png or jpg formats have own coding characteristics, and the special formats need to be identified in the process so as to select a proper content identification method in the subsequent process

2) File attribute index dimension

Including basic file attributes such as creation time, creator, file size, etc. of file

3) File feature index dimension

The characteristic index comprises a traditional unique index and also comprises a plurality of characteristic identification and index modes such as an index method for acquiring the characteristic code of the file by an auxiliary means. The multi-dimensional data features more space geometric features, and different data types have different abstract geometric features in the data space, for example, an image file has a planar feature in a time dimension, and a video file and an audio file have a stereo feature in the time dimension, and the like

4) File content index dimension

The indexes established according to the content of the files comprise indexes of visible characters and indexes of invisible characters, wherein the indexes of visible characters can create association relations among different files through artificial classification, and the invisible characters need to be associated through common content, for example, files of the same manufacturer have the same file signature and the like.

5) Spatial texture index dimension

When the file content index is not enough to meet the requirement, the retrieval range is further reduced by using the spatial texture index. Taking fingerprint data as an example, the size, feature structure and the like of a fingerprint can be used as the creation basis of a spatial texture index, and in an audio/video file, the facial features, the voice print features and the like of a character can also be used for creating the index by the method.

Step two, creating a multidimensional file index

And combining the dimensional feature indexes created in the step one to create a multidimensional index structure in a data space.

Step three, loading the multidimensional data space

And loading the multidimensional index structure into a data space used by the current retrieval.

Step four, index of configuration file

The index of the file is fused with the existing index content to form a multi-dimensional data space search tree

And step five, completing the creation of the multidimensional index.

The method utilizes the concept of multidimensional data space to carry out multidimensional induction on the index attribute of the data, and indexes are carried out among each data type one by one according to the similarity. For example, when searching fingerprint data, firstly indexing is carried out according to the size characteristics of fingerprints, the fingerprint data with different sizes are classified into a plurality of levels, and characteristic indexes are respectively created among the fingerprint data under the same level according to special attributes such as geometric characteristics, so that a hierarchical structure in a data space is formed. In addition, there are some commonalities between the picture file mainly based on fingerprint data and the picture file of other contents, and these commonalities are important features for processing the picture type index, and if the picture file is used as the dimension for indexing, the effective contents such as picture size, color depth, creation time, etc. become indexes.

Therefore, the method is based on the multi-dimensional data space technology to create the index for the binary data file, the principle of index creation lies in the common characteristic of the files of the same type, and the characteristic must accord with the uniqueness principle of the data of the type, so that other types of data can be quickly eliminated, and the retrieval range of the data is reduced at the fastest speed.

In addition to the above embodiments, the present invention may have other embodiments. All technical solutions formed by adopting equivalent substitutions or equivalent transformations fall within the protection scope of the claims of the present invention.

Claims

1. The index construction method based on the multidimensional data space technology comprises the steps of initializing a multidimensional data retrieval system and loading data files needing to establish indexes, and is characterized by further comprising the following steps:

firstly, setting a multi-dimensional file index item, wherein the multi-dimensional file index item at least comprises a file type index dimension, a file attribute index dimension, a file characteristic index dimension, a file content index dimension and a spatial texture index dimension, the file type index dimension is constructed by establishing a corresponding data space dimension according to a file type, and the file type comprises a file type determined based on a file name suffix, a file type determined based on a file header and a file type with own coding characteristics; the file feature index dimension comprises a uniqueness index, an index for acquiring a file feature code by an auxiliary means and a space geometric feature index;

step two, combining all the dimensional feature indexes constructed in the step one to create a multi-dimensional index structure in a data space, wherein the data space comprises space coordinates, a space geometric structure and interaction among the geometric structures;

and step four, completing the construction of the multi-dimensional index.

2. The index building method based on multidimensional data space technology as claimed in claim 1, wherein: the spatial geometry features include audio features and graphical texture features.

3. The index building method based on multidimensional data space technology as claimed in claim 1, wherein: the file content index dimension includes an index of visible characters and an index of non-visible characters.

4. The index building method based on multidimensional data space technology as claimed in claim 3, wherein: the non-visible characters are associated through file commonality characteristic content.

5. The index building method based on multidimensional data space technology according to any one of claims 1 to 4, wherein: when a data file needing to establish an index is loaded into a data space, the binary data is converted into three-dimensional data, then an index dimension is established, and the index dimension points to a specific coordinate in a space geometric body.