CN109074363A

CN109074363A - Data query method, data query system determine method and apparatus

Info

Publication number: CN109074363A
Application number: CN201680085206.6A
Authority: CN
Inventors: 宋风龙; 刘武龙; 汪涛
Original assignee: Huawei Technologies Co Ltd
Current assignee: Huawei Technologies Co Ltd
Priority date: 2016-05-09
Filing date: 2016-05-09
Publication date: 2018-12-21
Also published as: WO2017193263A1

Abstract

A kind of data query method, data query system determine method and apparatus.It include: reception inquiry request (401) first, then, the E under f kind mode the first subdatas (402) are determined according to the data to be checked under f kind mode, in turn, the E feature vector (403) under the E one-to-one f kind mode of the first subdata under f kind mode is determined；Further determine the E digital signal (404) under the one-to-one f kind mode of E feature vector under f kind mode；Finally, from e-th of abstract semantic description (405) corresponding with e-th of digital signal under f kind mode is obtained in storage system.Wherein, since the index of storage system is e-th of digital signal under f kind mode, so as to directly be retrieved storage system to obtain abstract semantic description corresponding with the digital signal according to the digital signal of data to be checked, to effectively reduce the retrieval required time, the requirement of real-time has been better met.

Description

Data query method, data query system determine method and apparatus

Technical field

The present invention relates to computer processing technologies more particularly to a kind of data query method, data query system to determine method and apparatus.

Background technique

As Intelligent treatment applies the increasingly promotion of status, the neural network that can be obviously improved Intelligent treatment application processing speed becomes industry focus of attention.

Since the energy consumption of the biological impulsive neural networks inspired at runtime is well below existing artificial neural network, thus become the development trend of Intelligent treatment computation platform, currently, class brain/neuron architecture processor of the development branch of the impulsive neural networks as biology inspiration is the hot spot of research.Currently, academia and industry are directed to class brain chip/neuron chip in class brain/neuron architecture processor the study found that significant challenge of its design is class brain chip/neuron chip storage system.

And the technical solution of existing storage system, it is all based on memory address index storage system, as shown in Figure 1, multiple keywords of data to be checked are obtained first, then pass through for each keyword in Content Addressable Memory (Content Addressable Memory, retrieve the memory address of corresponding with keyword data storage in referred to as are as follows: CAM), then from be to read the data inquired in the storage system indexed with memory address.

During searching data by above-mentioned storage system, it is delayed larger, is unable to satisfy the requirement of real-time.

Summary of the invention

The embodiment of the present invention provides a kind of data query method, data query system determines method and apparatus.During overcoming and search data to be checked in the prior art, the problem of being delayed larger, be unable to satisfy requirement of real-time.

First aspect present invention provides a kind of data query method, and method is applied to the multi-modal of intelligent chip In storage system, this method comprises: firstly, receiving the inquiry request including the data to be checked under F kind mode, wherein F is positive integer；Then, the first subdatas of the E under f kind mode are determined according to the data to be checked under f kind mode, the f successively take section (0, F] positive integer, the E is positive integer；Further, E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode is determined；In turn, E digital signal under the one-to-one f kind mode of E feature vector under f kind mode is determined；Finally, from corresponding with e-th of digital signal under the f kind mode e-th of abstract semantic description is obtained in storage system, the e successively take section (0, E] positive integer.

In the present embodiment, due to the corresponding digital signal of feature vector that the index of storage system is the data to be checked under f kind mode, so as to directly retrieve according to the content of data to be checked to storage system, the time needed for effectively reducing retrieval, the requirement of real-time has been better met.

In some embodiments of first aspect, from acquisition e-th of abstract semantic description corresponding with e-th of digital signal under f kind mode in storage system, it include: the matched index entry of e-th of digital signal under searching in the f concordance list under f kind mode with f kind mode, according to matched index entry from the abstract semantic description for obtaining e-th of digital signal under f kind mode in storage system.

In some embodiments of first aspect, if not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under f kind mode, e-th of second subdatas under f kind mode are then redefined from the data to be checked under f kind mode, wherein, e-th of second subdatas and e-th of first subdatas under f kind mode are not identical；E-th of first subdatas under replacement f kind mode are updated with e-th of second subdatas under f kind mode, to re-execute the step of e-th of first subdatas under the above-mentioned kind mode according to f obtain the abstract semantic description of e-th of digital signal from storage system.

In some embodiments of first aspect, if not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, determine multiple second subdatas, from e-th of first subdatas under the f mode then to replace e-th of first subdata with each of the multiple second subdata second subdata and to re-execute the step of e-th of first subdata under the above-mentioned kind mode according to f obtains the abstract semantic description of e-th of digital signal from the storage system.

In above-described embodiment, it is possible to prevente effectively from the first selected subdata can not inquire the corresponding abstract semantic description stored in storage system, and suitably change the range of the first subdata, It is retrieved again, to improve inquiry accuracy rate.

In some embodiments of first aspect, E digital signal under the one-to-one f kind mode of E feature vector obtained under f kind mode, it include: the potential pulse spike sequence for obtaining e-th of feature vector of the f kind mode using impulsive neural networks according to e-th of feature vector of the f kind mode；E-th of digital signal of the f kind mode is determined according to the potential pulse spike sequence of e-th of feature vector of the f kind mode.

Second aspect of the present invention provides a kind of data query system and determines method, comprising:

Firstly, obtaining the input data of M kind mode, M is positive integer；And determine m third subdata from the input data of i-th kind of mode, i successively take section (0, M] positive integer, m is positive integer；Also need m feature vector of the one-to-one i-th kind of mode of m third subdata of determining i-th kind of mode；And then determine m digital signal of the one-to-one i-th kind of mode of m feature vector of i-th kind of mode；Then determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode；Finally, saving and the one-to-one m abstract semantic description of the respective m digital signal of M kind mode within the storage system, wherein index of the digital signal of each mode as corresponding abstract semantic description.

In the present embodiment, storage system is determined according to abstract semantic description, and the index of the abstract semantic description stored in storage system is digital signal corresponding with the abstract semantic description, when needing to retrieve data, it only need to be that index obtains abstract semantic description corresponding with the digital signal directly from storage system according to the digital signal of data to be checked, it carries out retrieving the required time using storage system so as to be effectively reduced, has better met the requirement of real-time.

In some embodiments of second aspect: determining the i-th concordance list according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively the m digital signal of i-th kind of mode, and m index entry in the i-th concordance list is for corresponding m abstract semantic description in index storage system.

In some embodiment of second aspect, the i-th concordance list is determined according to m digital signal of i-th kind of mode, is specifically included: judge storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；If comprising, n-th of digital signal of i-th kind of mode this index entry is set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, n successively take section (0, m] positive integer.

The i-th concordance list is determined according to m digital signal of i-th kind of mode, is specifically included: judgement storage System whether include i-th kind of mode n-th of digital signal abstract semantic description；If not including, n-th of digital signal of i-th kind of mode this index entry is then set as in the i-th concordance list the index of the abstract semantic description of n-th of digital signal of i-th kind of mode, and saves the abstract semantic description of n-th of digital signal of i-th kind of mode in storage system.

In the present embodiment, the storage item that can be stored to storage system is updated in real time, so that storage system is more complete.

In some embodiments of second aspect, determine m digital signal of the one-to-one i-th kind of mode of the m feature vector of i-th kind of mode, it include: n-th of feature vector according to i-th kind of mode, the potential pulse spike sequence of n-th of feature vector of i-th kind of mode is obtained using impulsive neural networks, n successively take section (0, m] positive integer；The digital signal of n-th of feature vector of i-th kind of mode is determined according to the potential pulse spike sequence of n-th of feature vector of i-th kind of mode.

Third aspect present invention provides a kind of data query device, comprising:

Receiving module includes the data to be checked under F kind mode for receiving inquiry request, in inquiry request, and F is positive integer；

Determining module, for determining the first subdatas of the E under f kind mode according to the data to be checked under f kind mode, f successively take section (0, F] positive integer, E is positive integer；

Determining module is also used to determine E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode；

Determining module is also used to determine E digital signal under the one-to-one f kind mode of E feature vector under f kind mode；

Obtain module, for from obtaining corresponding with e-th of digital signal under f kind mode e-th of abstract semantic description in storage system, e successively take section (0, E] positive integer.

Optionally, in terms of from acquisition e-th of abstract semantic description corresponding with e-th of digital signal under f kind mode in storage system, obtain module, for the matched index entry of e-th of digital signal under searching in the f concordance list under f kind mode with f kind mode, according to matched index entry from the abstract semantic description for obtaining e-th of digital signal under f kind mode in storage system.

Optionally, device further include: judgment module:

Judgment module, for judge whether to find from the f concordance list under f kind mode with the matched index entry of e-th of digital signal under f kind mode,

If do not found from the f concordance list under f kind mode with the matched index entry of e-th of digital signal under f kind mode,

Determining module, for redefining e-th of second subdatas under f kind mode from the data to be checked under f kind mode, wherein e-th of second subdatas and e-th of first subdatas under f kind mode are not identical；

Determining module, it is also used to update e-th of first subdatas under replacement f kind mode with e-th of second subdatas under f kind mode, to re-execute the step of e-th of first subdatas under the above-mentioned kind mode according to f obtain the abstract semantic description of e-th of digital signal from storage system.

Optionally, device further include: judgment module:

Determining module, for determining multiple second subdatas from e-th of first subdatas under f mode, to replace e-th of first subdatas with the second subdata of each of multiple second subdatas and to re-execute the step of e-th of first subdatas under the above-mentioned kind mode according to f obtain the abstract semantic description of e-th of digital signal from storage system.

Optionally, in terms of determining E digital signal under the one-to-one f kind mode of E feature vector under f kind mode, determining module is used for:

According to e-th of feature vector of f kind mode, the potential pulse spike sequence of e-th of feature vector of f kind mode is obtained using impulsive neural networks；

E-th of digital signal of f kind mode is determined according to the potential pulse spike sequence of e-th of feature vector of f kind mode.

Fourth aspect present invention provides a kind of data query system determining device characterized by comprising

Module is obtained, for obtaining the input data of M kind mode, M is positive integer；

Determining module, for determining m third subdata from the input data of i-th kind of mode, i successively take section (0, M] positive integer, m is positive integer；

Determining module is also used to determine m feature vector of the one-to-one i-th kind of mode of m third subdata of i-th kind of mode；

Determining module is also used to determine the one-to-one i-th kind of mould of m feature vector of i-th kind of mode M digital signal of state；

Determining module, be also used to determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode；

Preserving module, for saving and the one-to-one m abstract semantic description of the respective m digital signal of M kind mode within the storage system, wherein index of the digital signal of each mode as corresponding abstract semantic description.

Optionally, determining module, it is also used to determine the i-th concordance list according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively the m digital signal of i-th kind of mode, and m index entry in the i-th concordance list is for corresponding m abstract semantic description in index storage system.

Optionally, device further include: judgment module, judgment module be used for judge storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If comprising, determining module, the index of the abstract semantic description of n-th of digital signal for n-th of digital signal of i-th kind of mode this index entry to be set as to i-th kind of mode in the i-th concordance list, n successively take section (0, m] positive integer.

If not including, determining module, for n-th of digital signal of i-th kind of mode this index entry to be set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, and the abstract semantic description of n-th of digital signal of i-th kind of mode is saved in storage system.

Optionally, in terms of the m digital signal of the one-to-one i-th kind of mode of m feature vector for determining i-th kind of mode,

Determining module obtains the potential pulse spike sequence of n-th of feature vector of i-th kind of mode using impulsive neural networks for n-th of feature vector according to i-th kind of mode, n successively take section (0, m] positive integer；

The digital signal of n-th of feature vector of i-th kind of mode is determined according to the potential pulse spike sequence of n-th of feature vector of i-th kind of mode.

Fifth aspect present invention provides a kind of data query device, and data query device includes processor and the memory that is of coupled connections with processor；

Memory stores computer instruction；

Processor reads and executes the computer instruction of memory storage, so that data query device executes the data query method such as any one of first aspect.

Sixth aspect present invention provides a kind of data query system determining device, including processor and the memory being of coupled connections with processor；

Memory stores computer instruction；

Processor reads and executes the computer instruction of memory storage, so that data query system determining device executes the data query system of any one of second aspect such as and determines method.

Seventh aspect present invention provides a kind of computer-readable medium, for storing computer program, the computer program includes the instruction for executing the data query method in the arbitrarily possible design of first aspect or first aspect, or the instruction of the data query method in the arbitrarily possible design for executing first aspect.

Eighth method of the present invention provides a kind of computer-readable medium, for storing computer program, the computer program includes the instruction that method is determined for executing the data query system in the arbitrarily possible design of second aspect or second aspect, or the data query system in the arbitrarily possible design for executing first aspect determines the instruction of method.

In the embodiment of the present invention, inquiry request is received first, then, determines the E under f kind mode the first subdatas according to the data to be checked under f kind mode, in turn, E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode is determined；The further E digital signal determined under the one-to-one f kind mode of E feature vector under f kind mode；Finally, from e-th of abstract semantic description corresponding with e-th of digital signal under f kind mode is obtained in storage system.Wherein, since the index of storage system is e-th of digital signal under f kind mode, so as to directly be retrieved storage system to obtain abstract semantic description corresponding with the digital signal according to the digital signal of data to be checked, to effectively reduce the retrieval required time, the requirement of real-time has been better met.

Detailed description of the invention

In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, a brief description of the drawings needed to be used in the description of the embodiments or the prior art will be given below, apparently, drawings in the following description are some embodiments of the invention, for those of ordinary skill in the art, without any creative labor, others be can also be obtained according to these attached drawings Attached drawing.

Fig. 1 show the flow chart of the data query method provided in the prior art；

Fig. 2 show the flow diagram that data query system provided in an embodiment of the present invention determines method；

Fig. 3 show a kind of optional implementation of step 204；

Fig. 4 show the illustration of abstract semantic description；

Fig. 5 show the schematic diagram one of concordance list and storage system in the embodiment of the present invention；

Fig. 6 show the schematic diagram two of concordance list and storage system in the embodiment of the present invention；

Fig. 7 show the schematic diagram three of concordance list and storage system in the embodiment of the present invention；

Fig. 8 show the flow chart of data query method provided in an embodiment of the present invention；

Fig. 9 show the structural schematic diagram one of data query device provided in an embodiment of the present invention；

Figure 10 show the structural schematic diagram two of data query device provided in an embodiment of the present invention；

Figure 11 show the structural schematic diagram one of data query system determining device provided in an embodiment of the present invention；

Figure 12 show the structural schematic diagram two of data query system determining device provided in an embodiment of the present invention；

Figure 13 show the structural schematic diagram three of data query device provided in an embodiment of the present invention；

Figure 14 show the structural schematic diagram three of data query system determining device provided in an embodiment of the present invention.

Specific embodiment

With social progress, great variation has occurred in the application field of various technologies, so that various technologies are not limited to application field single in the past, such as in the past for scientific algorithm parallel computation also used in recent years some complexity cloud service apply and mobile terminal on Intelligent treatment application in, such as music and voice identification Shazam and Siri, image and video analysis Picasa and Flickr, online navigate.

The algorithm core of Intelligent treatment application is neural network.Neural network is described below, the research of neural network includes two class of neural network and artificial neural network that biology inspires.Artificial neural network is one kind of machine learning method, and representational in the recent period is deep neural network and convolutional neural networks.The neural network that biology inspires is to establish a kind of suitable model come from biology, and to help to understand how neural network and brain work, representational is impulsive neural networks.

As Intelligent treatment applies the increasingly promotion of status, Intelligent treatment speed can be obviously improved, reduce the neural network processor of Intelligent treatment energy consumption as industry focus of attention.And the classification according to neural network, neural network processor is corresponding, and also there are two types: one is the Hardware for Artificial Neural Networks accelerators of Machine oriented study, and another kind is neuromorphic processor.The former includes being based on graphics processor (Graphics Processing Unit, neural network hardware referred to as are as follows: GPU) accelerates platform, is based on field programmable gate array (Field Programmable Gate Array, the neural network accelerator of convolutional neural networks accelerator referred to as are as follows: FPGA) and the specific integrated circuit (Application Specific Integrated Circuit, referred to as are as follows: ASIC) using customed circuit.And the latter is based on the neural network close to biological characteristic, therefore industry thinks that the impulsive neural networks hardware of biology inspiration is only the foundation stone of the real strong artificial intelligence of building.

Specifically, pulsed discharge is concept general in the neural network that biology inspires.Information entrained by single input stimulus, is transmitted to subsequent neuron by a series of pulse.Its coding mode includes two classes, and one kind is more traditional working method, that is, is encoded information onto the discharge frequency of pulse, i.e. frequency coding (Rate Coding)；Another kind of is the precise time for emphasizing single pulse electric discharge, i.e. sequential coding (Temporal Coding).In the impulsive neural networks hardware realization that biology inspires, currently used method is that the building of neural network, the method using nonvolatile memory or the accelerator using conventional complementary metal-oxide semiconductor (MOS) (Complementary Metal Oxide Semiconductor, referred to as are as follows: CMOS) circuit design are realized using memristor (Memristor).

From the point of view of the development of industry, although in a short time, accelerator based on conventional CMOS circuit design in neural network is a kind of feasible technical solution, but from the point of view of the high-level semantic analysis demand of practical application, existing depth learning technology is difficult to come out two different picture recognitions with identical high-level semantic by layer-by-layer feature extraction；Meanwhile from the point of view of hardware configuration, impulsive neural networks hardware has very strong asynchronism, and only some circuit is executing entire hardware platform most of the time, and energy consumption can be far below existing Hardware for Artificial Neural Networks.The impulsive neural networks hardware that thus biology inspires will be the development trend of Intelligent treatment computation platform, wherein class brain/neuron architecture processor becomes an important development branch and trend in the impulsive neural networks hardware architecture that biology inspires.

Currently, academia and industry be directed to class brain chip/neuron chip the study found that the key hardware challenge of its architecture design first is that following storage system.The technical solution of existing storage system, is all based on memory address index storage system, i.e., by retrieving the memory address of data storage to be checked in CAM, then from be to read data to be checked in the storage system indexed with memory address.But During searching data to be checked by above-mentioned storage system, need to carry out quadratic search, therefore it is larger to be delayed, and is unable to satisfy the requirement of real-time.

And allocation index storage system requires pre-programmed based on memory, therefore can effectively work in the traditional computer architectural framework of pre-programmed；And since class brain chip/neuron chip is the impossible complete pre-programmed of entire implementation procedure by impulse stimulation autonomous operation, therefore based on memory, allocation index storage system is not particularly suited for the access of class brain chip/neuron chip storage system.

Based on the above issues, the present invention proposes a kind of storage system based on content indexing, is directly index with content, retrieves to storage system, the time needed for without passing through quadratic search, effectively reducing retrieval, has better met the requirement of real-time.

Data query method provided by the invention, can be applied to include Intelligent treatment application field, such as: robot, automatic Pilot or Internet of Things etc., more specifically, in multi-modal storage system applied to the intelligent chip in Intelligent treatment application, these intelligent chips can be with are as follows: class brain processor chips, neuron chip or neuromorphic chip.

Technical solution of the present invention is described in detail with specifically embodiment below.These specific embodiments can be combined with each other below, and the same or similar concept or process may be repeated no more in some embodiments.

Before introducing data query method provided by the invention, the present embodiment introduces the determination method of the data query system for data query method provided by the invention first.

Fig. 2 show the flow diagram that data query system provided in an embodiment of the present invention determines method, as shown in Fig. 2, the method for the present embodiment includes:

Step 201, the input data for obtaining M kind mode, wherein M is positive integer.

Wherein, M kind mode refers to senses of touch data such as the voice data such as the different type of input data, such as the vision datas such as image and video, natural language, temperature and other environmental parameters, etc..

Step 202 determines m third subdata from the input data of i-th kind of mode, wherein i successively take section (0, M] positive integer, wherein m is positive integer.

A kind of optional implementation of step 202 are as follows: determine m identification region (such as the Region of Interest of the input data of i-th kind of mode, referred to as are as follows: ROI), then, each identification region can correspond to a third subdata, m third subdata is obtained at this time, namely, each third subdata corresponds to an identification region of input data, each identification region corresponds to a feature of input data, at this point, a feature of the corresponding as input data of each third subdata

In a kind of achievable mode of the invention, the identification region of input data can be determined according to such as under type:

According to scheduled window size and predefined starting point rule, chooses starting point and identification region is determined according to window size, each window is an identification region.Wherein, the size of window can pre-define, and can also be arranged according to actual needs by user；The rule of starting point can equally pre-define, and can also be arranged according to actual needs by user, simplest starting point rule is to randomly select.

For different mode, the definition of window and starting point also has different meanings, and such as image data, window refers to that coordinates regional, starting point are coordinates；For the data of the having times sequence such as video or voice, window then refers to that period, starting point are the moment；The data such as the text for not time sequencing, window refer to that length, starting point are displacements.

Such as: when voice data when a length of 2 small when input data is, starting point is the 1st second, and window is 5 seconds, then extracts the voice data of every five seconds to obtain multiple identification regions since the 1st second, to obtain multiple third subdatas.

Step 203, m feature vector for determining the one-to-one i-th kind of mode of the m third subdata of i-th kind of mode.

In this step, the feature in third subdata can be extracted；The corresponding vector of those features is the feature vector of above-mentioned third subdata.

By taking third subdata is picture as an example, the feature of third subdata can be at this time are as follows: color characteristic, textural characteristics, shape feature or spatial relation characteristics etc., then the feature vector of third subdata is also just the corresponding vector of color characteristic, the corresponding vector of textural characteristics, the corresponding vector of shape feature or the corresponding vector of spatial relation characteristics accordingly.

Step 204, m digital signal for determining the one-to-one i-th kind of mode of the m feature vector of i-th kind of mode.

It is illustrated in figure 3 a kind of optional implementation of step 204, as shown in Figure 3: it is illustrated by taking the 1st feature vector of i-th kind of mode as an example in the present embodiment, specific:

A: the 1st feature vector of i-th kind of mode is input to impulsive neural networks；

B: impulsive neural networks determine potential pulse spike (Spike) sequence of the 1st feature vector of i-th kind of mode；

C: it is determined according to the potential pulse spike sequence of the 1st feature vector of i-th kind of determining mode The digital signal of 1st feature vector of i-th kind of mode.

Fig. 3 additionally provides a kind of achievable mode of step C, it is specific: potential pulse spike sequence being converted into discrete sequences of pulsed signals using predefined sequence of time slots, namely, potential pulse spike sequence is divided into n sub- potential pulse spike sequences, the duration of the duration of every sub- potential pulse spike sequence time slot corresponding with the sub- potential pulse spike sequence is identical, for every sub- potential pulse spike sequence, its corresponding signal is converted into digital coding according to default transformation rule, wherein, default transformation rule can be with are as follows: when amplitude is within the scope of some corresponding digitally coded 1, when amplitude is within the scope of another corresponding digitally coded 0, then digital coding is sorted by the serial number of time slot, constitute n digital codings, to which the 1st feature vector for obtaining i-th kind of mode is corresponding Digital signal.Wherein, the size of n is determined by time span, the total time length of each time slot.The circuit of above-mentioned function can may be implemented by building, to complete above-mentioned conversion process, it can directly be realized using analog-digital converter in actual application, that is, the process for determining digital signal according to potential pulse spike sequence is directly realized by, without building circuit with the same function.

Step 205, determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode.

In order to realize the data correlation of cross-module state, need for the feature of the multiple modalities of the same entity object to be described with the same abstract semanteme, thus also corresponding with Biological Principles to the greatest extent.

By taking feature 1 as an example: uniformly using the feature of the feature 1 of i-th kind of mode to state the digital signal of the feature 1 of every kind of mode in the M-1 kind mode in addition to i-th kind of mode, wherein, the feature of the feature 1 of i-th kind of mode is referred to as the abstract semantic description of this feature 1, and i-th kind of mode be characterized in it is relevant to the third subdata of i-th kind of mode.

If i-th kind of mode is visual modalities, then the feature of the feature 1 of i-th kind of mode can be the edge contour of the feature vector of the corresponding third subdata of feature 1 of i-th kind of mode.

As shown in Figure 4, the input of two mode of vision and the sense of hearing for the same feature 1, namely input is the corresponding third subdata 1 of feature 1 under visual modalities and the corresponding third subdata 2 of the feature under audio modality 1, the feature vector of 1 third subdata 2 of third subdata is obtained first, and the digital signal a of third subdata 1 and the digital signal b of third subdata 2 are obtained by impulsive neural networks；Assuming that the edge contour of the corresponding feature vector of third subdata 1 under using visual modalities as abstract semantic description, then in the scope of this mode of the sense of hearing, the digital signal b (digital signal of the third subdata under audio modality) of third subdata 2 is directed toward third subdata 1 The edge contour of corresponding feature vector.

Step 206 saves and the one-to-one m abstract semantic description of the respective m digital signal of M kind mode within the storage system, wherein index of the digital signal of each mode as corresponding abstract semantic description.

When needing to inquire data using storage system, digital signal corresponding with data are inquired only need to be obtained, is then that index retrieves abstract semantic description corresponding with the index within the storage system with digital signal.

Further, the method for above-mentioned determining data query system further include:

The i-th concordance list is determined according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively the m digital signal of i-th kind of mode, and m index entry in the i-th concordance list is for corresponding m abstract semantic description in index storage system.

Such as: the digital signal of the 1st feature vector in i-th kind of mode determined according to the above method are as follows: 11001010, the digital signal of the 2nd feature vector in i-th kind of mode are as follows: 10101010, ..., the digital signal of n-th of feature vector in i-th kind of mode are as follows: 10101101, then the i-th concordance list as shown in Table 1 can be generated according to above- mentioned information:

I-th concordance list

In this step, the concordance list of different mode can be identified, such as the concordance list of mark visual modalities are as follows: the 1st concordance list of the 1st mode, to which the concordance list that the mark of mode quickly finds out current mode to be checked from the concordance list of multiple modalities when being inquired using the data query system, can be corresponded to according to data to be checked.

According to above-mentioned method, concordance list and storage system as described in Figure 5 can be obtained, as shown in Figure 5:

The digital signal of 1st feature vector of (i-1)-th kind of mode are as follows: 11001011, the digital signal of the 2nd feature vector of (i-1)-th kind of mode are as follows: 11001010 ..., the of (i-1)-th kind of mode The digital signal of n feature vector are as follows: 11011011, and the corresponding n-th abstract semantic description of digital signal of the 1st feature vector of (i-1)-th kind of mode, the corresponding 2nd abstract semantic description of the digital signal of 2nd feature vector of (i-1)-th kind of mode, the corresponding 1st abstract semantic description of the digital signal of n-th of feature vector of (i-1)-th kind of mode, to the (i-1)-th concordance list of available (i-1)-th kind of mode and the index relative of storage system, the arrow in figure is the corresponding relationship indexed.

Further, the present embodiment provides the implementations that a kind of m digital signal according to i-th kind of mode determines the i-th concordance list:

Judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If comprising, n-th of digital signal of i-th kind of mode this index entry is set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, the n successively take section (0, m] positive integer.

If not including, n-th of digital signal of i-th kind of mode this index entry is then set as in the i-th concordance list the index of the abstract semantic description of n-th of digital signal of i-th kind of mode, and saves the abstract semantic description of n-th of digital signal of i-th kind of mode within the storage system.

Continue according to above-mentioned example, assuming that having saved storage system as shown in Figure 5 in system, storage system is determined according to the input data of other mode in addition to i-th kind of mode, after obtaining the 1st digital signal 11001010 of i-th kind of mode, judge in storage system whether the abstract semantic description of the 1st digital signal 11001010 comprising i-th kind of mode, if, the abstract semantic description of 1st digital signal 11001010 of i-th kind of mode is the 2nd abstract semantic description, as shown in Figure 5, the abstract semantic description (the 2nd abstract semantic description) of the 1st digital signal 11001010 of i-th kind of mode is contained in storage system, then in the i-th concordance list by the 1st digital signal 11001010 of i-th kind of mode this Index entry is set as the index of the 2nd abstract semantic description of storage system storage, as shown in Figure 6.

If the abstract semantic description of the 6th digital signal 10101101 of i-th kind of mode is the (n+1)th abstract semantic description in the i-th concordance list, since there is no include the (n+1)th abstract semantic description in storage system, the (n+1)th abstract semantic description is then saved within the storage system, and the 6th digital signal this index entry of i-th kind of mode is set as to the index of the store in storage system (n+1)th abstract semantic description in the i-th concordance list, as shown in Figure 7.

Optionally, it is above-mentioned judge in storage system whether include i-th kind of mode n-th of digital signal Abstract semantic description after, if in storage system not including the abstract semantic description of n-th of digital signal of i-th kind of mode, in order to more accurately determine storage system, in the index for the abstract semantic description for executing above-mentioned n-th of digital signal that n-th of digital signal of i-th kind of mode this index entry is set as to i-th kind of mode in the i-th concordance list, and before the abstract semantic description that storage system saves n-th of digital signal of i-th kind of mode, can also include:

Redefine multiple 4th subdatas of the input data of i-th kind of mode, then the third subdata for replacing i-th kind of above-mentioned mode is updated, to each 4th subdata to re-execute the method and step that the above-mentioned third subdata according to i-th kind of mode determines storage system and concordance list.And the number for redefining multiple 4th subdatas of the input data of i-th kind of mode can be preset, and can also be determined in real time according to demand.

Wherein, redefine a kind of achievable mode of multiple 4th subdatas of the input data of i-th kind of mode are as follows: reduce to identification region in step 202 according to default rule, default rule can reduce identification region, two points of formulas diminution identification regions etc. of reducing by half to be random.The present invention is not defined the rule.

Such as, " this is a chair " is inputted for voice, the identification region got for the first time may be " this is one ", reduce by half without the feature of key message " chair ", thus by identification region, until " chair " can be got.

Redefine the achievable mode of another kind of multiple 4th subdatas of the input data of i-th kind of mode are as follows: to each third subdata of i-th kind of mode according to preset rules determine in determine multiple 4th subdatas of each third subdata, also a third subdata is divided into multiple 4th subdatas, for example divided equally, trisection etc..

Such as: if third subdata be audio data, and third subdata when it is 6 seconds a length of, preset rules be to third subdata carry out trisection, then third subdata can be obtained to 32 seconds the 4th subdatas as unit of the time.

Wherein, before executing step 201, the input data of i-th kind of mode can also be pre-processed, such as ambient noise can be removed using extracting profile, the methods of time threshold and zero-crossing rate being arranged, purpose is to avoid the same object due to the difference of the external conditions such as illumination or noise, caused by recognition result it is different.

Data query system provided in this embodiment determines method, and obtained data query system includes concordance list and storage system, and the index entry in concordance list is the digital signal of i-th kind of mode, and in storage system The index of the abstract semantic description of the digital signal of i-th kind of mode of storage is the digital signal of corresponding i-th kind of mode in concordance list, to when carrying out data query using the data query system, the digital signal that the digital signals of data to be checked can be obtained directly need to only be obtained and retrieve abstract semantic description corresponding with the digital signal from storage system for index, time needed for effectively reducing retrieval, the requirement of real-time is better met.

Fig. 8 show the flow chart of data query method provided in an embodiment of the present invention, as shown in figure 8, the method for the present embodiment is applied in the multi-modal storage system of intelligent chip, the method for the present embodiment may include:

Step 401: receiving inquiry request, include the data to be checked under F kind mode in above-mentioned inquiry request, wherein F is positive integer.

Wherein, the data to be checked under F kind mode refer to the F seed type of data to be checked, such as: senses of touch modal data such as the speech modalities data such as the visual modalities such as image and video data, natural language, temperature and other environmental parameters etc..

Such as: data to be checked are apple, then may include the apple (picture that a width includes apple) under visual modalities, the apple (one section of audio including the pronunciation of two words of apple) under speech modality or apple (sentence including two Chinese characters of apple) under text mode etc. in inquiry request.

Step 402: determining the first subdatas of the E under f kind mode according to the data to be checked under f kind mode, wherein f successively take section (0, F] positive integer, E is positive integer.

The method that the first subdata is determined in this step is identical as the method for third subdata determining in step 201, and details are not described herein again.

Step 403: determining E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode；

Step 404: determining E digital signal under the one-to-one f kind mode of E feature vector under f kind mode；

The method for determining feature vector and digital signal is identical as step 203 and step 204, and details are not described herein again.

Step 405: from obtaining corresponding with e-th of digital signal under f kind mode e-th of abstract semantic description in storage system, e successively take section (0, E] positive integer.

The storage system is the above-mentioned storage system for determining that method is determined according to data-storage system, wherein The corresponding index of the abstract semantic description stored in storage system is digital signal, then correspondingly, can be that index finds corresponding abstract semantic description within the storage system according to obtained digital signal.

If concordance list is also obtained according to the method for above-mentioned determination data query system, so, from acquisition e-th of abstract semantic description corresponding with e-th of digital signal under f kind mode in storage system, it include: the corresponding index entry of e-th of digital signal searched from the f concordance list under f kind mode under f kind mode, index due to the index entry as the corresponding abstract semantic description of e-th of digital signal under the f kind mode stored in storage system, therefore when can directly retrieve abstract semantic description corresponding with the index entry from storage system after finding the index entry in concordance list.

Further, if not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, e-th of second subdatas under f kind mode are then redefined from the data to be checked under f kind mode, wherein, e-th of second subdata and e-th of first subdata under f kind mode be not identical；

The step of updating e-th of first subdata under replacement f kind mode with e-th of second subdata under f kind mode, obtain the abstract semantic description of e-th of digital signal from the storage system so as to e-th of first subdata re-executed under the above-mentioned kind mode according to f.

Obtained in this step the second subdata method and above-described embodiment obtain the method for the 4th subdata it is identical, details are not described herein again.

Wherein, the second subdata and the first subdata may exist intersection, such as Data duplication rate is greater than 70% intersection.The data that is, the second subdata and the first subdata are had any different, but distinguishes data, with respect to repeated data, ratio is smaller；It is equivalent to, the second subdata is the adjustment to the first subdata；In this way, when corresponding abstract semantic description can not be found based on the first subdata, make appropriate partial adjustment and obtains the second subdata, corresponding abstract semantic description is searched further according to the second subdata, and so on, lookup can be terminated finding corresponding abstract semantic description, or lookup can be terminated when still finding corresponding abstract semantic description not according to newest second subdata after carrying out prediction number adjustment.

Further, the method also includes: if not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, determine multiple second subdatas, from e-th of first subdatas under the f mode then to use the multiple second subdata Each of second subdata replace e-th of first subdata and re-execute the step of e-th of first subdata under the above-mentioned kind mode according to f obtains the abstract semantic description of e-th of digital signal from the storage system.

The method that the second subdata is obtained in this step is same as the previously described embodiments, and details are not described herein again.

Further, E digital signal under the one-to-one f kind mode of E feature vector under above-mentioned acquisition f kind mode, it include: the potential pulse spike sequence for obtaining e-th of feature vector of f kind mode using impulsive neural networks according to e-th of feature vector of f kind mode；E-th of digital signal of f kind mode is determined according to the potential pulse spike sequence of e-th of feature vector of f kind mode.

The implementation of this step is same as the previously described embodiments, and details are not described herein again.

In a kind of achievable mode, the storage address of the abstract semantic description stored in storage system can be the corresponding digital signal of the corresponding feature vector of abstract semantic description stored in the storage item.

After feature vector 1 has been determined, using the digital signal of feature vector 1 as storage address, abstract semantic description corresponding with storage address is determined from storage system, wherein the corresponding storage address of Storage Item of the abstract semantic description stored in storage system is the digital signal of feature vector 1.

On the basis of the above embodiments, it is also necessary to the corresponding f concordance list of f mode is determined from the concordance list of multiple and different mode.

After determining at least one first subdata of the data to be checked under f kind mode, can be determined according to first subdata corresponding to the subdata be which kind of mode data to be checked, and then from multiple Concordance list corresponding with the mode is determined in the concordance list of different modalities.

Such as: it can be used which kind of mode is specific mark be shown to be, and corresponding concordance list, such as: if indicating visual modalities with 1, then after obtaining the first subdata, can determine be corresponding to current subdata visual modalities data to be checked, namely the first subdata at this time is corresponding is identified as 1, and the 1st concordance list of the 1st mode is then searched in the concordance list of multiple modalities.

The above method is a kind of citing, and other modes can be used in practical applications and determine corresponding concordance list.

Further, in the concordance list from the multiple and different mode prestored before the corresponding f concordance list of determination f mode further include: the concordance list and storage system for determining multiple and different mode, the concordance list for specifically how determining multiple and different mode and storage system are in the description referring to above-described embodiment.

The present embodiment provides one kind to obtain the concordance list of multiple and different mode and the specific method of storage system for the first time: obtaining the concordance list and storage system of multiple and different mode in the present embodiment using impulsive neural networks (Spike Neural Network, referred to as are as follows: SNN).

Specifically, first, use the input data of SNN processing multiple modalities, obtain multiple feature vectors of the input data of each of them mode, each feature vector is converted into a series of potential pulse spike sequences, the pulse train for being able to reflect different characteristic vector is generated, such as: gray value of image can be converted to by pulse train using uniform enconding mode.Then potential pulse spike sequence back obtained passes through analog-digital converter, it is converted into potential pulse peak voltage signal, sequences of pulsed signals is obtained, and sequences of pulsed signals is converted into corresponding digital coding (digital signal), concordance list is obtained according to the digital signal of conversion；Then it determines the corresponding abstract semantic description of the digital signal of each feature vector, and storage system is generated according to the corresponding abstract semantic description of each feature vector.

In order to improve treatment effeciency, multiple SNN can be used in the present embodiment, one of SNN handles a kind of input data of mode, but calculation amount and energy consumption are all relatively higher；It can also use the input data of a SNN concurrent processing multiple modalities in the present embodiment, but its performance can relative reduction.The number of SNN can be specifically adjusted according to the actual situation.

The present embodiment provides a kind of examples for realizing the data query method in the present invention, in the present embodiment, scene reproduction and information retrieval based on content indexing storage system are realized using SNN.

First example: by taking the description of the voice of input " Boston marathon explosion " as an example.

Identify the feature vector in the voice of input, each feature vector corresponds to different local features, such as the determining corresponding feature of feature vector is respectively as follows: " Boston ", " marathon " and " explosion ", determine the digital signal of each feature vector, using those digital signals as search index storage system, and pass through the information retrieval of cross-module state within the storage system, obtain related content.

Second example: by taking the picture or video of input " Boston marathon explosion " as an example.

Identify the feature vector in the picture or video of input, each feature vector corresponds to different local features, such as the determining corresponding feature of feature vector is respectively as follows: " explosion " and " marathon " two features, determine the digital signal of each feature vector, using those digital signals as search index storage system, the abstract semantic description of each feature vector is obtained, and then identifies that scene is " marathon explosion "；It in conjunction with the information of other mode, such as the news report content of text or voice, is associated according to data such as time, positions, obtains the identification of overall scenario and the abstract semantic description of scene.

Fig. 9 show the structural schematic diagram one of data query device provided in an embodiment of the present invention, as shown in figure 9, device includes:

Figure 10 show the structural schematic diagram two of data query device provided in an embodiment of the present invention, as shown in Figure 10, on the basis of Fig. 9, above-mentioned device further include: judgment module；

As shown in Figure 10, device further include: judgment module:

Further, in terms of determining E digital signal under the one-to-one f kind mode of E feature vector under f kind mode, determining module is used for:

The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 8, and it is similar that the realization principle and technical effect are similar, and details are not described herein again.

Figure 11 show the structural schematic diagram one of data query system determining device provided in an embodiment of the present invention, and as shown in figure 11, above-mentioned device includes:

Module is obtained, for obtaining the input data of M kind mode, the M is positive integer；

Determining module, for determining m third subdata from the input data of i-th kind of mode, the i successively take section (0, M] positive integer, the m is positive integer；

The determining module is also used to determine m feature vector of the one-to-one i-th kind of mode of m third subdata of i-th kind of mode；

The determining module is also used to determine m digital signal of the one-to-one i-th kind of mode of m feature vector of i-th kind of mode；

The determining module, be also used to determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode；

Optionally, the determining module, it is also used to determine the i-th concordance list according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively m digital signal of i-th kind of mode, and m index entry in i-th concordance list is for indexing corresponding m abstract semantic description in the storage system.

Figure 12 show the structural schematic diagram two of data query system determining device provided in an embodiment of the present invention, and as shown in figure 12, on the basis of Figure 11, above-mentioned device includes: judgment module,

The judgment module be used for judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If comprising, the determining module, for n-th of digital signal of i-th kind of mode this index entry to be set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, the n successively take section (0, m] positive integer.

As shown in figure 12, described device further include: judgment module, the judgment module be used for judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If not including, the determining module, for n-th of digital signal of i-th kind of mode this index entry to be set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, and the abstract semantic description of n-th of digital signal of i-th kind of mode is saved in the storage system.

Optionally, in the m for the one-to-one i-th kind of mode of m feature vector for determining i-th kind of mode The aspect of a digital signal,

The determining module obtains the potential pulse spike sequence of n-th of feature vector of i-th kind of mode using impulsive neural networks for n-th of feature vector according to i-th kind of mode, the n successively take section (0, m] positive integer；

The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 2, and it is similar that the realization principle and technical effect are similar, and details are not described herein again.

Figure 13 show the structural schematic diagram three of data query device provided in an embodiment of the present invention, and as shown in Figure 10, the data query device includes processor and the memory that is of coupled connections with the processor；

The memory stores computer instruction；

The processor reads and executes the computer instruction of the memory storage, so that the data query device executes following steps:

Inquiry request is received, includes the data to be checked under F kind mode in the inquiry request, the F is positive integer；

Determine the first subdatas of the E under f kind mode according to the data to be checked under f kind mode, the f successively take section (0, F] positive integer, the E is positive integer；

Determine E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode；

Determine E digital signal under the one-to-one f kind mode of E feature vector under f kind mode；

From obtaining corresponding with e-th of digital signal under the f kind mode e-th of abstract semantic description in storage system, the e successively take section (0, E] positive integer.

It is described from corresponding with e-th of digital signal under the f kind mode e-th of abstract semantic description is obtained in storage system, include:

The matched index entry of e-th of digital signal under searching in the f concordance list under f kind mode with the f kind mode, according to matched index entry from the abstract semantic description for obtaining e-th of digital signal under the f kind mode in the storage system.

Further include:

If not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, e-th of second subdatas under f kind mode are then redefined from the data to be checked under f kind mode, wherein, e-th of second subdata and e-th of first subdata under f kind mode be not identical；

If further include: it is not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, determine multiple second subdatas, from e-th of first subdatas under the f mode then to replace e-th of first subdata with each of the multiple second subdata second subdata and to re-execute the step of e-th of first subdata under the above-mentioned kind mode according to f obtains the abstract semantic description of e-th of digital signal from the storage system.

E digital signal under the one-to-one f kind mode of E feature vector obtained under f kind mode, comprising:

According to e-th of feature vector of the f kind mode, the potential pulse spike sequence of e-th of feature vector of the f kind mode is obtained using impulsive neural networks；

E-th of digital signal of the f kind mode is determined according to the potential pulse spike sequence of e-th of feature vector of the f kind mode.

The device of the present embodiment can be used for executing the technical solution of embodiment of the method shown in Fig. 1, and it is similar that the realization principle and technical effect are similar, and details are not described herein again.

Figure 14 show the structural schematic diagram three of data query system determining device provided in an embodiment of the present invention, and as shown in figure 14, the data query system determining device includes processor and the memory that is of coupled connections with the processor；

The memory stores computer instruction；

The processor reads and executes the computer instruction of the memory storage, so that the data query system determining device executes following steps:

The input data of M kind mode is obtained, the M is positive integer；

Determine m third subdata from the input data of i-th kind of mode, the i successively take section (0, M] Positive integer, the m be positive integer；

Determine m feature vector of the one-to-one i-th kind of mode of the m third subdata of i-th kind of mode；

Determine m digital signal of the one-to-one i-th kind of mode of the m feature vector of i-th kind of mode；

Determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode；

It saves and the one-to-one m abstract semantic description of the respective m digital signal of M kind mode within the storage system, wherein index of the digital signal of each mode as corresponding abstract semantic description.

Further include:

The i-th concordance list is determined according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively m digital signal of i-th kind of mode, and m index entry in i-th concordance list is for indexing corresponding m abstract semantic description in the storage system.

The m digital signal according to i-th kind of mode determines the i-th concordance list, specifically includes:

If not including, n-th of digital signal of i-th kind of mode this index entry is then set as in the i-th concordance list the index of the abstract semantic description of n-th of digital signal of i-th kind of mode, and saves the abstract semantic description of n-th of digital signal of i-th kind of mode in the storage system.

M digital signal of the one-to-one i-th kind of mode of the m feature vector of i-th kind of mode of the determination, comprising:

According to n-th of feature vector of i-th kind of mode, obtain the potential pulse spike sequence of n-th of feature vector of i-th kind of mode using impulsive neural networks, the n successively take section (0, m] positive integer；

The term " includes " of description and claims of this specification and " having " and their any deformation, it is intended to cover and non-exclusive includes, such as, the process, method, system, product or equipment for containing a series of steps or units those of are not necessarily limited to be clearly listed step or unit, but may include other step or units being not clearly listed or intrinsic for these process, methods, product or equipment.

Those of ordinary skill in the art will appreciate that: realizing all or part of the steps of above method embodiment, this can be accomplished by hardware associated with program instructions, program above-mentioned can be stored in a computer readable storage medium, the program when being executed, executes step including the steps of the foregoing method embodiments；And storage medium above-mentioned includes: read-only memory (Read-Only Memory, referred to as are as follows: ROM), the various media that can store program code such as random access memory (Random Access Memory, referred to as are as follows: RAM), magnetic or disk.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention., rather than its limitations；Although present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that: it is still possible to modify the technical solutions described in the foregoing embodiments, or equivalent substitution of some or all of the technical features；And these are modified or replaceed, the range for technical solution of various embodiments of the present invention that it does not separate the essence of the corresponding technical solution.

Claims

A kind of data query method, which is characterized in that the described method includes:

Inquiry request is received, includes the data to be checked under F kind mode in the inquiry request, the F is positive integer；

Determine the first subdatas of the E under f kind mode according to the data to be checked under f kind mode, the f successively take section (0, F] positive integer, the E is positive integer；

Determine E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode；

Determine E digital signal under the one-to-one f kind mode of E feature vector under f kind mode；

From obtaining corresponding with e-th of digital signal under the f kind mode e-th of abstract semantic description in storage system, the e successively take section (0, E] positive integer.
The method according to claim 1, wherein described from acquisition e-th of abstract semantic description corresponding with e-th of digital signal under the f kind mode in storage system, comprising:

The matched index entry of e-th of digital signal under searching in the f concordance list under f kind mode with the f kind mode, according to matched index entry from the abstract semantic description for obtaining e-th of digital signal under the f kind mode in the storage system.
Method according to claim 1 or 2, which is characterized in that the method also includes:

If not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, e-th of second subdatas under f kind mode are then redefined from the data to be checked under f kind mode, wherein, e-th of second subdata and e-th of first subdata under f kind mode be not identical；

The step of updating e-th of first subdata under replacement f kind mode with e-th of second subdata under f kind mode, obtain the abstract semantic description of e-th of digital signal from the storage system so as to e-th of first subdata re-executed under the above-mentioned kind mode according to f.
Method according to claim 1 or 2, it is characterized in that, the method also includes: if not found from the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode, determine multiple second subdatas, from e-th of first subdatas under the f mode then to replace e-th of first subdata with each of the multiple second subdata second subdata and to re-execute e-th under the above-mentioned kind mode according to f first son Data obtain the step of abstract semantic description of e-th of digital signal from the storage system.
Method according to claim 1-4, which is characterized in that E digital signal under the one-to-one f kind mode of E feature vector obtained under f kind mode, comprising:

According to e-th of feature vector of the f kind mode, the potential pulse spike sequence of e-th of feature vector of the f kind mode is obtained using impulsive neural networks；

E-th of digital signal of the f kind mode is determined according to the potential pulse spike sequence of e-th of feature vector of the f kind mode.
A kind of data query system determines method characterized by comprising

The input data of M kind mode is obtained, the M is positive integer；

Determine m third subdata from the input data of i-th kind of mode, the i successively take section (0, M] positive integer, the m is positive integer；

Determine m feature vector of the one-to-one i-th kind of mode of the m third subdata of i-th kind of mode；

Determine m digital signal of the one-to-one i-th kind of mode of the m feature vector of i-th kind of mode；

Determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode；

It saves and the one-to-one m abstract semantic description of the respective m digital signal of M kind mode within the storage system, wherein index of the digital signal of each mode as corresponding abstract semantic description.
According to the method described in claim 6, it is characterized in that, the method also includes:

The i-th concordance list is determined according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively m digital signal of i-th kind of mode, and m index entry in i-th concordance list is for indexing corresponding m abstract semantic description in the storage system.
The method according to the description of claim 7 is characterized in that the m digital signal according to i-th kind of mode determines the i-th concordance list, specifically include:

Judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If comprising, n-th of digital signal of i-th kind of mode this index entry is set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, the n successively take section (0, m] positive integer.
Method according to claim 7 or 8, which is characterized in that the m digital signal according to i-th kind of mode determines the i-th concordance list, specifically includes:

Judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If not including, n-th of digital signal of i-th kind of mode this index entry is then set as in the i-th concordance list the index of the abstract semantic description of n-th of digital signal of i-th kind of mode, and saves the abstract semantic description of n-th of digital signal of i-th kind of mode in the storage system.
The method according to claim 6, which is characterized in that m digital signal of the one-to-one i-th kind of mode of the m feature vector of i-th kind of mode of the determination, comprising:

According to n-th of feature vector of i-th kind of mode, obtain the potential pulse spike sequence of n-th of feature vector of i-th kind of mode using impulsive neural networks, the n successively take section (0, m] positive integer；

The digital signal of n-th of feature vector of i-th kind of mode is determined according to the potential pulse spike sequence of n-th of feature vector of i-th kind of mode.
A kind of data query device, which is characterized in that described device includes:

Receiving module includes the data to be checked under F kind mode for receiving inquiry request, in the inquiry request, and the F is positive integer；

Determining module, for determining the first subdatas of the E under f kind mode according to the data to be checked under f kind mode, the f successively take section (0, F] positive integer, the E is positive integer；

The determining module is also used to determine E feature vector under the E one-to-one f kind mode of the first subdata under f kind mode；

The determining module is also used to determine E digital signal under the one-to-one f kind mode of E feature vector under f kind mode；

Obtain module, for from obtaining corresponding with e-th of digital signal under the f kind mode e-th of abstract semantic description in storage system, the e successively take section (0, E] positive integer.
Device according to claim 11, it is characterized in that, described in terms of obtain corresponding with e-th of digital signal under the f kind mode e-th of abstract semantic description in storage system, the acquisition module, for being from the storage according to matched index entry from lookup in the f concordance list under f kind mode and the matched index entry of e-th of digital signal under the f kind mode The abstract semantic description of e-th of digital signal under the f kind mode is obtained in system.
Device according to claim 11 or 12, which is characterized in that described device further include: judgment module:

The judgment module, for judge whether to find from the f concordance list under the f kind mode with the matched index entry of e-th of digital signal under the f kind mode,

If do not found from the f concordance list under f kind mode with the matched index entry of e-th of digital signal under the f kind mode,

The determining module, for redefining e-th of second subdatas under f kind mode from the data to be checked under f kind mode, wherein e-th of second subdata and e-th of first subdata under f kind mode be not identical；

The determining module, it is also used to the step of updating e-th of first subdata under replacement f kind mode with e-th of second subdata under f kind mode, obtaining the abstract semantic description of e-th of digital signal from the storage system so as to e-th of first subdata re-executed under the above-mentioned kind mode according to f.
Device according to claim 11 or 12, which is characterized in that described device further include: judgment module:

The judgment module, for judge whether to find from the f concordance list under the f kind mode with the matched index entry of e-th of digital signal under the f kind mode,

If do not found from the f concordance list under f kind mode with the matched index entry of e-th of digital signal under the f kind mode,

The determining module, for determining multiple second subdatas from e-th of first subdatas under the f mode, to replace e-th of first subdata with each of the multiple second subdata second subdata and to re-execute the step of e-th of first subdata under the above-mentioned kind mode according to f obtains the abstract semantic description of e-th of digital signal from the storage system.
The described in any item devices of 1-14 according to claim 1, which is characterized in that in terms of E digital signal under the one-to-one f kind mode of E feature vector under the determination f kind mode, the determining module is used for:

According to e-th of feature vector of the f kind mode, the potential pulse spike sequence of e-th of feature vector of the f kind mode is obtained using impulsive neural networks；

Described is determined according to the potential pulse spike sequence of e-th of feature vector of the f kind mode E-th of digital signal of f kind mode.
A kind of data query system determining device characterized by comprising

Module is obtained, for obtaining the input data of M kind mode, the M is positive integer；

Determining module, for determining m third subdata from the input data of i-th kind of mode, the i successively take section (0, M] positive integer, the m is positive integer；

The determining module is also used to determine m feature vector of the one-to-one i-th kind of mode of m third subdata of i-th kind of mode；

The determining module is also used to determine m digital signal of the one-to-one i-th kind of mode of m feature vector of i-th kind of mode；

The determining module, be also used to determine respectively with the one-to-one m abstract semantic description of the respective m digital signal of M kind mode；

Preserving module, for saving and the one-to-one m abstract semantic description of the respective m digital signal of M kind mode within the storage system, wherein index of the digital signal of each mode as corresponding abstract semantic description.
Device according to claim 16, it is characterized in that, the determining module, it is also used to determine the i-th concordance list according to m digital signal of i-th kind of mode, wherein, m index entry in i-th concordance list is respectively m digital signal of i-th kind of mode, and m index entry in i-th concordance list is for indexing corresponding m abstract semantic description in the storage system.
Device according to claim 17, which is characterized in that described device further include: judgment module, the judgment module be used for judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If comprising, the determining module, for n-th of digital signal of i-th kind of mode this index entry to be set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, the n successively take section (0, m] positive integer.
Device described in 7 or 18 according to claim 1, which is characterized in that described device further include: judgment module, the judgment module be used for judge the storage system whether include i-th kind of mode n-th of digital signal abstract semantic description；

If not including, the determining module, for n-th of digital signal of i-th kind of mode this index entry to be set as to the index of the abstract semantic description of n-th of digital signal of i-th kind of mode in the i-th concordance list, and the abstract of n-th of digital signal of i-th kind of mode is saved in the storage system Semantic description.
The described in any item devices of 6-19 according to claim 1, which is characterized in that in terms of the m digital signal of the one-to-one i-th kind of mode of m feature vector for determining i-th kind of mode,

The determining module obtains the potential pulse spike sequence of n-th of feature vector of i-th kind of mode using impulsive neural networks for n-th of feature vector according to i-th kind of mode, the n successively take section (0, m] positive integer；

The digital signal of n-th of feature vector of i-th kind of mode is determined according to the potential pulse spike sequence of n-th of feature vector of i-th kind of mode.
A kind of data query device, which is characterized in that the data query device includes processor and the memory that is of coupled connections with the processor；

The memory stores computer instruction；

The processor reads and executes the computer instruction of the memory storage, so that the data query device executes such as data query method described in any one of claim 1 to 5.
A kind of data query system determining device, which is characterized in that the data query system determining device includes processor and the memory that is of coupled connections with the processor；

The memory stores computer instruction；

The processor reads and executes the computer instruction of the memory storage, so that the data query system determining device is executed as the described in any item data query systems of claim 6 to 9 determine method.