WO2022025216A1 - Information processing device using compression data search engine, and information processing method therefor - Google Patents

Abstract

[Problem] To eliminate the shortcomings of inductive deep learning that requires an enormous amount of repetition and learning, and to enable reliable inference operations at higher speed and with lower power consumption. [Solution] Provided is an information processing device for general machine learning, the device comprising: a retrieval engine which includes a learning means that, in order to extract registration/retrieval data features, adds learning information to filtered data serving as a learning component, and registers the filtered data in a learning memory, a retrieval means for extracting the learning information by retrieving the learning component data, and a means for outputting correct learning information by a majority decision of the retrieved learning information; and/or a retrieval engine which includes a retrieval means for diffusion processing retrieval information and extracting learning information.

Description

Information processing equipment using a compressed data search engine and its information processing method

The present invention relates to an information processing apparatus using a compressed data search engine and an information processing method thereof, and in particular, information processing capable of performing general-purpose machine learning / inference and performing reliable inference operation at high speed and low power consumption. It relates to an apparatus, an information processing method, an information registration / extraction method, a face recognition feature extraction method, and a computer program.

In recent years, in the age of cloud computing, the amount of information processing that handles big data has increased. In particular, in artificial intelligence (AI: Artificial Intelligence) processing, high-speed access with high frequency is required from among big data. Historically, there are two methods for search engines, one is a hardware-based method using (CAM: Contensable Memory) and the other is a software-based method using a Hash function. The former is capable of high-speed search, but when the number of memory mats to be searched increases, all memory mats are accessed at the same time, so the larger the mat, the enormous power consumption, and it is used only for small-scale limited applications. .. As an example, there is an associative memory of Patent Document 1. Generally, in order to realize the search function described in Patent Document 1, it is necessary to access the entire memory space in the CAM, and in a large space, not only the circuit configuration becomes complicated but also the power consumption is large. There was a problem of becoming. In particular, the problem that the power consumption increases is a very serious problem at present because it increases with the scale of the CAM.

On the other hand, the latter is applicable to large-scale storage objects and is often used as a general search engine. However, a small database is constructed using the operation of the Hash function so that it can be easily searched. The efficiency is improved by searching the small database, but when converting from a large amount of data to a small amount of data, collisions of the addresses occur, repeated operations for collision avoidance are required, and high-speed processing cannot be performed in real time. It is about 100ms to 1s on Google that has achieved the fastest performance in one keyword search.

In order to withstand high-speed access during AI processing, the related Patent Document 2 of this application proposes a search engine mainly composed of high-speed, large-capacity processing, and low power consumption hardware.

Japanese Patent No. 5575997 JP-A-2019-144872 Japanese Patent No. 6166306 Japanese Patent No. 6205386

In Patent Document 2, the original data is lost, which makes it possible to search for a very small bit entry address. Losing the original data is not suitable for general purpose search engines. Furthermore, the original data is an RDB (Relational Data Base), and the necessary data is efficiently extracted from the big data by heading search such as SQL (Structured Quality Language), but it is necessary to consider this heading item in the search function. be. Although it can be seen in Patent Document 3 which refers to the original data, the association with the heading is insufficient.

We propose a new method to search from these big data in high speed and in real time with low power consumption. This application includes a new idea developed for a general-purpose real-time high-speed search, not a limited search engine associated with this AI processing.

One embodiment of the present invention is an information processing apparatus for performing general-purpose machine learning, in which a filter for filtering data with a filter for feature extraction and the filtered data are used as learning components, and learning information is stored in the learning components. A learning means that is assigned and registered in the learning memory mat, a search means that searches for learning parts and extracts the learning information registered in the learning memory mat, and a means that determines a majority of correct learning information from the searched learning information. It is an information processing device characterized by being equipped with a search engine including.

Further, one embodiment of the present invention is an information processing apparatus for performing general-purpose machine learning, in which a masking means that masks a part of search information data and learning that the data masked by the masking means is searched. From the learning means as a component, the learning information is added to the learning component and registered in the search memory mat, the search means for searching the learning component and extracting the learning information registered in the search memory mat, and the searched learning information. It is an information processing device provided with a search engine including a means for determining correct answer learning information.

According to the present invention, general-purpose search becomes possible, and the drawbacks of a conventional search engine that requires enormous power and calculation time can be eliminated, that is, a reliable search can be performed from an enormous database at high speed and with low power consumption. It will be possible.

Search engine process block diagram Search engine process block diagram using registration / search data Diagram illustrating diffusion processing Figure explaining the effectiveness of diffusion processing The figure which shows the structure of the registration data of a learning memory The figure which shows the configuration example of the search memory 1. The figure which shows the search method of the search memory 1. Calculation example of the number of blocks (divided memory) required for registering search memory Figure showing collision of divided data (registered address) that occurs at the time of registration The figure which shows the composition example (example of a person image) of the registration data of a learning memory. The figure which shows the composition example (example of a numerical image) of the registration data of a learning memory. Diagram showing the basic concept of pooling by even-odd separation The figure which shows the configuration example of the search memory 1 and the data registration method. The figure which shows the search method of the search memory 1. Diagram showing an example of learning information A diagram showing a specific example when learning information in face recognition is set. A diagram showing an algorithm for face recognition image processing by a search engine using registration / search data. Diagram showing the schematic flow of face recognition Diagram showing the algorithm for creating face part data A diagram showing an example of creating a reference image (reference data) Diagram showing program processing for learning Diagram showing program processing for performing search (inference) Diagram showing an example of diffusion processing Diagram showing an example of creating main unit data Search engine process block diagram using search data The figure which shows the processing method of the data including the mask data. The figure which shows the example of the preprocessing when the mask data is replaced. The figure which shows the data which converted the data which replaced the mask data with "0" into a binary number. The figure which shows the example of the diffusion processing when the mask data is replaced with "0". The figure which shows an example of the 48-bit diffusion type The figure which shows the registration to the search memory 2 (registration from the left side) The figure which shows the registration to the search memory 2 (registration from the right side). The figure which shows the registration result in the search memory 2. The figure which shows the example when the search memory 2 is read. Diagram showing specific examples of search information The figure which shows the mask example of the search information (the example of the two division data mask). The figure which shows the example of the diffusion processing of the search information (the example of a two-part data mask). The figure which shows the number of registrations in the division memory (one side of the left-right registration) with respect to the registration number (nx) when the division unit is 5 bits and the number of words of the division memory is 32. The figure which shows the example (example of a 2-division data mask) of the data registered in the search memory 2 after the diffusion process. The figure which shows the registration result (example of a 2-division data mask) in search memory 2 The figure which shows the example (example of a 2-division data mask) when the search memory 2 is read out. The figure which shows the example of the registration data after mask processing (the example of 3 division data mask). The figure which shows the example of the diffusion processing when the insertion position of the input data is changed. The figure which shows the example which connected each diffusion output of FIG. 43 and expanded the diffusion output. The figure which shows the relationship between the number of mask division data and the number when a plurality of left-right matches occur. Diagram showing other trial examples and calculation results for the number of left-right matches The figure which shows the flow of creating search information based on feature information and registering it in search memory 2. Diagram showing an example of a search information list Diagram showing the relationship between multiple mask states and mask classification numbers The figure which shows the example of the circuit structure of this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The present invention is not limited to the contents described in the following embodiments and examples. Further, the constituent requirements in the embodiments and examples described below include those that can be easily assumed by those skilled in the art, those that are substantially the same, and those that are in a so-called equal range. Further, the components disclosed in the embodiments and examples described below may be appropriately combined or appropriately selected and used.

FIG. 1 shows an overall picture of the search engine of this embodiment using a process block diagram. In this embodiment, two types of searches, a search using the search memory 1 and the learning memory and a search using the search memory 2, are possible. Then, these search results are selectively output from the output selector (SEL) by the search output selection data.

In the search engine of this embodiment,
-Registration / search data-Registration / search is performed using two types of data called search information. The registration / search data is used in the search memory 1 and the learning memory, and the search information is used in the search memory 2.

Registration / search data is generated from data such as images to be searched (this data is called original data). The original data becomes very large data in terms of the number of bits, and if the original data is registered in the search engine as it is, there arises a problem of an increase in memory resources. Further, the original data is, for example, image data having a high resolution, and is generally an image having various resolutions. In order to process this image data, the resolution is adjusted and preprocessing is performed to add a search condition called learning information. The preprocessed data can be stored in the search memory 1 and the learning memory as input data of the search engine.

On the other hand, the search information is like a search condition for searching the books recorded in the library such as features that specify the image data even if there is no image data. From this information, it is possible to search for learning information that is shared information of registered data. These search information and learning information can be stored in the search memory 2.

One of the features of this embodiment is that all data can be searched by specifying the learning information using the registration / search data and / or the search information.

Then, by taking the and of the search result using two different types of information, the registration / search data and the search information, a more advanced search becomes possible. For example, when searching for a certain person, it is possible to verify whether or not the person specified by the search using the registration / search data is correctly specified by the search result using the search information.

<< Search by search memory 1 and learning memory >>
First, a search using the registration / search data, the search memory 1, and the learning memory will be described. FIG. 2 is a diagram in which a part related to a search using the registration / search data, the search memory 1, and the learning memory is extracted from FIG.

Since the basic function is the same as CAM, if the same name is used, the input data to be registered in the search engine will be the key data, and the registered address will be the entry address. However, this search engine is composed of two types of memories, a search memory and a learning memory, and the input data has a different data format depending on the purpose of registration. Here, the registered address of the learning memory is referred to as a registration number (entry address), and the input data of the search memory 1 is referred to as key data.

The data registered in the search engine becomes learning parts (basic registration data) after pre-processing such as filtering with a filter for feature extraction. Therefore, in order to process the data and register it in the learning memory, as an important concept, the original data is converted to feature data (basic registration data) here, and not only the original data is not registered, but the original data itself is then Disappears from the processing process of.

Since multiple filters are used, the data lengths of the learning parts are generally different. Therefore, after adjusting the bit lengths of the learning components, a diffusion process is performed to remove the bias of the data.

As shown in FIG. 3, the diffusion process is a process in which the bit data mathematically represented in the bit space of the registered data is associated with the bit data in the new bit space generated by the diffusion process on a one-to-one basis. .. Therefore, by performing the diffusion process, it is possible not only to eliminate the bias of the original data but also to convert the original data into an arbitrary number of bits having reproducibility. Further, as shown in FIG. 4, the diffusion process is a process of converting the biased data of “0” and “1” into the data near the center of the normal distribution as shown in the figure with less bias. It is a process to convert some data into data that can be regarded as random. If the position of the bias data is moved to the vicinity of the center of the normal distribution, the numbers of "0" and "1" become almost the same.

Specifically, as a diffusion process, a process of converting the data to be processed into a BCH (Bose-Chaudhuri-Hocqenghem) inspection code having an arbitrary number of bits is performed. This conversion is irreversible from the original data to any number of bits. Therefore, even if the information stored in the learning memory or the search memory 1 can be obtained by reverse engineering, the information before the diffusion process cannot be restored from them. This has an advantage in terms of confidentiality. For details of BCH, refer to the description of Patent Document 2 as necessary.

As shown in FIG. 5, in the learning memory, the learning parts (main body data) after the diffusion processing and the data to which the learning information is added are registered in the learning memory. For example, when registering 1k word data, the registration number is address data from 0 to 1023, the main body data is composed of 40 to 120 bit binary numbers, and the learning information specifies any of A to N. The data is composed of 14-bit binary numbers.

Further, as shown in FIG. 6, the search memory 1 is composed of a plurality of divided memories (k divided memories in FIG. 6), and the registered data is composed of k blocks divided in q-bit units. Using the block data as an address, register the registration number of the learning memory different from the search memory in the divided memory dedicated to each block. In this example, the input data of the search memory is divided into k pieces, and each divided data is associated with the address of each divided memory. In the registration of the search memory, the same registration number is registered in two places of a plurality of divided memories constituting the search memory in order to perform the left-right registration described later. The registration data of the divided memory can include flag data.
The flag data becomes "1" when it is registered, and becomes "0" when it is not registered.

As shown in FIG. 7, in the search, the search data, that is, the data of all blocks of the key data is used as an address, and all the divided memories are simultaneously read in parallel. If there is matching data among the data read at the same time, it is the search result, and if not, it is judged that there is no registered data. If the same registration number is read when the divided memories are read in parallel, the registration number becomes the registration number of the learning memory.

The total number of data (learning memory registration number) that can be registered in the search memory is determined by the number of words in the divided memory and the number of divided memories.

Assuming that the registered data is random data without specifying a mask, the number that can be registered can be obtained by using probability theory. In probability theory, if the probability that can be registered in n-word memory is p,
p = 1 / n
Will be. If the probability of not being able to register is p', then p + p'= 1 (where p'is p'= 1-p)
Will be. Here, when registering m data in the memory, the registration will be repeated m times.
(P + p') ^m = 1
Can be expressed as. When this formula is expanded, it becomes as follows.
_{m m m}
1 = Σ _m C _r p ^{r p'mrb} = Σ _m C _r p ^r (1-p) ^mr = Σ _m C _r (1 / n) ^r (1-1 / n) ^{n (m / nr / n)
r = 0 r = 0 r = 0}
Here, if n is large enough
(1-1 / n) n = 1 / e
(The number of digits when n is expressed in decimal is correct, and the error is less than that). Specifically, 1 / e = 0.367879441
On the other hand, it becomes as follows.
(1-1 / 100) ¹⁰⁰ = 0.36 6032
(1-1 / 1000) ¹⁰⁰⁰ = 0.367 695
(1-1 / 10000) ¹⁰⁰⁰⁰ = 0. 3678 61
(1-1 / 100000) ¹⁰⁰⁰⁰⁰ = 0. 36787 8
(1-1 / 100000) ^1000000 = 0. 367879
(1-1 / 10000000) ^10000000 = 0. 3678794
 
Therefore, the following equation holds.
_{m m nx}
1 = Σ _m C _r p ^{r p'mrb} = Σ _m C _r (1 / n) ^r (1 / e) ^{xr / n} = Σ _nx C _r (1 / n) ^r (1 / e) ^{xr / n
r = 0 r = 0 r = 0}
(x = m / n)
The meaning of each term in this formula represents the probability that r data out of m registered data will be registered in the same word in n words. In other words, it is shown that if registration is performed according to this equation, r-fold collision occurs and only the last data is registered. The method of avoiding this multiple collision is the method of registering in the adjacent divided memory using the adjacent block address.
Also, where n is sufficiently large, when r becomes large,
(1 / n) ^r ≒ 0
And when r becomes smaller,
(1 / n) ^{-(r / n)} ≒ 1
Therefore, the above formula is practically
_{m nx}
1 = Σ _m C _r p ^{r p'mrb} = Σ _nx C _r (1 / n) ^r (1 / e) ^x (x = m / n)
^{r = 0 r = 0}
Can be regarded as.

When q = 10, n = 1024, if the above formula is applied according to this registration method, the relationship between the number of registered data and the number of divided memories required for registration can be understood as shown in FIG. Circles (1) to (20) represent the number of divided memories required for registration, and it is necessary to secure a number in which the registration rate of the divided memories is 0.01 or less for registration. Actually, since the left and right registration is performed and the left and right matching data is read, twice the number of divided memories below is required. In the figure, x represents the number of registrations in units of the number of words in the divided memory. The number of divided memories required for registration has a relation of approximately +3 with respect to x. Therefore, it is necessary to set the number of divided memories of the search memory to x + 3 or more on one side with respect to the assumed maximum number of registrations x. For example, if the number of registrations is m = 1024, it is sufficient to have four divided memories on one side.

From the binomial distribution formula obtained earlier for the address collision that occurs when data is registered in the divided memory, the general formulas of the unregistered state PE, the registered state _Pw , and the collision state (probabilistically multiple registration state) Pc are obtained. calculate.
First, if you ask for an unregistered expression,
P _E = _m C ₀ P ⁰ (1-P) ^m
= (1-1 / n) ^m
= (1-1 / n) ^{n (m / n)}
here,
m / n = x
If you say
P _E = (1-1 / n) ^nx
When n is large enough (1-1 / n) ⁿ = 1 / e
P _E = (1 / e) ^x
Will be.
Similarly, when the formula of the registration state Pw is obtained,
Pw = _m C ₁ P ¹ (1-P) ^m-1
= (m / n) (1-1 / n) ^{n (m-1) / n}
= x (1 / e) ^x
The formula for the collision state Pc is Pc = 1-P _E -P _w .
Will be.

In this method, every time a collision occurs, the adjacent divided data that is not related to the collided divided data is registered in the adjacent divided memory, so when viewed from the divided memory, the state of x = 1 is always maintained. It will be drooping. This means that the registration is performed in the state where the registration probability is the highest, as can be seen from FIG.

Here, the data to be registered will be described by taking as an example image recognition in which the image of the person registered in the system is searched. In the preprocessing of the data to be registered in the system, the resolution of each image is adjusted, and at the same time, the basic image is ANDed with the image of the same registrant taken under different conditions. By filtering this data and performing diffusion processing on the filtered data, two types of compressed data are created. One of the two types of data is one of the data registered in the learning memory, and is referred to as the main body data here. The main body data that can be registered is only unique data, and the same data that has already been registered cannot be registered. A registration number and learning information are added to this learning memory in addition to the main body data at the time of registration. Additional registration is possible for learning information even if the main body data is the same. The basic configuration of the learning information is as shown in FIG. 10 when the registration data of the learning memory for image recognition is shown as an example. The learning information is composed of bits corresponding to each of A to N. A indicates a part number, and B to N indicate a registrant of face recognition (in this case, B to N represent different images based on the image). If B is 1 among B to N, the main body data indicates the registrant B and the basic image corresponding to B of the original data registration file. Here, the basic image is a specific facial photograph in which the face of the registrant is fitted to the frame. If the flag is "1", the main body data or learning information is already registered, and if it is "0", it means that the main body data or learning information is not registered.

The other data is the data to be input to the search memory 1 which is the memory for searching the registration number of the learning memory. The search memory 1 is composed of a plurality of divided memories, and each of the divided data obtained by dividing the input data corresponds to the address of each divided memory. The registration number of the learning memory is read from the search memory 1 using the input data of the search memory 1, and the learning information of the learning memory is read using this registration number. The learning information is composed of the registrant, the position information of the divided image when the image is divided, and the like. The basic image can be known by searching the original data registration file shown in FIGS. 1 and 2 in the learning information. That is, in this system, even if there is no image data, the registered image can be searched if the data characteristic of the image data is known.

In the original data registration file, the data represented by B to N in FIG. 10 is used as the registration number, and the basic images corresponding to B to N are registered respectively. In this example, the data represented by B to N represents the distinction between the basic registered images. In this system, learning is performed in this system by registering learning information in a learning memory using basic registered images with different shooting conditions as input images. When searching with any of these registered images, for example, if the search image belongs to the registrant B, the read learning information B is 1. This is because the image that is slightly different from the basic registered image is learned to be the same as the basic image. From this relationship, only the basic registered image can be registered in the original data registration file. When registering an image different from the basic registered image in the original data file, it is necessary to register it in another area with another name.

FIG. 11 shows an example of registration data of the learning memory for recognizing the numbers 0 to 9. In this example, the registration data of the learning memory regarding the number "2" will be described. First, based on the MS P Gothic "2", pooling is performed by even-odd separation to learn four types of "2" with different shapes as shown in circles (1) to (4). Parallel parking data corresponding to part numbers (part coordinates) 0 to N is created for these four types of "2". Taking the part number I as an example, the learning memory has 14 to 30 bits of main body data (data in column I of column separation data, that is, bit data of part number “I”) and learning information with the registration number as an address. Numerical information (“0010000000”) and part number (data in which the position of “I” is “1” and the others are “0”) are registered as. Explaining the relationship with the search memory 1, the main body data is diffused and becomes key data, and the key data is registered in the search memory 1. Then, the registration number of the learning memory is registered in the search memory 1 with the key data as the address.

FIG. 12 shows the basic concept of pooling by even-odd separation based on the above-mentioned example of MS P Gothic “2”. First, digital conversion of the image data (original image) of "2" is performed. As the gray display data of the original image, the pixel value is represented by 256 gradations (0 to 255). Then, the binary display data of the original image is obtained by setting a constant gradation as a threshold value, setting the pixel to be white as "0", and setting the image to be black as "1". The data obtained by extracting only the columns having odd column numbers in the binary display data of the original image and the data extracting only the columns having even numbers in the parallel columns are created. The process of extracting even-numbered and odd-numbered data in a column in this way is called even-odd separation. Furthermore, even-odd separation of the rows is performed for each of the two data obtained by parallel-odd separation, and circles (1) and circles (3) consisting only of odd-numbered data in the rows, and even numbers in the rows. Obtain circles (2) and circles (4) consisting only of the data of. In this way, four types of pooling form image data are generated from one original image. The image data of the circles (1) and (4) saves all the image information contained in the binary display data of the original image.

From here, the preprocessing of the data to be registered in the learning memory and the search memory 1 will be explained in more detail. For example, when determining how close an image input during a search is to the image to be compared, it is necessary to increase the resolution of the image in order to make an accurate determination, and the higher the resolution, the larger the amount of information. .. As the amount of information increases, it becomes necessary to focus on individual information, and it becomes necessary to divide the image according to the resolution. In order to compare images in divided units, it is essential to make the images into parts, and the divided images (image data) are called parts (part data).

The image data registered in the learning memory and the search memory 1 are as follows. Taking one image data to be registered as an example, the data when the image is divided has the following hierarchical structure.

(1) Overall image data (2) Parts data obtained by dividing the entire data (3) Filtering data created from the parts data Here, for example, the number of divided images (number of parts) in (2) is 64, and (3). Assuming that the number of filtering is 8, one image is composed of 512 data from 64 × 8 = 512.

In the face recognition example, in order to create part data and filtering data from the entire data, AND of comparison data (reference image) and registration data (image at the time of search / registration) as preprocessing before creating part data. Performs processing and rough feature extraction. Next, the entire data after feature extraction is divided to create component data.

As mentioned above, if it is divided into 64, it is divided into 8 in both vertical and horizontal directions, numbers are added to the divided images (parts) according to the processing order, for example, from 0 to 63, and the images are processed in the order of the numbers. There are various possibilities for filtering component data, but here, pooling processing is performed. The pooling process is effective in removing minute image deformation. The pooling process is an effective filtering process when comparing front images of almost the same size. To give a specific example, when the component data is divided into areas of "vertical 2 dots x horizontal 2 dots" and a predetermined amount of numerical information, for example, black and white is represented by "0" and "1", "1". If even one "" is included, it is judged as "1". By doing so, for example, even if the vertical thickness is slightly different, it is recognized as the same thickness. That is, if the data are similar to each other by filtering, they can be recognized as the same data. By appropriately changing the number of dots in the vertical and horizontal directions, it is possible to adjust how similar the data can be recognized as the same data.

When registering the divided image data as the main body data of the learning memory, the following method is used.

(A) Create filtering data at the bottom level. Filtering component data will generate several types of data with different degrees of feature extraction. Therefore, at this level, even if the component data is different, it may be recognized as the same data as filtering data. If the data is the same at the level of filtering data, it will be the same data even if it is diffused. If the data is the same, the registration number is also the same. However, the learning information will be different if the parts are different. In this sense, the filtering data is the most basic data among the registered data and can be said to be the data at the bottom layer.

(B) Diffuse the filtering data in the order of creation to create 40-120 bit main unit data. The

(C) Add learning information and a flag to confirm whether or not it has already been registered to the main body data, and register it in the learning memory. Multiple registration of main unit data is not possible, but additional registration is possible for learning information.

In the previous face recognition example, 512 filtering data are created for one image, and this data is serially registered according to the creation order. However, if the created filtering data is already registered, it cannot be registered, and it will be registered only if it is not registered. Therefore, the registration operation is always performed 512 times per image, but the actual number of registrations is 512 or less.

The input data of the search memory 1 is created by the same procedure using the same data as the main body data creation of the learning memory except for the diffusion processing part. The big difference is that it is necessary to match the number of bits of the input data with the configuration of the divided memory of the search memory 1, and it is necessary to perform the spreading process according to this configuration.

The input data (key data) of the search memory 1 whose number of bits is matched to the configuration of the divided memory is divided into n pieces, and the registration number of the learning memory is the search memory 1 as shown in FIG. To be registered in. In this example, the input data of the search memory is divided into n pieces, and each divided data is associated with the address of each divided memory. For registration in the search memory 1, the same registration number is registered from the left and right. If the same registration number is read when the divided memories are read in parallel, the registration number is the address (registration number) of the learning memory storing the learning information. In FIG. 13, since the search memory 1 is composed of n divided memories of 2 ^m words, the bit length of the input data is m × n bits, and the number of bits after diffusion processing created from the filtering data is adjusted to this number of bits. ing. In the search memory 1, as described above, the registration numbers of the learning memory are registered in the divided memory from the left and right. Register to the address indicated by the m-bit split data in the corresponding split memory.
By registering the same registration number in two places from the left and right, the same registration number is read when the divided memory is read in parallel, and this registration number becomes the read address of the learning memory.
The number of data that can be registered in the search memory 1 can be roughly expressed by the following equation.

Maximum number of registrations = (n-8) (2 ^m ) / 2

Taking the number of divided memories of 10 (n = 10) as an example, left and right registration will be described. In the registration to the search memory 1, the division key data of the first division is used as an address and the learning memory is stored in the divided memory of the first column. Register the registration number and repeat this registration. However, even if the key data is unique, it will not be unique when viewed from the divided key data, so that a collision of the divided key data will occur. In this case, the registration number of the learning memory is registered in the divided memory of the second column using the divided key data of the second division as the address. If a collision still occurs, the data is registered in the non-collision split memory of the third to fifth columns. After registration is complete, return to the first category and continue registration. In this left-right registration, the same registration number is registered on the 10th division side at the same time as the registration of the 1st division by the same method. For example, if the division key data collides when registering in the division memory of the 10th column using the division key data of the 10th division, the division key data of the 9th division is used and the adjacent 9th column is used. Register the same registration number as the first division in the divided memory. If a collision still occurs, it is registered in the non-collision split memory of the 6th column to the 8th column. After registration is complete, return to the 10th category and continue registration. In the case of left / right registration, the matching data among the plurality of read data becomes the registration number to be read. If there is no matching data, it means that there is no registration number to be read. If this method is used, the read result can be obtained by one read operation. The search memories 1 and 2 also include a control circuit for controlling these steps.

Next, the inference to read the learning information and read the registered data based on it will be explained. In inference, key data is created from each component, the search memory is searched by the key data, the search result is used as a registration number, the learning memory is accessed based on the registration number, and the learning information is read out. The majority of the read learning information is judged and the correct answer information is extracted. In particular, a method of filtering the input input data, creating the filtering data of the search memory 1, and using this data to read out a specific image will be described.

In the above example of the divided image, first, the comparison image which is the input data is divided, and filtering data is created for each. For example, if eight types of filtering processing are performed for one component, eight filtering data can be obtained for one component. Next, the input data of the search memory 1 is created by spreading the filtering data. Using the input data of the search memory 1, the registration number of the learning memory is read from the search memory 1. As shown in FIG. 14, if the data is registered, there is matching data among the n read data. If there is matching data, it is the registration number of the learning memory in which the search data is stored. If there is no matching data, it is judged that there is no registered data.
The learning information is read from the learning memory using this registration number. FIG. 15 shows an example of learning information. Information such as Mr. X (registrant), 26 (part coordinates), and h (filtering number indicating the type of filtering) may be registered in the learning information A to N shown in FIG.

FIG. 16 shows a specific example when learning information is set in face recognition. As a matter of course, the component image after the filtering process may be shared by a large number of people. That is, even if the images of different people are the same, the main body data may be the same. However, the filtered component image (main unit data) that can be registered in the learning memory cannot be registered when the same data appears. Therefore, the characteristics or identification information of each person can be added to the main body data as learning information, and if the learning information is different, additional learning information can be registered even if the main body data is the same. In this way, when authentication is performed with the learned learning data, the feature level and recognition information of the person to be authenticated appear in each component. When authentication is performed with unlearned data, the ratio of learning information matching at the component level becomes very small. Based on this matching ratio, it becomes possible to judge whether or not learning has been completed. Data that characterizes whether or not the person has been learned (various information necessary to identify the person, such as an identification flag that identifies the person to be authenticated, a photo number, a baht number of a photo, and a pooling number (filtering number)). , Matching information at each component level, etc.) is learning information. If a file corresponding to the learning information and the original data (which may be a reference image) that has been made into parts is created, the matching part can be confirmed from the inference data in the original data registration file.

The search data is judged using the learning information read out for each part, and the entire image is specified and read out by taking a majority vote for each judgment result for each part. That is, a majority decision is made on a plurality of learning information read out for each component, and the image of the registrant specified by the majority decision is read out. Explaining the relevant part in FIG. 2, the learning information (read data circle (3)) from the learning memory for each part is determined by majority vote for each whole image, and the registrant is specified. Using that information, the image of the registrant (read data circle (1)) can be obtained as output data.

Next, face recognition image processing using this search engine will be described. FIG. 17 is an algorithm for face recognition image processing. In this example, the learning memory is composed of 8kword × 64bit, and the search memory 1 is composed of 10 8kword × (13bit + αbit) memories.

First, in learning (registering the data to be searched in the system), the first original image of learning is HD (1080 x 800 pixels), and since the data is large, it is registered in order to grasp the characteristics of the registrant from this image. A reference image is created by extracting a part of the face as a face image from the representative image of the person. For other images of the same registrant, AND processing with the reference image is performed, and image data obtained by extracting the characteristic portion of the input image is created. Next, filtering processing is performed to create data that is a learning component that is the source of registration. Learning components will be created according to the type (number) of filters.

The data after the filtering process becomes the registered data of the search engine, and the original image data is converted into learning parts by this work.
Since the number of data bits of the learning component differs depending on the content of filtering, after filtering, "0" is added to the lower bits to adjust the number of bits in order to align the data with the maximum number of bits. In this example, it is standardized to 216 bits. Since the number of aligned bits is large and the data bias is large, diffusion processing is performed to convert the bits into 40 bits, and the data is registered in the learning memory. On the other hand, the registration in the search memory 1 is 120 bits by diffusion processing according to the number of registrations in the divided memory, and is registered in the search memory 1.

In addition, inference is the same as the above learning until the input image is filtered and diffused. In the inference, the learning component obtained by filtering is diffused to create key data, and then the search memory 1 is searched with the key data. In the search of the search memory 1, the all-divided memory is simultaneously searched with the divided key data, and the matching data is obtained. The learning memory is searched with the obtained matching data, and the learning information is fetched. The inference result is output by the majority judgment of the extracted learning information.

Figure 18 shows the outline flow of face recognition by this algorithm. For example, when processing an image having a huge number of pixels of 1080 × 800, an image processing tool is used to extract only the luminance information from this image, and then perform operations such as binarization and pixel adjustment. A normal person image has many hair parts, and there is a background part, which is influenced by the hairstyle and the background. For this reason, instead of simply dividing the entire face, a part of the face related to facial features is often taken out as a facial image. The process of deleting information outside the contour of the registrant is called contour process. In this example, the contour processing is performed, the AND processing using the reference image, and the componentization are performed. If the contours of the eyes, nose and mouth of the face and their distances are known, it is possible to recognize whether or not the person is a registered person, so contour processing is performed. Contour processing is also effective for automatic driving if the contour information about the road can be grasped. When contour processing is performed by reducing the resolution of the image within the recognizable range, random rectangular information (random dots) appears around the contour, but this information cannot be removed by normal filtering. In order to remove information that cannot be removed even by performing this filtering, a reference contour image is determined, and AND processing is performed between the reference contour image and the contour image to be learned. This method is also effective in identifying defective parts by ANDing normal parts and defective parts. When the data after AND processing is decomposed into a large number of parts, many of the same parts are generated at the filtering level for similar images, and it is possible to identify different locations. Therefore, by performing AND processing and dividing into parts, it becomes possible to quantify how much the learned image and the search image match.

After these pre-processing, filtering processing is performed, but for learning, many learning parts are created and learning information corresponding to the learning parts is registered. Since face recognition is too large to process the entire face, it is necessary to create face component data that divides the entire face into parts, that is, to make parts. This componentization is performed for all registrants. FIG. 19 shows an algorithm for creating face component data. Here, the face photograph (having 144 × 144 pixels) after the luminance conversion created by the preprocessing is converted into a binary image (for example, the data format is BMP) and converted into a binary signal (in this case, BMP). The numerical data is R = G = B). Then, the image data is compressed to 1/4 by the pooling process, the image size (number of dots) is adjusted to 72 × 72 pixels, and the data obtained by using the first photographic data is processed to be a reference. Create data (reference image). Then, the image is divided to create twelve component data composed of 12 × 36 pixels. The reference image is used to extract the characteristic part of the input image. Therefore, a plurality of images to be learned are grouped into images capable of feature extraction, and a reference image is created for each group. When the rotation and misalignment of the image are large and there are many images, the images are classified into a plurality of groups, a reference image is created for each group, and the features in the group are extracted with this reference image. FIG. 20 shows an example of creating a reference image (reference data). In this example, a reference image is created for each registrant. For the input image that is the representative of the registrant, only the part surrounded by the square is left, and other information is deleted. The part you want to keep can be specified according to the purpose.

For the second and subsequent photo data of the same registrant taken under different conditions, the same processing as the standard data of the first image is performed until the pooling process, and then the data after pooling and the standard data are ANDed. By performing the above, the characteristic portion of the input image is extracted, the obtained image is divided, and 12 component data (face component data) composed of 12 × 36 pixels are created. These processes are repeated for other photo data of the same registrant.

In the face recognition of this example, as shown in FIG. 21, each of the face component data is subjected to feature extraction filtering processing. In this example, nine types of filters are used, and each data after filtering using this filter becomes a learning component. Since the data after filtering has various bit numbers, "0" is added to the lower bits to match the bit lengths to the same length (here, 216 bits are matched). Then, 5-bit password information is added to the data to create data having a length of 221 bits. The password is an encryption key for guiding the correct data after the spreading process. If this encryption key is not correct, the correct output cannot be obtained in the spreading process. After that, the learning component of the 40-bit learning memory of the learning memory (corresponding to the main body data of FIG. 5) and the key data of the 120-bit search memory 1 (corresponding to the registered data of FIG. 6) are created by the spreading process. Here, the 40 bits of the learning memory may be created directly from the 221-bit data or may be created from the key data of the 120-bit search memory 1. Further, a 64-bit password can be added to the key data of the 120-bit search memory 1 to create 40 bits of the learning memory.

In the inference, as shown in FIG. 22, the face component data of 12 × 36 pixels is filtered, and the diffusion processing is performed by the same method as at the time of learning to create 120-bit key data of the search memory 1. Using this key data, the entire divided memory of the search memory is read, the matching data is extracted, and the matching data is used as the registration number (entry address) to access the learning memory and read the learning information. If a password is set at the time of learning, the correct key data cannot be created unless the correct password is known, and the correct learning information cannot be read unless the key data is correct.

If there is no matching data in the read result of the search memory 1, it is determined that the learning information is not registered. If the learning information is not registered, additional registration is possible if additional registration is required on the spot. When performing additional registration, the learning process is performed in the same procedure as the learning process. Unlike the conventional CNN, the part that can be additionally registered on the spot does not require a huge recalculation of other registered data due to the additional registration.

As a specific example of the diffusion process, FIG. 23 shows an example of creating registration data (120 bits) of the search memory from 216-bit input data. As described above, the filtered data is 54 bits to 216 bits, and the bit length varies. Therefore, "0" is added to the lower part of the filtered data to match the data length, but the data is biased. Therefore, the 216-bit data including the total length is divided into 12 columns of circles (1) to circles (12) with an equal length of 18 bits. The data for each third column is extracted in order from the top 1 column and designated as A, then the data for each third column is designated as B in the same manner as A in order from the top 2 columns, and finally in order from the top 3 columns. Let the data for each third column be C in the same way as A. Shuffle 1 to shuffle 3 are created by arranging three of these data strings horizontally as (A, B, C) (B, C, A) (C, A, B) bits (A, B, C). The operation of combining to triple the number of bits is called shuffle). Shuffle 1 to shuffle 3 are each spread to 40 bits, and 40-bit spread-processed data D, E, and F are created and combined to make 120 bits. Here, 5 bits of password (here, all 0) are added to 216 bits of shuffle 1 to shuffle 3, and a diffusion formula of 221 bit input and 40 bit output is applied, and 221 bits to 40 bits are applied respectively. Creating. This 120-bit data is used as the key data of the search memory 1.

Further, this 120-bit data is converted into the form shown in FIG. 24 and diffused to be a learning component (main body data) of a 40-bit learning memory. Here, the upper 37 bits and the lower 64 bits are all set to "0", but the lower 64 bits can be used as the bit for inputting the password. Resources can be reduced by reducing the number of bits to one-third. Since the registration number of the learning memory becomes the registration data of the search memory 1, it is efficient to create the registration number of the learning memory together with the registration data of the search memory 1.

Learning information is added to the obtained 40 bits and registered in the learning memory. In the case of face recognition, the learning information is data that can identify the face to be recognized, such as assigning a number to the face to be registered and using the number as learning information. The method of creating the key data of the search memory 1 and the method of creating the learning component and the learning information of the learning memory are one example, and various methods can be considered.

The learning and inference algorithms have been explained using image recognition as an example, but this algorithm is not limited to this, and language recognition treats words, phrases, and contexts as groups according to conversation themes and filters them like images. You can also register.

Here, the filtering process is the same as the CNN feature extraction of registered data, and instead of extracting the features while repeating convolution and pooling, it is converted into a characteristic pattern (learning component) using a plurality of filters. Represents.

Regarding the filter pattern, for example, instead of an image, various characteristic filters such as a T-type corresponding to the human eye and nose and an inverted T-type corresponding to the nose and mouth can be considered.

Obtaining a plurality of learning parts by filtering from inference data, searching the search memory 1 with a plurality of learning parts, obtaining the learning information of the learning memory from the search results, and making a majority decision of the learning information is effectively CNN. Same result.

<< Search using search memory 2 >>
From here, the search using the search memory 2 will be described. That is, in the search engine using the search memory 2, data that can identify an image is registered as search information together with the image data. By doing so, the registered image data can be searched from the search information even if there is no image data.

For example, when registering a person's face, any information that can identify the person is acceptable, and that information is called feature information. Then, the aggregate of the feature information can be registered as the search information. However, it will be registered according to the fixed format.

The following can be taken up as specific feature information.

・ Height (very tall, high, normal, low, etc.) → 2 bits ・ Age (old, middle-aged, young, children) → 2 bits ・ Gender (male, female) → 2 bits ・ Occupation (service, manufacturing, civil engineering)・ Construction, transportation / logistics, distribution, food / drink / accommodation, real estate, finance / insurance, hospital / welfare, government / local government, information / communication, education, distribution, students, housewife / husband, etc.) → 4 bits ・ Body type (thin) Type, normal, fat, fat) → 2 bits ・ Physique (large, normal, petite, etc.) → 2 bits ・ Race (Japanese, Asian foreigners, Westerners, etc.) → 2 bits ・ Hair ( Black, gray hair, gray hair, etc.) → 2 bits ・ Face shape (slender, round, square, etc.) → 2 bits ・ ・ ・ ・ ・ Search information list that is a registration table that describes feature information such as Is created, and the search information, which is a collection of feature information, is displayed in binary. If there are 20 bits in this binary display, it is possible to identify one million types. In the case of face recognition, if the registrant is registered in B to N, the search information, which is the feature information, is diffused and used as the input data of the search memory 2, and is used as learning information instead of the registration number. Images can be searched.

In this search, the search information becomes the input data of the search memory 2.

The search information is data like a table of contents that can identify and narrow down the original data after satisfying the condition that it is unique data. Therefore, the search can be performed even if there is no data in the same format as the original data. Since this search operation does not go through the learning memory, the part related to the learning memory is deleted from the search operation, but the search information is used instead of the input data of the search memory 1, and the search memory 2 is used. It is the same as the search of the search memory 1 except that. FIG. 25 shows a diagram in which the search information and the portion related to the search using the search memory 2 are extracted from FIG.

In this search, even if there is no search data corresponding to the original data, the original data registered only with the search information can be searched, so the so-called ambiguous search that narrows down the information you want to know by repeating the search operation while changing the search information is possible. It is possible.
However, since some information is unknown, the number of bits of this information is multiplied by n so that it can be divided into a plurality of groups and masked in group units.

The search information is converted into a bit length that matches the configuration of the search memory 2 to be registered by diffusion processing, and it is divided according to the number of words in the divided memory that constitutes the search memory 2 (this divided data is called a block). On the other hand, the mask data is masked in the division unit of the search information. Since the search information is created from a completely different viewpoint from the image data, it is not necessary to match the divided data length of the search information with the block length of the search memory 2 in the case of diffusion processing. For example, the search information can be divided into 5 bit units, masked in that unit, and the block length of the search memory 2 can be set to 10 bits. Since the mask position is ignored at the time of registration, if arbitrary data is specified for the mask position at the time of search, the arbitrary data is mistaken for the registered data, and the erroneous information corresponding to the data is read out. By specifying arbitrary data for the mask position, a large amount of unexpected data may be read out and narrowing down may not be possible. Therefore, it can be seen that it is important to specify the mask position at the time of reading when considering the narrowing down of the data read.

<< Introduction of mask function >>
The search information is described in a predetermined format and coded in binary, but unknown information may be included. In that case, the information is masked and registered and searched. Therefore, it is necessary to introduce a mask function for registration and search of search information.

First, the basic idea of how to introduce the mask function is to ignore the mask data at the time of registration / search. By doing so, the mask portion is not included in the registered data and the search data, and as a result, the information including the mask data is registered / searched.

As a specific method, there are the following two methods.

-A method to execute the basic idea as it is-A method to allocate the least significant bit or the most significant bit of the divided memory to the mask position and register and search using the same method as before after the diffusion process << Basic idea as it is How to do >>
FIG. 26 shows the results when the registered data circles (1) to (4) having an extremely large bias are registered in a plurality of search memories 2 and searched. Here, the registered data having a length of 20 bits is divided into four divided data with 5 bits as a divided unit and registered (data divided according to the number of words in the divided memory constituting the search memory 2). The number of bits in a block is the same as the number of bits in the division unit of registered data). Here, the numbers shown in the respective cells of the registered data circles (1) to (4) are 5-bit binary numbers expressed in decimal numbers. * Indicates mask data. The method of registering in the search memory 2 is the same as the method of registering in the search memory 1 described above, except that the mask data is ignored and the data is registered. This method can always be registered by increasing the number of divided memories of the search memory 2, but the number of required is determined by the bias of the registered data and the number of masks. In this case, since the bias of the registered data circles (1) to (4) is extremely large, a total of 4 sets of divided memories are required with 4 blocks as one set in order to register all the data. There is. When four sets of search memory 2 are simultaneously read by "0232" which is search data (key data), a plurality of data are read from the divided memory of each circle (1) to (4). As shown in FIG. 26, for the circle (1), four registration data of 0, 4, 10, 13, 13, for the circle (3), one registration data of 14, and for the circle (4), one of 15. The registration data is read out. If the search condition is the registration order, 0, 14, and 15 will be selected. In this way, if the mask position is not specified at the time of search, three pieces of data satisfying the conditions can be obtained (naturally, if the mask position is specified, any of the three pieces will be clarified). Since this method cannot perform diffusion processing, it is necessary to match the bit length of the input data with the key data length of the search memory. Further, if a random value is entered in the mask portion at the time of reading, erroneous reading may occur, and it is necessary to specify the mask at the time of reading.

<< Method of allocating the least significant bit or most significant bit of the divided memory to the mask position and registering and searching using the same method as before after the spreading process >>
The least significant bit (all "0") or the most significant bit (all "1") of the divided memory is assigned to the mask position, and after the spreading process, registration and retrieval are performed using the same method as before. Since this method can perform diffusion processing, it is not necessary to match the bit length of the input data with the key data length of the search memory. However, it is necessary to specify the mask at the time of reading in the same way as the method of executing the basic idea as it is. It should be noted that all "0" tend to have less change in data properties than all "1". When the divided memory is 64 words and the number of registered data is 64, when the left-right registration described above is used, the number of divided memories of the search memory 2 needs to be 8 or more. However, when the number of words in the divided memory becomes small, the accuracy of e becomes low, so that the deviation from the number registered in the divided memory described with respect to FIG. 8 becomes large.

The theoretical value and mask data calculated using the establishment theory for the number of registered data in the divided memory when 64 random data without mask are registered in the divided memory of circles (1) to (8) in order from the left end are described as " Table 1 shows a comparison of the number of registered trial results registered by replacing with "0" or "63".

The trial result is the data when the registered data and the position of the random number are randomly determined, and is the average value of the trial results performed 12 times. The registered data used in the trial is 48-bit, 6 types of data composed of 8 divided data, and the mask data is replaced with "0" or "63" and registered. From the above, it can be seen that the same result as the theoretical value can be obtained even if the mask data is set to "0" or "63" in bit units.

Figure 27 shows an example of the trial results. In this example, the blank part of the registered data is mask data, and "0" or "63" is placed in that part as preprocessing. Next, FIG. 28 shows the data obtained by converting the mask data replaced with “0” into a binary number. The data converted into the binary number is the diffusion input data to be diffused.

As shown in FIG. 29, this diffusion input data is diffused to become binary diffusion output data. As shown in the figure, 48-bit input data is diffused and 48-bit output data is generated. What is converted into a decimal number is the diffused output data after the decimal number conversion.

FIG. 30 shows a diffusion type (48-bit input, 48-bit output). By this diffusion formula, the output data (c47, c46, ... c1, c0) after the diffusion process can be obtained from the input data (a47, a46, ... a1, a0). The figure shows that, for example, the output data c23 of the 24th bit after the diffusion process can be obtained from the input data (a47, a46, ... a1, a0) by the following equation.

c23 = a44 + a39 + a38 + a36 + a34 + a27 + a26 + a24 + a22 + a17 + a16 + a12 + a10 + a09 + a08 + a06 + a05 + a02 + a00

Here, + in the equation represents an EOR operation.
In EOR operation
0 + 0 = 0
0 + 1 = 1
1 + 0 = 1
1 + 1 = 0
Will be.
Other output data can be obtained in the same way.

FIG. 31 shows the registration result of the search memory 2 in the left split memory. The search memory 2 of this example is composed of a total of 12 columns of divided memory on the left and right, but shows the registration results of the 6 columns on the left side (originally, in order to register data 0 to 63 from the table of FIG. 8). Four split memories are sufficient for this, which is consistent with the results in FIG. 31). Consider a case where the key data No. 0 in the figure is used and registered in the search memory 2. First, since the leftmost division key data is 39, the registration number 0 (the number corresponding to the reference image representing the registrant) is registered at the address 39 of the search memory 2. Originally, the learning information is registered, but here, in order to facilitate understanding, the registration number is registered. This is repeated, and if there is a collision, the registration number is registered using the adjacent division key data. For example, considering the case of registering in the search memory 2 using the key data of No. 34 in the figure, since the leftmost divided data is 39, the registration number 34 should be registered in the address 39 of the search memory 2. However, the registration number 0 has already been registered. Therefore, the registration number 34 is registered in the address 9 of the search memory 2 in the second column from the adjacent division key data 9.

FIG. 32 shows the registration result of the search memory 2 in the right split memory. Consider the case of registering in the search memory 2 using the same 0th key data as before. The diffusion output data in the figure is the same as the data in FIG. 31, but the left and right sides are reversed for convenience. Since the leftmost division key data is 62, the registration number 0 is registered at the address 62 of the search memory 2. When the registration work in the left-right divided memory is repeated and the registration in the search memory 2 is completed, the result is as shown in FIG. 33. Here, the four columns of the circles (1) to (4) of the registration result of the search memory 2 correspond to the four columns of 1 to 4 of the "left side registration result" in FIG. The four columns of circles (5) to (8) correspond to the left-right inverted four columns of columns 1 to 4 of the "right registration result" in FIG. 32.

FIG. 34 shows an example when the search memory 2 is read out by using the registered data as the search input data. The process from the search input to the calculation of the diffusion output is the same as the registration process. The difference from the registration is that the block data of the obtained diffusion output is used as an address and the corresponding divided memory of the search memory 2 is read out in parallel. If there is a match in the parallel read results, that is the read result. For example, in the case of the 0th key data, the registration number 0 (the number corresponding to the reference image representing the registrant) is read out from the divided memory at the left and right ends. Since they match, it can be seen that the information to be read by the key data is the registration number 0 (the number corresponding to the reference image representing the registrant). As described above, unlike the search memory 1, the search memory 2 does not register the registration number of the learning memory, but registers the number corresponding to the reference image representing the registrant. A to C in FIG. 34 indicate in what column on the left and right the data after the diffusion processing was registered. For example, in the figure, the data in which ◯ is entered in each column A indicates that the data is registered in the first column on the left and right. Similarly, it indicates that B is registered in the second left and right columns and C is registered in the third column on the left and right (note that the number of registrations of A, B, and C represents the registration rate with respect to the number of registrations. become).

From the above, it can be seen that the registered information (learning information) is correctly read even if the search information includes mask data. Moreover, as shown above, it can be seen that the number of registrations registered in each divided memory is almost the same as in the case without a mask.

As described above, if the search information is divided into arbitrary units, masked in this unit, and the masked information is registered, even if the search information contains unknown information, the unit containing the unknown information can be used. It becomes searchable by masking. A specific example of the search information is shown in FIG. 35. To simplify the story, the case where the search information is represented by 20 bits, the division unit is 5 bits, and the number of registered data is 160 is shown. Although these search information is not masked, the search information is registered with data masked for two division units and the results are compared. As shown in FIG. 36, in order to mask the divided data in the second and fourth columns, "0" is input in the second and fourth columns (an example of the two divided data mask).

It was found that the number of left-right matches was significantly different from the calculation result, and it was found that there were many same data that violated the registration conditions as shown by the thick frame in the figure. The cause is that the number of masks is too large and the same data is generated many times. As will be described in detail later, it is necessary to exclude the multiple registration part in consideration of the condition that the same data cannot be registered.

Actually, several types of mask data are also registered at the same time when the search information is registered. Therefore, the number of registered data is
The number of registrations without mask data x the type of mask. The mask data corresponds to the filter at the time of image registration in the search memory 1 and the learning memory.

FIG. 37 shows an example of the case where the data shown in FIG. 36 is diffused. In this example, a 48-bit diffusion formula is used (a diffusion processing format is used in which the input range that can be set is a35 to a0 by fixing "0" to the upper 13 bits from a47 to a35. Yes). A47 to a36 are set to "0" (in FIG. 37, the description of a47 to a36 is omitted), search information is input to a35 to a16, a15 to a0 are set to "0", and diffusion processing is performed to c47 to c0. The result is output. Since the number of registered data is 160, x = 5 in the calculation example of FIG. In this case, the number of divided memories of the search memory 2 required is 7 on each side. In the table of FIG. 8, when x = 5, the registration rate of the 7th divided memory is 0.1, and the registration rate of the 8th divided memory is 0.01. Here, since it is divided in units of 5 bits, the number of words in the divided memory is 32. From now on, when the number of registered 7th and 8th divided memories is calculated,
The number of registered 7th divided memory is 0.1 × 32 = 3.2 (pieces) → 3 pieces The number of registered 8th divided memory is 0.01 × 32 = 0.32 (pieces) → 0 pieces. FIG. 38 shows the number of registrations in the division memory (one side of the left and right registration) circles 1 to 20 with respect to the registration number (nx) when the division unit is 5 bits and the number of words in the division memory is 32. From this, since the number of registered 8th divided memory is zero, it is possible to register even 7 divided memories. As described above, when the number of registrations in the last column of left and right registration is zero or close to zero, it can be seen that registration is possible in many cases even if the column is ignored.

From now on, the number of bits required to register the search memory 2 will be 5 × 7 = 35, and 35 bits from c47 to c13 will be registered on the left side of the divided memory. Similarly, the upper / lower inverted data of the above search information is created, the same diffusion processing as above is performed from this data, and the output 35 bits from c47 to c13 are registered on the right side of the divided memory. This method is applied because the number of divided memories required for registration increases as the number of registered data increases, that is, the output of 48 bits is insufficient when the number of registered data increases (70 bits calculated by 5 × 14). is required). This method can be applied not only to the system using the search memory 2 but also to the system using the search memory 1. FIG. 39 shows the data registered on the left side of the divided memory and the data converted into a decimal number, and the data registered on the right side of the divided memory and the data converted into a decimal number based on the example of FIG. 37. ..

FIG. 40 shows a part of the registration source data, a part of the input data of the search memory 2, and the registration result in the search memory 2 (a part of the registration result of 160 data of the 2-division data mask). The number in the cell in the registration result indicates the number of the registration source data (corresponding to the registration number of the learning memory in the search memory 1).

FIG. 41 shows the result of reading the search memory 2 of this example. The figure shows a part of the registration source data and the reading result by the data. For example, if the input data (key data) of the search memory 2 is created from the registration source data "13.0 2 0" of the 0th as in the previous example, and the search memory 2 is read using the created input data. , It can be seen that the values in the first column and the 14th column of the divided memory match (left and right match) with "0", and the information to be read by the key data is the information corresponding to the registration number 0. ..

Here, a method of expanding the extended output (number of bits) by spreading processing each of a plurality of registered data created by shifting the input data and concatenating the spread outputs will be described.

FIG. 42 shows an example of registered data after mask processing (an example of masking three divided data). First, the registration data is shown in decimal. It is shown as a binary number (each data is divided by 5 bits from the left and is called 1 to 6 divided data from the left). Next, the mask position is specified to mask the registered data. Here, based on the mask information registration file, the 1st, 2nd, and 6th division data are specified as the mask position, and the 1st, 2nd, and 6th division data data. By setting all to "0", the registration data after mask processing is created. Next, as shown in FIG. 43, in the circle (1), the registered data after the mask processing is input to the 48-bit diffusion type a47 to a18, and “0” is input to a17 to a0. Next to it, the diffusion outputs c47 to c0 after diffusion are shown in binary. Next to it, the data obtained by converting those binary numbers into decimal numbers is shown. Here, the valid data are c47 to c3, and they are converted into decimal numbers every 5 bits, and 9 decimal numbers are shown. Similarly, in the circle (2) in the figure, the same registered data after mask processing is input to the 48-bit diffusion type a42 to a13, and "0" is input to a47 to a43 and a12 to a0 for diffusion processing. The results are shown in the same way. As shown by circles (3) to (5) in the figure, by shifting the registration data after the same mask processing and performing the diffusion processing, a total of 5 patterns of diffusion output results can be obtained. By concatenating the obtained diffusion outputs, the bit length of the diffusion output can be expanded five times.

FIG. 44 shows an example in which the diffusion outputs of the circles (1) to (5) in FIG. 43 are connected to expand the diffusion output. The diffusion outputs of the circles (1) to (5) in FIG. 43 are connected in order from the left. In this example, since the number of words in the divided memory is 32 words, the divided key data length is 5 bits. Therefore, the above data is obtained by extracting only 45 bits out of 48 bits and concatenating them in the order of creation. However, there is another method for expanding this diffusion output. For example, the 48-bit diffusion output may be concatenated as it is in the order of creation and then divided according to the division key length. In this case, in the above method, the output data of 45 divisions can be obtained for each division key, whereas the output data of 48 divisions can be obtained. By using these methods, the key data length of the divided memory can be freely set according to the number of registered data.

FIG. 45 shows registration source data without masked division data (mask division data) in division units, registration source data including one mask division data, and registration source data including two mask division data (without mask in the figure). , Mask 1 and Mask 2 are supported respectively), and registration / reading is performed 3 times each (indicating how many times the circles (1) to (3) in the figure indicate), and there are multiple left-right matches. The number of occurrences is shown. For each of 32, 64, 160, and 190 registered data, the number of times when multiple left-right matches occur in which the read results of the left split memory and the right split memory show the same information. Shows. In addition, each registration source data is created by using random numbers.

Further, FIG. 45 shows the number of columns of the divided memory required for the left and right registration in each of the above. Looking at the trial results, the number of left-right matches for "Mask 2", which has two mask division data, has increased sharply, but this is because the number of mask division data has increased too much, and the same registration data is shown as shown in the representative example of FIG. It turned out that the cause was a large number of occurrences.

Considering the condition that the same data cannot be registered, if the data is added after removing the multiple registration part, the number will be the corrected number, which can be considered to be within the range of the theoretical calculation result. This calculation formula is as follows. The number of registered data can be expressed as nx when the number of words in the divided memory is n and the registration rate is x. Also, if the number of left-right matches at the time of search is r, it can be expressed by the following equation (1).

( _(X-1) C _(r-1) P ^(r-1) (1-P) ^(x-1)-(r-1) ) × nx (P = 1 / n) (1)
This formula is expressed by the registration rate, and has the same form as the following formula (2) when r data are registered in the divided memory when m data are registered. That is, the meaning of the formula is the same.

_m C _r P ^r (1-P) ^mr (P = 1 / n) (2)
This formula also holds for left-right match registration.
However, left and right registration is premised on registering the same data one by one from the left and right, and this premise is incorporated in the formula. Therefore, if the number of left-right matches is only one at the time of reading, the number of matches r represented by the equation (2) is "zero". This means that the value of r in equation (2) is set to r-1.
As a matter of course, it is necessary to delete the divided memory used for left and right registration from the formula according to the premise.
To view the registered data in units of divided memory, it is easier to see by the registration rate x than by the number m of registered data. Therefore, if the registration rate x is used, the registration rate when calculating the number of matches is x-1 instead of x. When this relationship is expressed by the registration rate, it can be seen that the equation that reflects the preconditions in equation (2) is equation (1) itself.

Table 2 shows the calculation of the number of left-right matches for the cases of n = 32 and x = 5 (registered number 160) using this formula. It can be seen that the number in which two or more matching numbers appear on the left and right at the time of reading is 19, and the example in FIG. 45 is within the range of variation.

If n = 32 and x = 6, the number of registrations is 192, and the number of left-right matches appearing is as shown in Table 3.

Further, if n is increased and n = 16k and x = 6, the number of registrations is 96k, and the number of occurrences of left-right matching is as shown in Table 4.

From the above, it can be seen that even if the number of registrations is significantly increased, there is almost no change in the number of multiple occurrences of left-right matching. This indicates that the multiple occurrence rate of left-right matching decreases in inverse proportion to the number of registrations. Moreover, from this calculation formula, it is shown that the multiple occurrence rate of the left-right match depends only on the number of registrations and the word of the divided memory, is not affected by the spreading process, and is not affected by the adjustment of the number of divided memories by the spreading process.

If the division data (block) length of the search memory 2 is q = 5 (n = 32), the key data length is 5 × 40, and the number of registrations is 512 (x = 512/32), the value of x is 16. In this case, 512 data were registered in the search memory 2 composed of the number of divided memories g: 19 × 2, and the trial result (reading circle (5)) was performed four times for the number of left-right matches when the data was read. -Reading circle (8): Here, the number indicates the number of trials) and the comparison results when g'in the following formula is changed from 15 to 19 are summarized in the table as follows. Become.

Assuming that the number of registered data in units of the number of words in the divided memory is x for one side, the number g'of the divided memory to be read is x-1 because one of x has already been registered. From now on, the number of divided memories g'that will be the target of probability calculation is g'= x-1
Will be. Also, for the number of left-right matches r, the executable r'is r'= r-1 for the same reason.
Will be. Applying this to the formula for probability calculation, the formula for calculating the number of left-right matches is _g'C r'P _r ^' (1-P) ^g'-r'
(g'= x-1, r'= r-1)
become that way.

 

Focusing on the case where the number of matches is one, comparing the read data, read circles (5) to read circles (8) and the results when g'in the calculation formula is 15 to 19, the read data is 307 to. It is distributed between 323 and the average value is 319. The closest calculation result is for g'= 15. From this we can see that we can replace x in the formula with g'= x-1. Also, focusing on the number of matches, it can be seen that if the number of matches in the calculation formula is set to +1 it matches well with the trial results.
The read circles (5) to read circles (8) and g'= 15 on the calculation formula,
Number of matches-1
The result when it is placed is as shown in FIG. As you can see, it fits well with the trial results.

At the time of reading, g'is represented by x-1 when the number of registrations is expressed in units of the number of words in the divided memory. When calculating the number of matches in the input, when reading the registered data, one of the theoretical x is always assigned to the read. This assigned one is removed from the calculation, the remaining x-1 is included in the calculation, and g'= x-1 in the calculation.
Will be. The number of divided memories actually used for registration is x + 3 because the registration status is stochastically registered. However, when finding the distribution of the number of matches, the theoretical x is used instead of the number of probabilistically registered divided memories. It is considered that the same idea holds for the number of matches. If the registered data is read, one has already been registered, and the calculation formula is considered to represent the remaining number of matches.
Number of matches in the formula +1
Is the actual number of matches.

The learning memory is not required for the search using the search information (the search memory 2 is sufficient), and the learning information to be registered in the search memory 2 is a registration number for registering the registration source data in the original data registration file. be able to. However, if information including a plurality of mask information is added in addition to the non-mask information, a plurality of search information is registered in the registration number of the original data registration file, which causes a problem that the registration condition is not satisfied. In order to solve this problem, a lower bit area is provided in the registration number of the search information, and the registration number of the mask information is assigned to this lower bit area. The number of mask types is fixed to a constant value like the number of filters in the main body data. For example, the number of mask types is assigned to the lower 3 bits of the registration information. By doing so, the number of registered data in the search memory 2 is a multiple of the number allocated to the original data registration file, but there is no operational problem even if there is no confirmation memory for storing the search information itself. If there is a confirmation memory, this data may be stored in the confirmation memory (original data registration file). In this case, for example, when the image data is registered in the original data, the contents of the search information can be viewed at the same time. In this way, if the lower bits defined by the registration number of the search information are deleted, the registration number of the original data file can be obtained.

FIG. 47 shows a flow of creating search information based on feature information, which is information for identifying a person, and registering the search information in the search memory 2. A search information list including feature information for specifying the registration number of image data is created, and the feature information is searched for each search information list to create search information. Then, the search information is diffused and the obtained key data is registered in the search memory 2.

FIG. 48 shows an example of the search information list. Search information can be created by selecting the code corresponding to the item in this list. If there is no item corresponding to this search information list, it is considered that this data should be masked.

The search information created based on each search information list (which may include a masked portion) is diffused to create key data of the search memory 2. The image data number of the original data registration file is registered in the search memory 2 using this key data. Here, if there are eight types of masks, for example, the registration number must be unique, so eight times as many registration numbers as when there is no mask are required.
Here, assuming that the same image is selected with or without a mask, the registration number is the image number (upper bit) + mask part (lower 3 bits).
With the above configuration, it is possible to absorb the increase in the registration number. By doing so, the registration number of the original data file can be obtained by deleting the lower 3 bits in which the registration number is defined.

FIG. 49 shows the relationship between a plurality of mask states and mask classification numbers when there are eight types of masks. Here, mask classification numbers 0 to 7 are assigned to eight types of masks, and the hatched portion in the search information list is used as the mask position. A method of enabling a plurality of images to be searched by a mask is common. However, here, it is necessary to be able to perform an image search even if the search information is uncertain, and the registration number of one image data can be searched from a plurality of mask information. For example, circles (1) and circles (2) are information in which mask states are unlikely to occur, and if the mask states are limited to these, eight types of mask states including no mask will occur. Moreover, in order to enable registration of registration information (learning information) using each of these states as search information, eight types of registration numbers are required for one image data. If the lower bits (the part corresponding to the mask classification number) of the registration number obtained at the time of search are deleted, the registration number of one image data can be obtained. Therefore, this example is suitable for applications in which mask position information for each purpose of use is registered and registered / searched for each purpose. If the mask position is specified incorrectly, it cannot be read, so it can be used as one security function. The search side can decide where to mask and how to change the search data at the time of search, and the searchable image number can be changed by changing the search information little by little, and the desired image data can be obtained by this operation. can. In addition, the search information of the image data can be grasped by this operation.

So far, the method of using the mask information has been described with respect to the relatively highly necessary "search using the search memory 2", but the method of using these mask information is the "search using the search memory 2". It can be applied not only to the system for "search memory 1" but also to the system for "search by search memory 1 and learning memory". For example, the data after preprocessing such as filtering can be masked by the filter for feature extraction to be the registered data of the search memory 1, and the mask classification number can be registered as the learning information in the learning memory. ..

FIG. 50 shows an example of the circuit configuration of this embodiment. As shown in the figure, the present embodiment mainly includes a search memory 1 (mat) 102, a search memory 2 (mat) 103, a learning memory mat 104, a control circuit 105, and a diffusion circuit 106. The control circuit 105 is connected to each of the search memory 1 (mat) 102, the search memory 2 (mat) 103, the learning memory mat 104, and the diffusion circuit 106, and has a function of comprehensively controlling information processing. ..

The search memory 1 (mat) 102 and the search memory 2 (mat) 103 are constructed by a plurality of divided memories 102a, 102b, 102c ... 103a, 103b, 103c ... For the plurality of divided memories, the key data for searching the search memory 1 (mat) 102 and the search memory 2 (mat) 103 is divided according to the number of divided memories, and the address of the divided memory is selected by the divided key data. Can be done. That is, the key data can be obtained by dividing the search memory 1 (mat) 102 and the search memory 2 (mat) 103 into a plurality of divided memories 102a, 102b, 102c ... 103a, 103b, 103c ... The allocated area is divided into multiple parts. The registration number of the learning memory mat 104 is registered based on the information selected in the search memory 1 (mat) 102.

Each divided memory can be configured by using a storage device such as SRAM. In this sense, it can be said that the present invention is a search engine realized by using a plurality of SRAMs.

In the configuration of this search engine, the diffusion circuit 106 creates diffusion data to be input to the learning memory mat, the search memory 1 (mat) 102, and the search memory 2 (mat) 103, and inputs the diffusion data to the control circuit 105. The configuration of the control circuit 105 consists of an input unit 1051 for inputting input data after diffusion, a division unit 1052 for dividing the input data, a writing unit 1053, and a learning memory mat 104 and a search memory 1 via a collision information storage unit 1054. It is composed of a writing system for registering data in the (mat) 102 and the search memory 2 (mat) 103, and a reading system described later. For writing to the search memory 1 (mat) 102, the divided upper input data and lower input data (that is, left and right) data are used as addresses, and the learning memory is stored in the divided memory for the upper input data and the lower input data (that is, left and right). Write the registration number (entry address) of the mat 104. As for the specific writing operation, the same registration number of the learning memory mat 104 is written in both the divided top-level input data and the bottom-level input data as the corresponding divided memory addresses. When the addresses collide in this writing operation, the registration number of the learning memory mat 104 is written in the adjacent divided memory with the adjacent divided input data as the address only for the collided ones. When the registration is completed, it returns to the original split input data and continues the registration.

In addition to the registration number of the learning memory mat, flag data for identifying the presence or absence of a collision is added to the divided memory of the search memory mat by registering the divided memory. When registering the divided memory, the registration number of the same learning memory mat is registered from the upper input data and the lower input data (that is, from the left and right) because the divided memory constituting the search memory mat is read at the same time at the time of reading. This is to narrow down the search data by finding the data to be used.

For writing to the search memory 2 (mat) 103, the registration information including the registration number of the original data file, the information for specifying the mask type, and the flag data for identifying the presence or absence of a collision is written as learning information.

Regarding the reading system, the reading unit 1055 for inputting the key data divided by the dividing unit 1052, and the reading unit 1055 are the divided memories 102a and 102b of the search memory 1 (mat) 102 and the search memory 2 (mat) 103 with the divided key data. , 102c ... 103a, 103b, 103c ... are read, the read result is sent to the confirmation unit 1056, the confirmation unit 1056 reads out the learning memory 104 with the matching data, and the learning memory 104 or the search memory 2 (mat) 103. The learning information registered in is read out, and the read-out result is sent to the output unit 1056. If there is no matching data, it means that there is no registered learning information. The output unit makes a majority decision on the learning information. If the result of the majority decision exceeds the set threshold, the majority data will be the result of inference. The output unit outputs the result as correct answer data. Further, the output unit 1056 outputs the inference result using both the search result obtained by using the search memory 1 (mat) 102 and the search result obtained by using the search memory 2 (mat) 103. Can be done. This enables more advanced searches.

As described above, the present invention provides a completely new logical methodology based on a conventional circuit configuration, particularly a memory circuit. That is, the present invention relates to a programless architecture that skillfully utilizes the structure of key data and its registration number. In other words, the invention relates to a brain-type architecture.

Although the present invention has been applied to image processing in the above description, the scope of application of the present invention is not limited to this.

102 ... Search memory 1 (matt)
103 ... Search memory 2 (matt)
104 ... Learning memory (mat)
105 ... Control circuit 106 ... Diffusion circuit 1051 ... Input unit 1052 ... Division unit 1053 ... Writing unit 1054 ... Collision information storage unit 1055 ... Reading unit 1056 ... Output unit

Claims (18)

1. An information processing device for general-purpose machine learning
   Features A filter that filters data with a filter for extraction,
   Learning memory mat and
   A learning means in which filtered data is used as a learning component to be searched, learning information is added to the learning component, and the learning component is registered in the learning memory mat.
   A search means for searching for learning components and extracting learning information registered in the learning memory mat,
   A means to determine the correct answer learning information by majority vote from the searched learning information,
   An information processing device characterized by being equipped with a search engine including.
2. The information processing apparatus according to claim 1, further comprising means for learning additional data and registering additional learning information.
3. The information processing apparatus according to claim 1 or 2, further comprising a diffusion processing means for compressing and decompressing the data.
4. The information processing apparatus according to claim 3, wherein the diffusion process is a conversion process that eliminates data bias.
5. The search means
   Input section for inputting data and
   The division part for dividing the input data and
   A search memory mat consisting of multiple split memories,
   A search memory writing unit for registering the registration number of the learning memory mat using the divided input data as an address in the search memory mat.
   A search means for accessing the search memory mat using the divided input data as an address,
   A reading unit that extracts the learning information of the learning memory mat based on the reading result from the search memory mat.
   The information processing apparatus according to any one of claims 1 to 4, wherein the information processing apparatus is provided.
6. The fifth aspect of claim 5, wherein the search memory writing unit registers the same registration number of the learning memory mat in both of the corresponding divided memories using the divided upper input data and the lower input data as addresses. Information processing device.
7. When the addresses of the upper input data or the lower input data collide, the registration number of the learning memory mat is registered using the adjacent divided data as the address, and when the registration is completed, the original upper input data or the lower input data is returned and registered. The information processing apparatus according to claim 6, wherein the information processing apparatus is continued.
8. The information processing device according to claim 6 or 7, wherein the registration data of the divided memory includes a registration number of the learning memory and flag data for identifying the presence / absence of registration.
9. The search means according to any one of claims 5 to 8, wherein the search means simultaneously reads all the corresponding divided memories of the divided input data, and the matching read data is used as an address for reading the data registered in the learning memory mat. Information processing device described in Crab.
10. An information processing device for general-purpose machine learning
    Masking means to mask a part of search information data,
    A learning means in which the data masked by the masking means is used as a learning component to be searched, learning information is added to the learning component, and the data is registered in the search memory mat.
    A search means for searching for learning parts and extracting learning information registered in the search memory mat,
    A means of determining correct learning information from the searched learning information,
    An information processing device characterized by being equipped with a search engine including.
11. The information processing apparatus according to claim 10, further comprising means for learning additional data and registering additional learning information.
12. The information processing apparatus according to claim 10, further comprising a diffusion processing means for compressing and decompressing the search information data.
13. The information processing apparatus according to claim 12, wherein the diffusion process is a conversion process that eliminates data bias.
14. The search means
    Input section for inputting data and
    The division part for dividing the input data and
    The search memory mat composed of a plurality of divided memories and
    A search memory writing unit for registering the learning information using the divided input data as an address in the search memory mat.
    A search means for accessing the search memory mat using the divided input data as an address,
    A reading unit that extracts the learning information based on the reading result from the search memory mat,
    The information processing apparatus according to any one of claims 10 to 13, wherein the information processing apparatus is provided.
15. The information processing apparatus according to claim 14, wherein the search memory writing unit registers the same learning information in both of the corresponding divided memories using the divided upper input data and the lower input data as addresses. ..
16. When the addresses of the upper input data or the lower input data collide, the same learning information is registered using the adjacent divided data as the address, and when the registration is completed, the original upper input data or the lower input data is returned and the registration is continued. The information processing apparatus according to claim 15, which is characterized.
17. The information processing apparatus according to claim 15, wherein the registration data of the divided memory includes the same learning information and flag data for identifying the presence or absence of registration.
18. The information processing apparatus according to any one of claims 14 to 17, wherein the search means simultaneously reads out all the corresponding divided memories with the divided input data, and uses the matching read data as the learning information. ..
    

Images