WO2013147022A1 - 集合演算機能を備えたメモリ及びこれを用いた集合演算処理方法 - Google Patents
集合演算機能を備えたメモリ及びこれを用いた集合演算処理方法 Download PDFInfo
- Publication number
- WO2013147022A1 WO2013147022A1 PCT/JP2013/059260 JP2013059260W WO2013147022A1 WO 2013147022 A1 WO2013147022 A1 WO 2013147022A1 JP 2013059260 W JP2013059260 W JP 2013059260W WO 2013147022 A1 WO2013147022 A1 WO 2013147022A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- information
- pattern
- image
- memory
- match
- Prior art date
Links
- 230000006870 function Effects 0.000 title claims abstract description 209
- 238000000034 method Methods 0.000 title claims description 152
- 238000012545 processing Methods 0.000 title claims description 108
- 230000010365 information processing Effects 0.000 claims description 102
- 238000005070 sampling Methods 0.000 claims description 80
- 238000001514 detection method Methods 0.000 claims description 74
- 230000008569 process Effects 0.000 claims description 36
- 238000003708 edge detection Methods 0.000 claims description 18
- 230000002829 reductive effect Effects 0.000 claims description 17
- 238000001228 spectrum Methods 0.000 claims description 17
- 238000004364 calculation method Methods 0.000 claims description 14
- 238000011425 standardization method Methods 0.000 claims description 12
- 230000036961 partial effect Effects 0.000 claims description 10
- 230000008859 change Effects 0.000 claims description 9
- 239000004065 semiconductor Substances 0.000 claims description 8
- 238000005259 measurement Methods 0.000 claims description 7
- 230000009466 transformation Effects 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 claims description 2
- 150000001875 compounds Chemical class 0.000 claims 3
- 238000010586 diagram Methods 0.000 description 58
- 238000005516 engineering process Methods 0.000 description 29
- 241000282414 Homo sapiens Species 0.000 description 20
- 239000003086 colorant Substances 0.000 description 19
- 239000000523 sample Substances 0.000 description 14
- 210000001508 eye Anatomy 0.000 description 10
- 238000003491 array Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 8
- 241000282412 Homo Species 0.000 description 7
- 238000006243 chemical reaction Methods 0.000 description 7
- 241000238633 Odonata Species 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 6
- 239000011521 glass Substances 0.000 description 6
- 210000004556 brain Anatomy 0.000 description 5
- 230000001965 increasing effect Effects 0.000 description 5
- 230000009467 reduction Effects 0.000 description 5
- 230000008901 benefit Effects 0.000 description 4
- 241000220225 Malus Species 0.000 description 3
- 241000700605 Viruses Species 0.000 description 3
- 230000002354 daily effect Effects 0.000 description 3
- 238000004880 explosion Methods 0.000 description 3
- 238000000059 patterning Methods 0.000 description 3
- 230000008685 targeting Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- 239000002699 waste material Substances 0.000 description 3
- YIWGJFPJRAEKMK-UHFFFAOYSA-N 1-(2H-benzotriazol-5-yl)-3-methyl-8-[2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidine-5-carbonyl]-1,3,8-triazaspiro[4.5]decane-2,4-dione Chemical compound CN1C(=O)N(c2ccc3n[nH]nc3c2)C2(CCN(CC2)C(=O)c2cnc(NCc3cccc(OC(F)(F)F)c3)nc2)C1=O YIWGJFPJRAEKMK-UHFFFAOYSA-N 0.000 description 2
- VZSRBBMJRBPUNF-UHFFFAOYSA-N 2-(2,3-dihydro-1H-inden-2-ylamino)-N-[3-oxo-3-(2,4,6,7-tetrahydrotriazolo[4,5-c]pyridin-5-yl)propyl]pyrimidine-5-carboxamide Chemical compound C1C(CC2=CC=CC=C12)NC1=NC=C(C=N1)C(=O)NCCC(N1CC2=C(CC1)NN=N2)=O VZSRBBMJRBPUNF-UHFFFAOYSA-N 0.000 description 2
- 108020004414 DNA Proteins 0.000 description 2
- MKYBYDHXWVHEJW-UHFFFAOYSA-N N-[1-oxo-1-(2,4,6,7-tetrahydrotriazolo[4,5-c]pyridin-5-yl)propan-2-yl]-2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidine-5-carboxamide Chemical compound O=C(C(C)NC(=O)C=1C=NC(=NC=1)NCC1=CC(=CC=C1)OC(F)(F)F)N1CC2=C(CC1)NN=N2 MKYBYDHXWVHEJW-UHFFFAOYSA-N 0.000 description 2
- NIPNSKYNPDTRPC-UHFFFAOYSA-N N-[2-oxo-2-(2,4,6,7-tetrahydrotriazolo[4,5-c]pyridin-5-yl)ethyl]-2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidine-5-carboxamide Chemical compound O=C(CNC(=O)C=1C=NC(=NC=1)NCC1=CC(=CC=C1)OC(F)(F)F)N1CC2=C(CC1)NN=N2 NIPNSKYNPDTRPC-UHFFFAOYSA-N 0.000 description 2
- AFCARXCZXQIEQB-UHFFFAOYSA-N N-[3-oxo-3-(2,4,6,7-tetrahydrotriazolo[4,5-c]pyridin-5-yl)propyl]-2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidine-5-carboxamide Chemical compound O=C(CCNC(=O)C=1C=NC(=NC=1)NCC1=CC(=CC=C1)OC(F)(F)F)N1CC2=C(CC1)NN=N2 AFCARXCZXQIEQB-UHFFFAOYSA-N 0.000 description 2
- 108091028043 Nucleic acid sequence Proteins 0.000 description 2
- 208000003443 Unconsciousness Diseases 0.000 description 2
- JAWMENYCRQKKJY-UHFFFAOYSA-N [3-(2,4,6,7-tetrahydrotriazolo[4,5-c]pyridin-5-ylmethyl)-1-oxa-2,8-diazaspiro[4.5]dec-2-en-8-yl]-[2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidin-5-yl]methanone Chemical compound N1N=NC=2CN(CCC=21)CC1=NOC2(C1)CCN(CC2)C(=O)C=1C=NC(=NC=1)NCC1=CC(=CC=C1)OC(F)(F)F JAWMENYCRQKKJY-UHFFFAOYSA-N 0.000 description 2
- 230000009471 action Effects 0.000 description 2
- 235000021016 apples Nutrition 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 239000004744 fabric Substances 0.000 description 2
- 230000020169 heat generation Effects 0.000 description 2
- 238000007781 pre-processing Methods 0.000 description 2
- 239000013074 reference sample Substances 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 230000002441 reversible effect Effects 0.000 description 2
- VCUFZILGIRCDQQ-KRWDZBQOSA-N N-[[(5S)-2-oxo-3-(2-oxo-3H-1,3-benzoxazol-6-yl)-1,3-oxazolidin-5-yl]methyl]-2-[[3-(trifluoromethoxy)phenyl]methylamino]pyrimidine-5-carboxamide Chemical compound O=C1O[C@H](CN1C1=CC2=C(NC(O2)=O)C=C1)CNC(=O)C=1C=NC(=NC=1)NCC1=CC(=CC=C1)OC(F)(F)F VCUFZILGIRCDQQ-KRWDZBQOSA-N 0.000 description 1
- 241000287181 Sturnus vulgaris Species 0.000 description 1
- 230000002159 abnormal effect Effects 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000009825 accumulation Methods 0.000 description 1
- 230000002411 adverse Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000001186 cumulative effect Effects 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 238000007418 data mining Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 210000004709 eyebrow Anatomy 0.000 description 1
- 210000000744 eyelid Anatomy 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000007667 floating Methods 0.000 description 1
- 210000002683 foot Anatomy 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 210000004247 hand Anatomy 0.000 description 1
- 210000003128 head Anatomy 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000009191 jumping Effects 0.000 description 1
- 238000013332 literature search Methods 0.000 description 1
- 210000000214 mouth Anatomy 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 238000003909 pattern recognition Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000003672 processing method Methods 0.000 description 1
- 238000001454 recorded image Methods 0.000 description 1
- 230000001932 seasonal effect Effects 0.000 description 1
- 230000035945 sensitivity Effects 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000007619 statistical method Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/903—Querying
- G06F16/90335—Query processing
- G06F16/90339—Query processing by using parallel associative memories or content-addressable memories
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/28—Constructional details of speech recognition systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/56—Information retrieval; Database structures therefor; File system structures therefor of still image data having vectorial format
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/02—Feature extraction for speech recognition; Selection of recognition unit
- G10L2015/025—Phonemes, fenemes or fenones being the recognition units
Definitions
- the present invention relates to a memory having a set operation function and a set operation processing method using the same.
- a set operation by a program using a CPU is a process of searching for specific information from set information data stored in a memory.
- Information (original) on the memory is sequentially accessed and referenced. In this way, the solution of the set operation is obtained.
- the concept of information processing will change if a processor that can process a set of information (the entire memory) in a batch, such as the concepts of Euler and Venn diagrams, can be created. This is because an element that performs a logical operation on a set of information collectively can be expected to perform a logical operation at a speed that is not comparable compared to individual sequential information processing for the original.
- Recognition in information processing is a technology that finds some features from the target information and applies them to concepts such as nouns and adjectives that we can understand and judge from those features. Some features usually need to be looked up individually, and the information needs to be found repeatedly over and over.
- pattern matching is one of the most important technologies for knowledge processing, and is a basic technology that is the backbone of pattern recognition and the mainstay. It is indispensable for recognition information processing in all fields of images, speech, and text. is there.
- pattern match is a typical example of information set logical operations, there is currently no specialized processor for information set operations including pattern match. It has become.
- pattern matching is defined and implemented so that it can be used universally and commonly for any information, and if an element processor dedicated to collective logic operations that extends this pattern matching concept can be realized, information processing You will get enormous benefits. (About "pattern matching" in information processing) Next, the importance and outline of patterns and pattern matching in information will be explained.
- the information we want to search for and recognize is not an independent piece of data, it is a block of data (array pattern).
- the image data to be recognized is a collection of pixel data
- the sound data to be recognized is a collection of sound spectra.
- the information to be recognized is a block of information, the pattern itself, which is nothing but to solve the set operation of information.
- patterns and pattern matching are extremely diverse and vague concepts that have not been standardized or generalized.
- Japanese Patent Application No. 4-298874 is a fuzzy set processing arithmetic unit, a storage device and a computer system. Absent.
- An object of the present invention is to provide a processor capable of performing collective processing by performing a set operation on information expressed in words.
- a memory capable of storing information for each memory address and reading the information is stored in each memory address given from the outside.
- Judgment based on Means for logical operation on result memory with set operation function and having a means for outputting the set operation result is provided.
- the memory repeatedly executes a set operation with the first to third inputs newly given to the set operation result with the first to third inputs. It has a means.
- means for executing at least one of the set operations of the information by the first to third inputs in parallel processing are provided.
- the first input includes a value representing information to be compared and designation of a complete match, partial match, range match, or a combination thereof as a comparison condition.
- the determination by the first input is realized by an associative memory means.
- the second input includes a position of information to be compared, a certain area based on the position, or a combination thereof.
- the position of the information to be compared in the second input includes a relative position, an absolute position, or a combination thereof.
- the means for performing the determination based on the second input includes means for executing the memory address by operating in parallel.
- the input means is for further inputting a fourth input (image size or the like) for designating an arrangement / order of information, and the fourth input The determination of the information is executed based on the arrangement / order of the information specified in the above.
- the first to third inputs specify inquiry information patterns that perform pattern matching with respect to the set information stored in the memory.
- the inquiry information pattern is inquiry information for edge detection.
- the pattern matching may be any one of one-dimensional information using character information as an example, two-dimensional information using image information as an example, three-dimensional information using movie information as an example, and N-dimensional information with an array defined. It is desirable that it be executed on the other side.
- at least one of visual recognition, auditory recognition, taste recognition, odor recognition, and tactile recognition is processed by the inquiry information pattern of the pattern matching.
- the memory is integrated with another semiconductor, for example, a CPU.
- an apparatus including a memory having a set operation function according to claim 1 is provided.
- the set information (entire memory) stored in the memory itself can be obtained without performing any set operation of information on the source of information (individual memory information) by the CPU.
- a memory (element) that has a set operation function that can be used in common for any process of searching for information, that is, search, search, collation, and recognition by performing a set operation in a lump. ) Can be realized.
- This configuration makes it possible to standardize and generalize the concept of pattern matching and edge detection, which are the most basic and most important technologies for information recognition, which was a weak point of current computers.
- FIG. 1 is an Euler diagram illustrating the concept of a set operation.
- FIG. 2 is an Euler diagram including the concept of position and region.
- FIG. 3 is an example of a block diagram of a content addressable memory (CAM).
- FIG. 4 is an example of a data comparison circuit of a content addressable memory (CAM).
- FIG. 5 is a first example of a block diagram of a memory having an information narrowing function.
- FIG. 6 is an example of full-text detection using a memory having an information narrowing function.
- FIG. 7 is a second example of a block diagram of a memory having an information narrowing function.
- FIG. 8 is an example 1 of image detection by a memory having an information narrowing function.
- FIG. 9 is an example 2 of image detection by a memory having an information narrowing function.
- FIG. 1 is an Euler diagram illustrating the concept of a set operation.
- FIG. 2 is an Euler diagram including the concept of position and region.
- FIG. 3 is an example of a
- FIG. 10 is an example 3 of image detection by a memory having an information narrowing function.
- FIG. 11 is an example 4 of image detection by a memory having an information narrowing function.
- FIG. 12 is an example 5 of image detection by a memory having an information narrowing function.
- FIG. 13 is an example 6 of image detection by a memory having an information narrowing function.
- FIG. 14 is an example 7 of image detection by a memory having an information narrowing function.
- FIG. 15 is an example 8 of image detection by a memory having an information narrowing function.
- FIG. 16 is an example 9 of image detection by a memory having an information narrowing function.
- FIG. 17 is an example 10 of image detection by a memory having an information narrowing function.
- FIG. 18 is an example 11 of image detection using a memory having an information narrowing function.
- FIG. 19 shows an example of a graphic user interface (GUI) of a memory having an information narrowing function.
- GUI graphic user interface
- FIG. 20 shows an example of detecting one-dimensional information.
- FIG. 21 shows an example of detection of two-dimensional information.
- FIG. 22 shows an example of detection of three-dimensional information.
- FIG. 23 shows an example of ambiguous detection of one-dimensional information.
- FIG. 24 shows an example of ambiguous detection of two-dimensional information.
- FIG. 25 shows an example of ambiguous detection of three-dimensional information.
- FIG. 26 shows an ambiguity detection example 2 of two-dimensional information.
- FIG. 27 shows an example of coordinate conversion of two-dimensional information.
- FIG. 28 is an example of a block diagram of a memory having a set operation function.
- FIG. 28 is an example of a block diagram of a memory having a set operation function.
- FIG. 29 is an example of a detailed block diagram of a memory having a set operation function.
- FIG. 30 shows an example of a graphic user interface (GUI) for document search.
- FIG. 31 is a first example of set operation using a memory having a set operation function.
- FIG. 32 shows a set operation example 2 using a memory having a set operation function.
- FIG. 33 shows a set operation example 3 using a memory having a set operation function.
- FIG. 34 is a fourth example of a set operation by a memory having a set operation function.
- FIG. 35 is an example of edge detection by a memory having a set operation function.
- FIG. 36 is an explanatory diagram of image patterns and image pattern matching.
- FIG. 37 is an explanatory diagram of the principle of image pattern matching using an information narrowing memory.
- FIG. 38 is an explanatory diagram of an example of an area / contour of an image.
- FIG. 39 is an explanatory diagram of exclusive pattern matching of images.
- Example 1-1 FIG. 40 is an explanatory diagram of edge code encoding using a neighborhood 4-pixel pattern.
- Example 1-2 FIG. 41 is an explanatory diagram of edge code encoding using a neighboring 8-pixel pattern.
- FIG. 42 is an explanatory diagram of an information array of image pattern matching using an information narrowing memory.
- FIG. 43 is an explanatory diagram of an application example of an edge code of an object.
- FIG. 44 is an explanatory diagram of random and random pattern matching by local pattern matching. (Example 1-6)
- FIG. 1-6 FIG.
- FIG. 45 is an explanatory diagram of object change image detection.
- FIG. 46 is an explanatory diagram of detection of corresponding points of an object by local pattern matching.
- FIG. 47 is an explanatory diagram of object recognition using edge codes.
- FIG. 48 is an explanatory diagram of person recognition using stereoscopic measurement.
- FIG. 49 is an explanatory diagram of object recognition in space.
- Example 1-10 FIG. 50 is a conceptual explanatory diagram of object recognition by pattern matching.
- FIG. 51 shows a reference example of the amplitude waveform of a phoneme.
- FIG. 52 is a reference example A of the frequency spectrum waveform of a phoneme.
- FIG. 51 shows a reference example of the amplitude waveform of a phoneme.
- FIG. 52 is a reference example A of the frequency spectrum waveform of a phoneme.
- FIG. 51 shows a reference example of the amplitude waveform of a phoneme.
- FIG. 52 is a reference example A of the frequency spectrum waveform of a
- FIG. 53 is a reference example B of the frequency spectrum waveform of a phoneme.
- FIG. 54 is an example of range data for phoneme discrimination.
- FIG. 55 is an example of phoneme recognition by a memory having an information narrowing function.
- FIG. 56 shows an example of vocabulary pattern matching.
- FIG. 57 is an explanatory diagram of image patterns and image pattern matching.
- FIG. 58 is an explanatory diagram of the principle of image pattern matching using an information narrowing memory.
- FIG. 59 is an explanatory diagram of exclusive pattern matching.
- FIG. 60 shows an example of a character typeface.
- 61 is an explanatory diagram of a character pattern sampling point creation example A.
- FIG. 62 is an explanatory diagram of a character pattern sampling point creation example B.
- FIG. 63 is an example of creating a character pattern sampling point for a specific typeface.
- FIG. 64 is an example of character recognition of an image with subtitles.
- FIG. 65 is an example of an information processing apparatus having a real-time OCR function.
- FIG. 66 is an example of character recognition of a document image.
- FIG. 67 is an example of pattern matching of one-dimensional information.
- FIG. 68 shows an example of pattern matching of two-dimensional information.
- FIG. 69 shows an example of a one-dimensional information pattern matching GUI.
- FIG. 70 shows an example of a GUI for two-dimensional information pattern matching.
- FIG. 71 shows an example of a GUI for image information pattern matching.
- FIG. 72 is a conceptual diagram of information processing for pattern matching using this method.
- the present invention provides a processor having an information set calculation function for performing batch calculation processing on a set of information.
- Such a processor By implementing such a processor, it can be used in common for any process that searches for information, that is, search, search, collation, and recognition. Arbitrary set operations are possible, and pattern matching and edge detection are realized at high speed. Therefore, without using a large-scale system, dedicated LSI, special software algorithm, or supercomputer, high-speed hardware pattern matching that is the starting point of recognition processing such as image, voice, and character recognition Edge detection technology can be made general-purpose technology, and full-fledged knowledge processing by computers can be made familiar to us. (Invention described in the patent application on which the priority claim is based) Prior to this application, the inventor of the present application has filed a patent application that serves as a basis for claiming priority as follows.
- Japanese Patent Application No. 2012-083361 relates to a phoneme recognition method, a vocabulary recognition method, and a speech pattern matching method.
- Japanese Patent Application No. 2012-101352 relates to an image recognition method, an object recognition method, and a pattern matching method.
- Japanese Patent Application No. 2012-110145 relates to an image character recognition method, an information processing apparatus having an image character recognition function, and the above three are related to the recognition of speech, images, and characters of the three major human recognitions.
- Japanese Patent Application No. 2012-121395 is a pattern matching standardization method and a pattern matching GUI standardization method that summarizes common items essential for pattern matching related to recognition, and summarizes the minimum necessary contents for general-purpose and standardized pattern matching. It is a thing.
- pattern matching is fundamental, and information processing, which is the basis of pattern matching, is performed using both information and the position of the information as input conditions, and the information processing result is output. These are obtained by giving calculation conditions to images, character information of images, and collective information such as sound, and calculating the calculation results.
- the present invention provides a processor capable of implementing the above concept.
- the final object of the present invention is to realize a logical operation processor that is completely unaware of the operation time as in the concept of a set of mathematics. (About "sets” in information processing) As mentioned above, according to Wikipedia, a set in mathematics is roughly a “collection” made up of several “things”. It is introduced that it is based on the individual “things” that make up the set.
- FIG. 1 is an Euler diagram showing the concept of set operation.
- Euler diagram is a concept for making it easy to understand the concept of set theory. From a whole set 103, a specific element 105 and its subset 104 are represented. It is frequently used when summarizing ideas for finding out.
- the element 105 to be searched for from the whole set information that is, the information (A and B shown in the figure) is designated as a subset, and the logical negation 111, logical sum 109, logical
- all set operations 115 such as logical differences and logical objects can be performed by the operation of the product 110 and combinations thereof, and this idea forms the basis of current information processing (computer).
- Fig. 2 is an Euler diagram including the concept of position and area.
- the set operation 115 described in FIG. 1 is combined with the concept of the information position 106 and the area 107. (Significance of information location in information set operations)
- the set operation that is, the process of finding information, is that the physical structure of the memory itself for which information is found is composed of only two elements, an address and a memory cell. There is nothing other than specifying where the information is on the address, and what the information is at a certain address.
- the information position 106 in FIG. 2 means the information data position 414 which is a specific address in the memory
- the area 107 means the information data area 415 which is a specific address.
- the calculation is performed by designating the position and region of the time
- This is a set operation 115 for calculating the position and area of the image and determining where the calculation result is located.
- the previous set operation 115 is for the element 105, the presence of the address 203 is an implicit condition, and although there is no need for special explanation, the set operation 115 is performed for the entire memory. This is a concept that cannot be avoided for batch calculation, and is an essential concept for pattern matching and edge detection, which will be described later in detail.
- the position 106 and the area 107 of information (information data) having these various and extremely important meanings are expressed and described as the information location 114.
- a set operation 115 is used in databases in various fields from a large database to a small database.
- a typical example of such a set operation 115 is a patent document search system of the JPO.
- Data mining which was born as one of the new industries of the information processing 101, just performs such a set operation 115 by changing the name.
- Information processing 101 of these set operations 115 is performed by an information processing action of the CPU based on a normal program.
- the Euler diagram for explaining the set theory is expressed by mathematical interpretation and it is difficult to understand the existence of the element 105, but in the actual information processing 101, the set operation 115 is performed for the element 105.
- the present invention is A memory capable of storing information for each memory address and reading the information.
- the memory is provided with a first input 221 for comparing information stored in each memory address provided from the outside, and each memory.
- a third input 223 for designating a selectable combination and means 208, 209 for determining by comparing with information stored in the memory for each address based on the first input; Based on the second input, the means 210 and 211 for comparing and determining the information stored in the memory, and the determination result based on the first and second inputs based on the third input.
- the memory of the present invention is a processor based on an associative memory that has a large potential and has not been able to fully demonstrate its ability. Therefore, first, the associative memory will be described.
- FIG. 3 is an example of a block diagram of a content addressable memory (CAM).
- CAM content addressable memory
- associative memory (CAM) 301 is known as a memory-based architecture device, that is, a device in which the memory itself performs information processing independently.
- This associative memory (CAM) 301 has a structure in which memory cells 202 are arranged for each memory address 203 as in a normal memory. Information can be read from and written into the memory cell 202 as in a normal memory, and at the same time, a data comparison circuit 208. In this device, the comparison operation can be performed in parallel based on the data condition 221 given from the outside and the result can be output.
- the address designation from the address bus is decoded by the address decoder circuit 206, the address is selected, and data can be written to and read from the memory, and at the same time, the memory addresses are arranged in parallel.
- the data comparison circuit 208 detects in parallel the memory cells 202 that are externally applied and meet the data condition 221, and in this example, the priority address encoder 207 outputs the detection result to the match address bus. ing.
- the content addressable memory (CAM) 301 having the above contents is generally of the exact match type and has a limited range of information that can be used practically. Therefore, it is currently used for detecting IP addresses of Internet communication devices. It is about.
- FIG. 4 shows an example of the data comparison circuit of the content addressable memory (CAM).
- the content addressable memory (CAM) 301 shown in the figure has a 1-byte data width of 1 byte, that is, an 8-bit configuration. However, if the data width of 1 address is free and appropriate for the target information, Good.
- the data comparison circuit 208 and the data range comparison circuit 209 can be configured to compare the magnitude of data and the comparison of coincidence. This greatly expands the types of information that can be used.
- image information 405 or the like is obtained by converting analog data having continuity into digital data, and in order to handle these data, a perfect match comparison is insufficient and a comparison with a range is indispensable. .
- the color 402 information of the image information 405 is formed as a pixel 406 by combining, for example, three colors of R (red), G (green), and B (blue). Data match, green data match, blue data match, etc. correspond to this.
- an important element is a set operation 115 including a data location 114.
- a parallel operation is indispensable in order to perform partial matching subsets a plurality of times and perform a set operation 115 of the information data location 114 at a high speed, but parallel four rules for each address of the content addressable memory (CAM) 301. Since it is not a practical method to provide the arithmetic units individually, it is not surprising to rely on sequential information processing devices such as CPUs and GPUs. That is, until now, even though the set operation 115 of the information data condition 221 can be realized by the associative memory (CAM) 301, it means that there is no technique of the parallel set operation 115 including the location 114 of the information data. That is, the associative memory (CAM) 301 cannot be used as a one-handed device because it cannot perform the set operation 115 on the information location 114.
- FIG. 5 is a first example of a block diagram of a memory having an information narrowing function.
- the memory 302 having this information narrowing function is an address comparison circuit 210 for detecting the data location 114 based on the address condition 222 given from the outside to the function of the content addressable memory (CAM) 301 described above.
- CAM content addressable memory
- FIG. 5 is a first example of a block diagram of a memory having an information narrowing function.
- the memory 302 having this information narrowing function is an address comparison circuit 210 for detecting the data location 114 based on the address condition 222 given from the outside to the function of the content addressable memory (CAM) 301 described above.
- CAM content addressable memory
- the memory 302 having the information narrowing function is configured such that the address comparison circuit 210 and the address area comparison circuit 211 attached in parallel to the output of the associative memory (CAM) 301 have the information location 114, that is, the address position. 106 and area 107 are designated, and information is narrowed down, that is, a logical product 110 set of information can be formed.
- CAM associative memory
- the address comparison circuit 210 and the address area comparison circuit 211 can be realized by a memory address parallel operation 216 that performs relocation of the address of the output flag of the content addressable memory (CAM) 301, and this configuration is realized by the content addressable memory (CAM) 301.
- CAM content addressable memory
- the memory 302 having the information narrowing function shown in this example is a configuration that can be simply realized by, for example, a one-dimensional (linear array) shift register, and is optimal for one-dimensional array information.
- FIG. 6 is an example of full text detection.
- a character string that is a set 102 of character data is stored as a database 407 in a memory 302 having an information narrowing function.
- the character “information” is detected by the data condition 221 given from the outside by the associative memory (CAM) 301 function of the memory 302 having the information narrowing function. This detection result becomes reference information 421 for future character detection.
- CAM associative memory
- the character “information” is similarly detected, and the secondary determination result is shifted by one address, and the entire address is shifted in parallel to the left side of the figure.
- the character “processing” is detected in the same manner, and the third determination result is shifted by two addresses, and the entire address is shifted in parallel to the left side of the figure.
- the character “physical” is similarly detected, and the quaternary determination result is shifted to the left side of the figure by paralleling the quaternary determination result by three addresses.
- the address at which the above four determination result flags are arranged is the match address 213 as shown in FIG.
- the address of the absolute address 204n ⁇ 0 is the head address of the character string “Information, information, processing, reason”.
- the match number counter 212 is equivalent to storing the cumulative AND 110 operation result.
- the order of “information, information, processing, and information” is not necessarily required.
- the shift direction of the address 203 is reversed, "Becomes the reference information 421 and becomes the match address 213. Furthermore, even if “process, reason, information, information” is skipped, any arrangement is possible as long as the correspondence between the direction and the position in the shift comparison of the address 203 has been made accurately.
- the information specified first is the reference information 421, and the reference information 421 is sequentially narrowed down and only the address that has been won to the end is detected.
- the amount by which the address is shifted by the parallel operation 216 of the memory address, that is, the location of the information to be compared is the relative position of each other, that is, the relative address 205, and the address detected as a result is the absolute address 204. That is. Therefore, any information can be searched if only the combination order of the source 105 of the information to be searched is known. Conventionally, when such a search is performed at high speed, it has been necessary to consider a special algorithm such as an index table or an array method that makes it easy to retrieve frequently searched information. This technique, which had to change tables and algorithms, eliminates the need for pre-processing of these information data.
- the above character string is representative of the pattern 401 described in the background art, and as described above, the character string as described above can be used as an associative memory (CAM) 301 function without performing a normal search.
- CAM associative memory
- the memory address parallel operation 216 for changing the position of the output flag can be easily detected by a shift operation of several clocks of the shift register.
- the set operation 115 of the entire information set by such a completely parallel method makes it unnecessary to scan (search) the individual (information source) of the memory space of the CPU at all, so it is not comparable to the conventional information processing 101. Faster full text detection is possible.
- FIG. 7 is a second example of a block diagram of a memory having an information narrowing function.
- the memory 302 having the information narrowing function shown in this example includes an address comparison circuit 210 and an address area comparison circuit 211 that perform the set operation 115 on the information location 114 described above in two dimensions (X-axis and Y-axis 2).
- Axis shift register which is optimal for two-dimensional array information.
- FIGS. 8 to 18 illustrate the concept of image detection that is two-dimensional information by the memory 302 having the above-structured information narrowing function. As shown in FIG. 8, image information 405 that is a set 102 of pixels 406 is stored in an array in a memory 302 having an information narrowing function.
- information data of four colors 402 of black, red, blue and green are arranged and stored in each address 204 from addresses 0 to N-1 shown in the figure.
- the information data value 117 may be of any type, such as the luminance 403 other than the color 402, or other types of information data. It is exactly the same as storing information in a normal memory.
- the inquiry pattern 408 is a pattern composed of sampling points 410 of three pixels indicated by black, red, and blue pixels 406.
- the two-dimensional information pattern match 409 as described above is easy to understand when grasped by the concept shown in FIG.
- pattern matching may be performed in any order.
- pattern matching 409 is performed in the order of “black, red, blue” pixels 406. I do.
- FIG. 10 shows a state where the black pixels 406 are detected in parallel by the content addressable memory (CAM) 301 function.
- CAM content addressable memory
- Three black pixels 406 are detected as coordinates 404 and data position 414.
- the information data 412 116 that matches the value 107 of the designated information data indicates the location 114 of the pixel 406, that is, the position 414 of the data by coordinates 404.
- the coordinates 404 and the data position 414 of the three black pixels described above will be reference information 421 for future pattern matching. As shown in FIG. 11, the coordinates 404 and the data position 414 of the reference information 421 are shown. A window is opened at the position of the mask 217 so that black pixels can be seen.
- the match number counter 212 is incremented to “1”.
- the match number counter 212 of the three pixels 406 in the entire image set becomes “1”, which means that there is a possibility that a pattern similar to the inquiry pattern 408 exists around this.
- the three pixels are red.
- red pixel coordinates 404 and data position 414 and the black pixel position detected earlier that is, coordinates 404 and data position 414 are calculated as shown in FIG.
- the mask 217 carrying the reference information 421 defined by the black pixel 406 is masked by an amount corresponding to the position of the black and red pixels, that is, the coordinate 404 and the data position 414 of the inquiry pattern. Try moving 217.
- the red pixel can be seen only from the position of the reference information 421, the coordinates 404, and the data position 414 of the reference information 421 that opened the window earlier, in the upper pixel in the drawing.
- the match number counter 212 of the reference information 421 is counted up to “2”, and the game remains unsuccessful, and the match number counters 212 of the other two reference positions remain “1” and are dropped.
- the next is detection of the blue pixel 406.
- 6 pixels are detected.
- the mask is moved by an amount corresponding to the position of the black and blue pixels of the inquiry pattern 408, that is, the coordinates 404 and the data position 414.
- the blue pixel can be seen only from the position of the reference information 421, the coordinates 404, and the data position 414 of the reference information 421 where the window is opened in the mask 217, only in the upper part of the drawing.
- the match number counter 212 of the reference information 421 is counted up to “3”, and the match is left, and the match number counters 212 of the other two reference positions remain “1”.
- the black of the inquiry pattern 408 is used as the reference information 421
- the positions of the two pixels 406 of red and blue, that is, the coordinates 404 and the data position 414 are the coordinates 404 and the data whose counter value of the match number counter 212 is “3”.
- the position 414 of the pattern 409 is a pattern match 409, and the match address 213 has been won and detected.
- the movement of the mask 217 described above is realized by the memory address parallel operation 216 by the address comparison circuit 210 and the address area comparison circuit 211.
- the coordinates 404 and the data position 414 of the inquiry pattern 408 which is a condition to be given from the outside are designated by the relative distance 108 between the pixels, that is, the relative address 205, and the pattern match 409 of the calculation result
- the ability to output the match address 213 as the absolute address 204 results in a reduction in the processing load of the subsequent process.
- this example is an image for explanation, it is an example of a pattern match 409 by an inquiry pattern 408 of an extremely small size and an extremely small sampling point 410.
- the size of the image becomes large, narrowing down in terms of probability Since the effect is great, it is generally possible to expect a pattern match 409 in which the match address 213 is sufficiently narrowed down by an inquiry pattern 408 of several to a dozen pixels.
- the above image pattern match is based on the associative memory (CAM) 301 function and a shift operation of several clocks of the shift register. Since the vector calculation of the location of the information based on the information is completely unnecessary, the detection can be performed at a speed higher than that of the conventional method.
- CAM associative memory
- FIG. 16 is an enlarged view of the reference information 421 region by ⁇ 1 region for both the X axis and the Y axis. In this manner, by providing an area at the coordinates 404 and the data position 414, an ambiguous pattern match 418 is possible.
- Such a pattern match 409 based on the ambiguous pattern 417 includes not only the area of the information position but also the range of the information data value, the number of mismatches of the number of times measured by the match number counter 212 is allowed 425, and the like.
- the ambiguous recognition 419 by the ambiguous pattern match 418 that is extremely rational and conforms to the human sensitivity can be performed.
- FIG. 17 shows an example in which the reference information 421 is black, and the area of the coordinates 404 and the data position 414 on both the X and Y axes is ⁇ 2.
- the pattern match 409 first determines the reference information 421 and the location 114 of the reference information based on the result by the method of specifying the reference information.
- a method for designating the location 114 of information such as the position 106 and the area 107 for 204 will be described.
- the image stored in the memory 302 having the information narrowing function is all white, and as shown in FIG. 18, specific color information, “green” in this example, is displayed at the target coordinates 404 and the data position 414.
- specific color information “green” in this example
- One pixel of the above image is detected by the associative memory (CAM) function 301 function described above, and this detection result flag is enlarged by the address area comparison circuit 211 to enlarge the area of the flag.
- the absolute position and its area, that is, the place 114 by the absolute address 204 can be designated.
- Such designation of the absolute address space can be used when detecting a color histogram or density of a limited space.
- Such absolute address area density detection is optimal for detecting the presence of an object having a color area such as a human face or hand, or a special color 402 or brightness 403.
- the address condition is the entire area
- the entire memory is the target of the operation
- the operation result is only for the corresponding area.
- pattern matching 409 which is the basis of recognition technology, is performed by means for specifying the value 117 of information (data) and means for specifying the location 114 of information (data).
- the information (data) value 117 is intended for various types of information and their matching 116, and the information location 114 is intended for both the information location 106 and the information area 107. In addition, the information location 114 is intended to be both a relative location and an absolute location.
- the concept of information location 114 in information processing is an extremely vague concept that is extremely broad and elusive, but an example of standardization of information set operations of information types that are indispensable for performing information processing will be described.
- FIG. 19 shows an example of a graphic user interface (GUI) of this memory.
- GUI graphic user interface
- GUI general-purpose graphic user interface
- the reference information 421 is used as a reference, and in this example, the match information 422 in the match order 420 from M1 to M16 is the information data 412, the range 413, and the information location 114, that is, the position of the information data.
- the match information 422 in the match order 420 from M1 to M16 is the information data 412, the range 413, and the information location 114, that is, the position of the information data.
- it is configured to include a function for permitting 425 specification of data array 411, coordinate conversion 428 of information location, and mismatch count of match count counter.
- the memory 302 having the information narrowing function performs pattern matching based on the specification, and the graphic user interface uses the match address 213 as the absolute address 204.
- a structure returned to (GUI) may be used.
- FIG. 20 shows an example of detecting one-dimensional information. This is an image for finding information that matches (matches) a query pattern from time-series collective information such as seasonal temperature changes and economic trends, that is, one-dimensional information. Text is also a member of these one-dimensional information.
- a database 407 that is an overall set 103, which is a set 102 of information sources 105 in which an array 411 of data is defined and stored in an absolute address 204.
- an inquiry pattern 408 shown on the right side of the figure is a pattern of information to be searched for consisting of several sampling points 410, and each sampling point 410 has reference information 421 in which the relationship between data and its location is specified.
- the match information 422 forms a set of inquiry patterns 401.
- each information has a relative distance between its data value (D value in the figure) and the reference information 421, in this case, a relative address 205 (X value in the figure).
- the pattern match 409 is determined by means for searching for information that matches the specified information (data) 116 in the memory having an information narrowing function and means for searching for the location 114 of the information (data).
- the match address 213 is output as the absolute address 204.
- FIG. 21 shows an example of detection of two-dimensional information.
- FIG. 22 shows an example of detection of three-dimensional information.
- FIG. 23 shows an example of ambiguous detection of one-dimensional information.
- 21 is an image of an ambiguous pattern match 418 that matches (matches) ambiguous inquiry information from the one-dimensional information shown in FIG.
- the data range 413 is specified for the information data, and the area 107 is also specified for the information location.
- the pattern matching as described above is, for example, optimal for fields such as stock price fluctuation patterns, temperature fluctuation patterns, and speech recognition phoneme pattern detection (recognition).
- FIG. 24 shows an example of ambiguous detection of two-dimensional information.
- the pattern matching as described above is, for example, optimal for fields such as high-speed detection of a human face position and non-face portion in an image and high-speed character reading of a car license plate.
- FIG. 25 shows an example of fuzzy detection of 3D information.
- the pattern matching as described above is, for example, optimal for fields such as molecular structure identification, space constellation identification, and weather data analysis.
- FIG. 26 shows an example 2 of ambiguous detection of two-dimensional information.
- the diagram further extends the concept of ambiguity detection of the two-dimensional information shown in FIG. As shown in the figure, it is detected whether there is target information at any location 114 in the region.
- the pattern detected by such a concept greatly expands the concept of the information location 114 of the pattern match 409 and is pronounced of a mathematical set operation 115.
- FIG. 27 shows an example of coordinate conversion of two-dimensional information.
- the figure is an example in which the transformation of the information location 114 at the time of pattern matching is coordinate transformation 428.
- the pattern match 409 is effectively performed even if the image is resized or rotated by enlarging, reducing or rotating the coordinates.
- the pattern 401 is a combination of the information (data) value 117 and the location 114 of the information data, and even with a small number of sampling points 410, the pattern 401 is sufficiently narrowed down in terms of probability. It is possible to extract specific addresses, and various kinds of recognition techniques can apply this concept.
- the pattern match 409 can be performed if the information array is all defined information, and all the data arrays 411 can be standardized and generalized for the information set operation of the pattern match 409. is there.
- An ambiguous pattern match that is, a pattern match having an information (data) area 415, is a combination vector operation of information and information.
- ambiguous pattern matching with an information area is an indispensable technique for image recognition.
- pattern matching is an indispensable technology for information recognition and is the most basic technology, but it shows that pattern matching cannot be put to practical use with large-size information such as image information.
- an image space is converted into frequency component data by Fourier transform, or an edge or region is detected by analog processing.
- a general video is a series of 30 still images per second, which is 33 milliseconds per frame.
- sampling points 410 can be randomly determined from the local image space, data can be extracted, and the sampling data can be used as an inquiry pattern 408.
- Such a pattern match is most suitable for moving object recognition and stereoscopic pattern match 409.
- an intelligent camera having a recognition processing function has a built-in CPU of several tens of watts.
- the housing of the camera becomes a heat sink and becomes large and cannot be reduced in weight.
- an ultra-high speed and high-accuracy recognition function can be realized, so that the CPU does not have to have a high performance.
- the above contents have a significant meaning for portable battery devices.
- An information pattern in which the pattern match 409 is an extremely effective information processing 101 means in the information processing 101 from various viewpoints, and the memory 302 having the information narrowing function is one of information processing that the CPU is not good at. It has been shown to be effective for match 409. In the first place, the pattern match is because the physical structure of the memory itself is composed of only two elements, the address and the memory cell, so the pattern information stored in the memory is always on the address. There is nothing more than specifying what pattern is in the address.
- the superordinate concept of the pattern match 409 is the information set operation 115. It is necessary to pay attention to this first.
- the narrowing down of information necessary for pattern matching was the set operation 115 mainly composed of the logical product 110 of the subsets. This idea is further developed, and functions necessary for the set operation 115, such as logical sum 109, logical negation 111, and the like. And a function for combining and calculating these functions, the set operation 115 that has been relied on the CPU so far can be performed mathematically without performing the set operation 115 for individual information, that is, the memory 105. Like the set operation 115, the information set 102 on the memory can be collected at once, and a processor can be realized with extremely high speed, high accuracy, low power consumption, and extremely simple operation.
- FIG. 28 is an example of a block diagram of a memory according to the embodiment of the present invention.
- the memory 303 having the set operation function replaces the part of the match number counter 212 of the memory 302 having the information narrowing function with the operation circuit 224, and performs logical OR based on the logical operation condition 223 given from the outside.
- 109, logical product 110, logical negation 111, and the like can be arbitrarily realized under designated conditions.
- the memory 302 having the information narrowing function mainly performs the AND operation 110 set operation 115 by the counter and mainly performs the set information operation 115 by narrowing down the target information such as the pattern match 409. It is configured to be evolved and developed to realize an arbitrary set operation 115 of all kinds of information.
- FIG. 29 is an example of a detailed block diagram of a memory having the set operation function.
- the memory includes circuits 208 and 209 that compare data according to a data condition 221 supplied from the outside (refer to the above description for a detailed configuration) and circuits 210 and 211 that compare addresses based on an address condition 222 supplied from the outside (detailed configuration).
- a configuration in which a logical operation condition 223 given from the outside, a circuit 224 that performs a logical operation based on the above conditions, and a match address 213 of the operation result are output by the priority address encoder 207.
- the arithmetic circuit 224 includes a positive logic 112 and negative logic 113 conversion circuit and one or more winning flags 214 or an area winning flag 215.
- the output flag from the associative memory (CAM) 301 is an address condition 222.
- the winning flag 214 or the area winning flag 215 is connected to the priority address encoder 207 and output.
- the winning flag 214 or the area winning flag 215 can be used as the match count counter 212 as a counter configuration like the memory 302 having a conventional information narrowing function by performing multi-stage connection of flags.
- the output from the winning flag 214 or the area winning flag 215 is added to the inputs of the address comparison circuits 210 and 211, and the re-logical operation can be performed in parallel based on the designation of the logical operation condition 223. It has become.
- the data comparison circuit 208 and the data range comparison circuit 209 use the associative memory (CAM) 301 function based on the data condition 221 given from the outside to detect in parallel the address 116 that matches the condition, and give it from the outside.
- the address comparison circuit 210 and the address area comparison circuit 211 specify the relative address, the absolute address location 114, that is, the address position 105 and the area 107 in parallel based on the address condition 222 to be given from the outside.
- the operation 115 is performed in parallel, and the resulting match address 21 It is possible to output in prioritizer Tay address encoder 207.
- the above set operation 115 is not a set operation 115 for the element 105 on the memory, but a set operation 115 that collects a set of information on the memory.
- Such a set operation 115 method can be realized with a circuit configuration in which usually only two flags are controlled for one address. Therefore, the set operation is extremely simple and has a large information processing capacity.
- a memory 303 having a function can be manufactured.
- FIG. 30 is an example of a graphic user interface (GUI) taking the case of document search as an example.
- GUI graphic user interface
- the figure shows an outline of a graphic user interface (GUI) when full text detection such as patent information search is performed in the memory 303 having a set operation function.
- GUI graphic user interface
- condition 1 there are eight calculation conditions from condition 1 to condition 8, a character string as a keyword is designated in each condition, operator designation, and logic positive logic 112 and negative logic 113 designation. Done.
- the operator can be selected from (1) subset, (2) logical sum, (3) logical product, (4) logical negation, or a combination of two or more thereof. It is configured so that it can be specified.
- a subset of a character string (information processing) and a subset of a character string (search + detection) are obtained by a logical product of positive logic, a matching document is searched, and a character string (recognition is recognized) )
- search + detection is obtained by a logical product of positive logic
- a matching document is searched
- a character string (recognition is recognized) ) Is an example in the case of searching for a document that matches the logical product of negative logic.
- FIG. 31 to FIG. 34 are examples of a set operation by a memory 303 having a set operation function.
- a plurality of target documents are stored in a memory 303 having a set operation function.
- FIG. 31 shows a logical address 110 set operation of character strings (information processing) and shows the winning address and the target document.
- the AND operation 110 set operation of (information processing) has already been performed in this specification with reference to FIG.
- “information”, “information”, “processing”, and “reason” correspond to the “first input” of the present invention
- “information”, “information”, “processing”, and “reaming” The mutual positional relationship corresponds to “second input”.
- “third input” indicates whether the operator is positive or negative.
- the character string (information processing) is obtained by a logical product 110 operation including the information location 114, and there are one match address 213 in the middle document and one in the right document.
- the priority address encoder 207 of the literature and the right literature is left unwinned.
- FIG. 32 performs a logical AND 110 set operation of character strings (searches), and one match address 213 exists in the right document, and the priority address encoder 207 of the right document is left unwinned. .
- FIG. 33 performs a logical AND 110 set operation of character strings (detection), and there is one match address 213 in the left document and one in the middle document.
- the priority address encoder 207 of the Chuo literature will continue to win.
- the priority address encoder 207 of the right document is left unwinned.
- FIG. 34 is an example in which a logical AND 110 set operation of character strings (recognition) is performed, and one match address 213 exists in the right document.
- the priority address encoder 207 of the right document is dropped from the winning list, and there is a match address 213 of the logical product 110 set operation of the character string (recognition).
- the middle document that does not do becomes the last remaining document.
- the set operation is performed collectively for the entire address space of the memory 303 having the set operation function, but it goes without saying that the set operation can be performed by designating a partial area.
- a memory 303 having a set operation function can be used for a two-dimensional array, a three-dimensional array, or all information with a fixed array. See the previous pattern matching examples for what is possible.
- the information data array 411 may be designated (corresponding to “fourth input” of the present invention).
- an exclusive pattern match 427 using the exclusive data 426 can be performed.
- FIG. 35 is an example of edge detection by a memory having a set operation function.
- This example is an effective utilization example of the exclusive pattern match 427 using the logical negation 111 of information.
- the actual image shown in the figure shows a set 102 of images 405 in which black, blue, green, white, and red pixels 406 are mixed in a complicated manner.
- the red set 102 is obtained by obtaining only the red pixel 102 from among them by a set operation 115 having a value 117.
- the red pixel has a spherical shape and has a certain area formed by the same pixel. Needless to say, pixels such as black, blue, green and white are intricately adjacent to this sphere. . In such a case, the edge detection using the exclusive pattern match 427 based on the exclusive data 426 is effective.
- the exclusive pattern match 427 is performed and the left edge of the sphere is detected as the match address 213. As a result of this exclusive pattern match, the red pixel (red pixel in the region) existing on the right side of the edge is ignored.
- the exclusive pattern match 427 is performed and the right edge of the sphere is detected as the match address 213.
- the exclusive pattern match results in the red pixel (red pixel in the region) existing on the left side of the edge being ignored.
- the exclusive pattern match 427 is performed and the right edge of the sphere is detected as the match address 213.
- the exclusive pattern match results in ignoring red pixels (in-region red pixels) existing below the edge.
- the exclusive pattern match 427 is performed as a condition, and the right edge of the sphere is detected as the match address 213. This exclusive pattern match results in the red pixels (red pixels in the region) existing above the edge being ignored. .
- exclusive pattern matching can be performed by setting the upper left, upper right, lower left, and lower right conditions on the basis of red, and by setting several pixels in order to ignore noise on the image. It is also free to use conventional filter effects such as
- the shape of an object can be recognized by an extremely simple set operation of edge detection.
- this edge detection is performed on the entire image space, and there is no problem in any shape even if the area of the object is wide.
- edge address it means that it is extremely easy to specify the object by obtaining the size, center of gravity, etc., and to follow the movement of the object by the edge.
- edge detection can be realized by a set operation of several clocks in any STEP, so that various conditions can be combined to perform effective edge detection.
- edge detection is an indispensable image processing for performing image recognition like pattern matching, and complex edge detection by color as well as gray scale is an image processing tool that greatly changes the conventional concept.
- the above example is only a part of the method of using the memory 303 having the set operation function.
- a memory 303 having a set operation function may be connected and used in parallel.
- the memory 303 having the set operation function is also characterized in that a performance proportional to the number of devices used can be expected.
- the memory cell 202 of the memory 303 having a set operation function can be realized by any type of memory such as DRAM, SRAM, ROM, FLASH type, and is not limited to a semiconductor memory.
- the logic circuit 110 can be fixed and used, or semi-fixed using a PLD (Programmable Logic Device) such as an FPGA.
- the set of information locations 114 is simple, it is possible to specify the locations 114 in consideration of address setting, bank switching, and virtual memory space that are normally performed.
- the memory 303 having the set operation function is incorporated in a semiconductor such as a CPU, CCD, or CMOS sensor
- the CPU 303 or other semiconductor is incorporated in the memory 303 having the set operation function. It is also possible to cause
- Each of the examples is a memory capable of storing information for each memory address and reading the information, and this memory is a first input for comparing information stored in each memory address given from the outside.
- 221 and the second input 222 for comparing each memory address and the set operation condition is (1) subset, (2) logical sum, (3) logical product, (4) logical negation or
- the means 210 and 211 for comparing and determining information stored in the memory.
- Example of image pattern matching Hereinafter, an example of image pattern matching will be described with reference to FIGS. It should be noted that in the following description, the reference numerals are left as they are for easy understanding of the correspondence with the basic application for which priority is claimed.
- FIG. 36 is an explanation of image patterns and image pattern matching.
- pattern 1 is a word that expresses the pattern of a fabric or printed matter, and at the same time, it is a term that is widely used to show a specific event or feature of an object.
- these patterns and patterns may be defined as those in which fine colors and brightness are combined and arranged at various positions.
- the temperature pattern 1 and the business pattern 1 are examples of the one-dimensional information pattern 1, and the character string, the DNA sequence, and the computer virus are also examples of this pattern 1. Regardless of whether it is a still image, moving image, or CG, a general image is displayed and reproduced based on the image information 5 on the memory, and the image information 5 and the image have a two-sided relationship. 5 is simply expressed as an image 5.
- the figure shows the concept of searching for a pattern designated by a magnifying glass such as a dragonfly.
- a specific pattern 1 is selected from the entire range of image information stored on the image 5. This is a situation detected with a magnifying glass such as a dragonfly.
- the pattern 1 by the image 5 is the color 2 information indicated by BL (black), R (red), G (green), O (orange), and B (blue) of the A pattern 1, and B
- the luminance 3 information indicated by 5, 3, 7, 8, 2 of the pattern 1 is combined on the coordinates.
- the image pattern match 17 is established when the color and brightness data of the pattern 1 and the position of the coordinate 4 are relatively matched.
- the above inquiry pattern 1 is based on the will of the person, when the color and brightness and their positions are appropriately combined as the pattern 1, and when a specific pixel and its position are extracted from some other image, the inquiry pattern 1 is obtained.
- the inquiry pattern 1 is obtained.
- the similar image pattern from the perfect match image pattern match 17 is obtained.
- the means can be expanded to match 17.
- the above is extremely simple for humans, but the contents that can be pattern-matched 17 for humans are also one of the troublesome information processing that is extremely burdensome in the information processing centered on the current CPU and memory. One.
- FIG. 37 is a diagram for explaining the principle of image pattern matching using the information refinement memory of the present invention.
- the image 5 is representative information of two-dimensional information treated as XY-axis biaxial information.
- the number of pixels (pixels) 6 constituting the image 5 is determined on both the XY axes, and the total number of pixels is the total number of pixels.
- the luminance 3 information that is the basis of the image 5 and the color 2 information such as the three primary colors of the color 2 are acquired in units of the pixels 6 and stored in the storage medium.
- the memory of the computer has a location for storing information and an address 7 for designating the location of the stored information.
- This address 7 is a one-dimensional, linear array and is normally designated by a hexadecimal value from address 0 to address N.
- the loop is repeated with the number of pixels 6 in one line (n, 2n, 3n. Is being written to.
- addresses such as addresses 0 to n are generally indicated as addresses, but in the figure, for simplification of description, an arrangement of 1 to n pixels is shown.
- addresses are assigned in order from the top of the figure, but there is no problem even when addresses are assigned in order from the bottom.
- the pixel 6 constituting the image 5 stores only one type of data in the memory in the case of luminance 3 information.
- the three primary colors R, G, and B of the normal colors are independently provided.
- the above pixel arrangement is not only image frame buffer information, but also bitmap image information, compressed image data such as JPEG and MPEG, and an artificially created image such as a map or animation image CG, that is, a two-dimensional array. This is common to all images and is a basic commitment for handling general images.
- the two image patterns 1 of A and B shown in FIG. 37 are the image pattern 1 composed of five pixels 6 and their positions, and the pattern matching conditions are five conditions.
- R red
- G green
- O range
- B blue
- BL black
- B blue
- “5”, “3”, “7”, and “8” are arranged at pixel positions shown in the figure with reference to “2” that is luminance 3 information.
- the reference pixel may be any pixel in the pattern, and the number of target pixels (pattern matching condition) may be large or small.
- the memory 51 (303) having the information narrowing down detection function of the present invention in order to eliminate the waste time due to the sequential processing of the CPU and the memory so far, by directly inputting the patterns A and B as described above, Only the information processing in the memory is used, the pattern match 17 is performed, and the address 7 subjected to the pattern match 17 is output.
- the operation principle will be introduced below based on the A and B patterns described above.
- a memory 51 (303) having an information narrowing detection function matches the specified data, and further matches the relative position of the arranged information.
- the memory 51 (303) can match both of them in the memory. It is.
- the position from the reference pixel 6 is converted into the pixel 6 position information in a linear array. It should be noted that the relative distance between the pixel 6 of the pattern 1 composed of the reference pixel 6 and the surrounding pixels 6 is always constant no matter where in the image space. It is the basis.
- a set of patterns 1 composed of a plurality of pixels 6 and their positions has a certain number of pixels, so that the probability that the pattern 1 exists elsewhere is extremely low. Therefore, it is not necessary to target all the pixels within the pattern range, and the specified pattern 1 can be narrowed down and detected by selecting an appropriate number of pixels 6 as a sample, and further, the pattern 1 for each part can be combined as a whole.
- an effective pattern match 17 is performed, such as detecting the first pattern 1.
- the target image has enlargement / reduction, rotation, etc.
- simple coordinate conversion is performed to match the pattern.
- the rate of image enlargement / reduction and the rotation angle are unknown, the number of pattern matching can be reduced to the limit by expanding the range of coordinates to be matched like the query B pattern. I can do it.
- this processing is performed for the pattern 1 specified at an extremely high speed without using any information processing means such as a CPU.
- the big point is that the detection can be realized only by hardware.
- FIG. 38 is a diagram illustrating an example of an image area / contour.
- the object 8 on the image 5 shown in the figure is information that is extracted on the basis of the color 2 information and the luminance 3 information in the area (area) 9 and the contour (edge) 10 on the image, and is the basic information for image processing.
- These areas (areas) 9 and contours (edges) 10 can be processed by analog information, and can be digitally converted at high speed to provide information.
- the CPU finds the characteristics of the image from this actual data, Therefore, it is necessary to search as soon as possible without knowing where and what kind of information, that is, outline and area information. Therefore, information processing is extremely burdensome, and various software is generally used to avoid this. Information processing or algorithms are considered.
- any software information processing or algorithm is not an essential solution, and a large amount of sequential information processing by the CPU cannot be avoided.
- the memory of the present invention solves such problems.
- Example 1-1 39 illustrates an example of object recognition using the features of the present invention.
- FIG. 39 is an explanatory diagram of exclusive pattern matching of an image, and the region (area) of the object 8 from the pixels 6 of the target image information 5.
- An example of efficiently detecting a contour (edge) is shown.
- exclusive pattern match 59 exclusive set operation is performed as the third input
- This example is an example in the case of an image having three spherical white (W) objects 8 such as balls in the image, for designating a certain region (area) 9 of the white ball 8 having a width of 6 pixels.
- the four white range (W) data 54 and the four non-white data (W (-)) data 54 circumscribing the white range (W) data 54, that is, the white exclusive data 58, are 4 at the boundary between the (W) and (-)).
- the contour (edge) 10 of the ball at the location can be detected. That is, only a white object 8 having a specific size, in this example, a white object (ball) having an area with a width of 6 pixels is detected.
- the exclusive pattern match 59 is performed between the pixels 6 that are completely adjacent to each other.
- the white object 8 having a slightly changed size can be easily obtained by setting a certain range between (W) and (W (-)). Can be detected.
- the exclusive data (W (-)) can be used regardless of the color of the background pixel 6 other than the ball area other than white, pattern matching is applied to about 8 pixels 6 as in this example. For example, a white ball with a width of 6 pixels can be found very simply.
- Pattern matching that is indispensable when recognizing and following a moving object becomes possible.
- the shape of the object may be updated every frame and matched with the object of the next frame.
- Such tracking of moving objects is an indispensable technique for video devices and security devices.
- FIG. 40 is an explanatory diagram of edge code encoding using a neighborhood 4-pixel pattern. Normal image information is acquired and stored for the purpose of expressing (displaying) luminance and color.
- Image processing is also performed based on the given luminance and color information, but it is possible to realize image processing that is extremely fast and effective by using new information.
- This code is a code considered for that purpose, and codes the difference from the 4 pixels in the vicinity of any one pixel in the image.
- the luminance or color data of all pixels is binarized, and in this example, the upper (U), lower (D), right (R), and left (L) pixels in the vicinity.
- edge code 12 obtained by coding the result into 16 types of codes from “0” to “F”. There are 16 types of pixels from one pixel that is completely different from the four neighboring pixels to the same pixel in the neighboring four pixels in the area (area). The meaning of this code is above (U), It shows which contour (edge) of lower (D), right (R), and left (L) exists.
- the four neighboring pixels do not necessarily have to be adjacent pixels, and noise in the image can be reduced by comparing with the appropriate neighboring pixels. In any case, it is important to have this code for the pixels in the entire image area.
- FIG. 41 is an explanatory diagram of edge code encoding using a neighboring 8-pixel pattern.
- neighboring pixels of the upper, lower, left, and right four pixels are edge-coded, but this figure further includes four pixels at the upper left, upper right, lower left, and lower right corners, and is edged by eight pixels.
- the code 12 is encoded. In this case, there are a total of 256 edge patterns, and finer edges can be detected.
- FIG. 42 is an explanatory diagram of an information array of image pattern matching using a memory having an information narrowing function.
- the address replacement function such as address shift can be used.
- This is an information detection device having an information processing function for performing parallel operation on both relative addresses and outputting the address 7 matching the condition as a match address 57.
- Memory having information narrowing-down detection function for the information of the pixel 6 described so far in this case, the total 6 types of pixel 6 information of the color 2 information of R, G, B 3 colors and the edge code 12 of R, G, B 51 (303) is an example of information arrangement.
- a method of storing these six types of information in the memory 51 (303) having an individual information narrowing function may be used, but in this example, the function of the memory 51 (303) having the information narrowing function is maximized. This is an example method.
- the entire memory 51 (303) having an information narrowing function is divided into six banks, and six types of information are stored in the respective address banks 52.
- Six types of information may be stored in the same array in the order of pixels from 1 address 7 to N address 7.
- the same address in any bank becomes the same pixel, and the information of color 2 and the information of region (area) 9 and contour (edge) 10 are selectively used to narrow down the information. That is, an efficient pattern match 17 can be performed.
- the bank designation 53 is a selection of which type of information is targeted, and the data designation 55 is to detect the coincidence address 7 of the stored data 54 values.
- the relative address designation 56 detects a relative address (relative position) between pixels, and the match address 57 is a narrowed address 7 (pixel 6) that matches the above conditions.
- a data value and a relative address can be designated one by one, or a data value and a relative address of a plurality of pixels can be designated collectively. It is also possible to specify a data value as a range and a relative address as a range. Of course, it is possible to match only the data values by simply using the content addressable memory (CAM) function.
- CAM content addressable memory
- FIG. 43 is an explanatory diagram of an application example of an edge code of an object.
- the method of effectively implementing the size of the object from the color 2 information and the edge code 12 is shown.
- the edge code 12 with the color 2 information of the two objects is stored and arranged as shown.
- the outline of the object size can be realized by the following extremely effective processing.
- the smallest (smallest) area minimum address of the address value and the opposite area maximum address indicate the height of the object.
- the area rightmost address and area leftmost address always exist in the range. If the height and width of the object are limited, it is easy to examine the details.
- all the contours of the object can be determined from “0” to “E” except for the code “F”. By matching the code 15 times, it is possible to identify all the contours around the region including the unevenness and the plane.
- FIG. 44 is an explanatory diagram of random and random pattern matching by local pattern matching. A method for logically and efficiently finding objects on an image using pattern matching technology will be described.
- This example is an explanation of the case where the space on the image is divided into several parts or parts, and the color of the part and the information pattern of the shape are collected.
- color information is a big part of our human recognition.
- Many objects are composed of a combination of several color regions on an image. Therefore, a local pattern or a partial pattern is constituted, and further, the partial pattern constitutes an entire pattern (entire image). Therefore, the object can be recognized if the pattern matching 17 is performed by appropriately combining local patterns or partial patterns.
- One is a randomly picked object with a cross pattern with red (R) color information, and the other is a pattern lacking a part of a circle with blue (B) color information. It is an object.
- R red
- B blue
- the pattern matching is intentionally performed (intentionally) and the details may be obtained.
- One of them is effective in detecting camera shake of a digital camera. Whether the entire screen has moved due to partial or local pattern matching of the image at the moment the shutter is pressed and the image just before the shutter is released (shake) or whether part of the image has moved (target object) Movement) and a combination of both can be detected very easily.
- the detailed detailed pattern match 17 for each local area is the basis for normal object detection.
- the objects that can be recognized in an instant are at most 3 to 4 types of objects.
- the level of recognition is one example, but the level of recognition of the vehicle in the photo is different from the color, shape, manufacturer, model name, and even the license plate, but it is normal driving Then, it is not necessary to recognize such details.
- the degree of recognition of human-made image recognition by a computer is how much the local or partial pattern matching is repeated with intention (artificial).
- Random pattern matches can be compared to human landscapes that recognize the license plate of the car in front of the driver's seat. .
- computer-based object recognition by the human eye and brain consists of random (unconscious) pattern matching and artificial (intentional) pattern matching when combining pixel information and its position as an inquiry pattern. In combination, it is only necessary to recognize the necessary object at that time with the necessary accuracy.
- the target image is not necessarily a large-screen, high-definition image, but it is only necessary to focus on what is desired to be recognized in the same way as a human eyeball and to enlarge and pattern match as necessary.
- this device is used in parallel as necessary and appropriate pattern matching and knowledge processing are executed, the number of objects that can be recognized in one second can be further improved to the level of humans.
- the above can be realized by artificial pattern matching characterized by color.
- FIG. 45 is an explanatory diagram of object change image detection.
- the movement of the image can be detected effectively by removing the non-moving pixels by taking the difference between the two images at time T0 and time T1, and moving the moved image portion effectively.
- various applications can be made by catching the movement of an object on the moving image and the movement of the camera angle. It is extremely simple, such as automatic camera angle tracking such as capturing a specific object as a pattern and making this pattern the center of the screen at all times.
- the difference image detection of this method has various utilization methods such as position shift detection by comparison with a standard image and product defect detection.
- FIG. 46 is an explanatory diagram of detection of corresponding points of an object by local pattern matching.
- images of objects 8 such as balloons of four colors of red (R), B (blue), yellow (Y), and green (G) floating in the space are captured as images from the left and right cameras. It is. Since the object image captured by the binocular camera and the actual object are composed of epipolar planes, the position of the XYZ axes including the depth of the object can be measured by triangulation if the distance of the binocular camera is known.
- the Y-axis (height) of the object is omitted, and the X-axis and the Z-axis are expressed by two axes.
- the left image 14 and the right image 15 have the Z-axis omitted and the two axes X and Y are It is expressed.
- a sample pattern is taken from the reference with either the left or right as a reference, and the corresponding point 21 is the place where the other image is inquired and matched.
- the pattern matching 17 in a single color is described. However, it is usually sufficient to perform local matching with a combination of colors other than a single color.
- Many parts of the image of an object should exist as images (patterns) that are similar or similar to the left and right images (patterns), and because of the same lighting problems and shooting environment that are issues in image processing
- Corresponding points 21 are easily associated. If the corresponding points 21 of the left and right images can be detected, three axes including the direction of the depth 18 of the object from the pixel position of the corresponding points 21 can be measured.
- FIG. 47 is an explanatory diagram of object recognition using edge codes.
- the edge code 12 is stored in the left image 14 and the right image 15, respectively, which is an actual example of the pattern match 17 principle shown in FIG. 46, the pattern match 17 is multiplied by the left and right edge codes 12.
- the corresponding point 21 pattern becomes a pattern unique to the object, and there is very little probability of existing in the screen space.
- either the left or right edge pattern is collected and used as a query pattern to find the left and right corresponding points 21 by pattern matching.
- FIG. 48 is an example of person recognition using stereoscopic measurement.
- the level of the face recognition technique can be greatly improved even with the local pattern match 17 of the monocular camera image, an example of person recognition using stereoscopic measurement will be described below.
- the person recognition described here means that the person name is identified from the image range to identify the individual name.
- Pattern matching is greatly advanced by pattern matching using the present invention.
- features such as facial eyes, nose, moles, and scratches displayed on the image are regarded as patterns and corresponding points by binocular parallax are subjected to pattern matching 17, and X,
- Y and Z that is, the size of the eye, the height of the nose, and the like are measured.
- the conventional face recognition will greatly advance to the person recognition. Combining conventional face recognition technology, an extremely fast and highly accurate person recognition technology is completed.
- FIG. 49 is an explanatory diagram of object recognition in space.
- the fact that the dimensions and distances of the objects as described above can be easily obtained means that the actual dimensions of the objects can be understood. From the three object sizes shown in the figure, the size of the object such as a truck, It is possible to limit the range of objects such as apples and apples, and then recognize the objects by classifying them based on color information and detailed information Is possible.
- the recognition rate of the object can be greatly improved by knowing the actual size of the object.
- the map information supports this recognition technology very effectively.
- Object recognition for ensuring safety in the vicinity of city intersections and on highways is of course very different.
- FIG. 50 is a conceptual explanatory diagram of object recognition by pattern matching, and summarizes the description so far.
- Object recognition is a combination of both image processing and knowledge processing.
- knowledge processing various features of the object are classified and registered in various categories, and the object registered in the database is searched by the knowledge processing based on the features given from the image processing.
- a feature is designated by knowledge processing, and it is similarly performed whether or not there is a feature that matches the feature by image processing.
- the image processing is divided into processing for searching for a feature that is the gist of the present invention and other processing.
- Typical processing of other processing is arithmetic processing and display processing, and these processing are performed by information processing centering on the CPU as before, and this part has no wasted search time and the CPU function is not 100% wasted. Available. Since the arithmetic processing is greatly reduced by the pattern matching of the present invention, the burden on this portion by the CPU is extremely reduced.
- the process of finding a feature is performed by pattern matching based on basic information such as color, brightness, edge, area, and depth, and the obtained object features include shape, dimensions, movement, and corresponding points. It is possible to effectively identify an object from an object feature database using important features such as depth, space, and so on. Needless to say, the time required for the process of searching for features can be drastically eliminated by using the memory 51 (303) having an information narrowing function. It can also be realized by processing.
- recognition of an object centering on high-speed and high-precision image processing is centered, but the process of searching for an object most similar to a given feature from a database in knowledge processing is a pattern.
- This technique is a match itself, and this technique can be used for both image processing pattern matching and knowledge processing pattern matching.
- the memory described in this example is In an image in which the size of the XY array of the image is defined, (1) An image inquiry pattern configured by appropriately combining both the image information data value of the pixel and the data position of the pixel that constitute the image
- the creation step has the contents shown in the method of creating a random or artificial inquiry pattern as an example of finding a specific pattern from images arranged on the memory 51 (303) having an information narrowing function. is there. (The process of searching for pattern-matched data) Further, (2) a step of detecting, from the target image, a pixel that matches the image query pattern 17 by querying the image query pattern for the image query pattern from the target image. And inquiring about the sampled pattern, and detecting a pattern-matched address (pixel).
- the image recognition method (combination of various pattern matches) characterized in that image processing is performed by the steps (1) and (2) above is a feature of image recognition of the present invention.
- Example of voice pattern matching Hereinafter, an example of voice pattern matching will be described with reference to FIGS. It should be noted that in the following description, the reference numerals are left as they are for easy understanding of the correspondence with the basic application for which priority is claimed.
- the address 51 replacement function such as the shift of the address 51
- This is an information detection device having an information processing function for outputting, as a match address 56 for pattern matching 9, an address 51 obtained by performing parallel calculation on both data 52 and a relative address 54 specified from the outside and narrowing down under the condition.
- the voice recognition technology is a cluster of pattern match 9 technology, and this device is optimal for voice recognition.
- Recent research in linguistics has reported that the largest phoneme language is 200 in African languages, 46 in English, 20 in Japanese, and 13 in Hawaiian. Although the number varies depending on the researcher, speech recognition can be dramatically improved if it can recognize phonemes of languages around the world as described above and recognize up to 256 phonemes with high accuracy.
- FIG. 51 is a reference example of the amplitude waveform of a phoneme.
- This figure is an example of the instantaneous amplitude 3 waveform of the phoneme 5 with our words.
- the phoneme 5 is a signal including various frequencies 2 as a speech 1 signal.
- 50 Iueo sounds and onomatopoeia can be generated by combining about 20 phonemes such as vowels, consonants and semi-vowels.
- FIG. 52 is a reference example A of the frequency spectrum waveform of a phoneme.
- the intensity (power) 4 for each frequency 2 of a certain phoneme is measured by the spectrum 16, and the intensity (power) 4 for each frequency 2 is arrayed 8 as array number 15.
- the intensity (power) 4 is shown for each array number 15.
- the voice 1 and the phoneme 5 in the figure are phoneme patterns 17 having a large intensity (power) 4 in the low sound range and the high sound range.
- FIG. 53 is a reference example B of the frequency spectrum waveform of a phoneme.
- the phoneme pattern 17 in the figure is a phoneme pattern 17 having a large intensity (power) 4 in the high sound range.
- the spectrum of the phoneme 5 has a spectrum 16 waveform means the phoneme pattern 17, and if this pattern can be correctly matched and read, the phoneme 5 can be surely recognized. .
- the cepstrum sequence obtained by logarithmically transforming the speech spectrum and performing the inverse Fourier transform is often used, focusing on the shape of the vocal tract of the speech utterance rather than using the phoneme spectrum pattern itself. The same applies to cases where the data for each frequency for each phoneme is interpreted as a pattern.
- FIG. 54 is an example of range data for phoneme discrimination.
- the range 18 is provided in the data to be inquired as described above.
- the level of intensity (power) 4 is 16 levels, and the range of the maximum value 10 and the minimum value 11 is given to the given data 52.
- An example of an ambiguous pattern match 13 in which a pattern match 9 is applied with 18 is shown.
- a range 52 of uniform ⁇ 2 is given to the given data, and data 52 is shown when five data ranges are given. If the given data is the lower limit value, the upper limit value and its surroundings, the range decreases. Based on the above idea, an ambiguous pattern match 13 with intensity (power) 4 may be performed.
- FIG. 55 is an example of phoneme recognition by a memory having an information narrowing function.
- the array 8 described in FIG. 54 is stored in the memory 50 (303) having an information narrowing function.
- one phoneme 5 pattern is arranged as an array 8 of 50, the array is allocated to the absolute address 51 of the memory 50 (303) having the information narrowing function, and the strength (power) 4 is assigned to the data 52. Data is stored and registered.
- the address space is about 12K addresses.
- the phoneme spectrum 16 that is generated and spectrum-converted is input as a query phoneme 14 to a memory 50 (303) having an information narrowing function.
- the phoneme 5 data is obtained by arranging data 52 of intensity (power) 4 for each array number 15, and this array number is a relative address specification 55 for specifying a relative address 54 of the absolute address 51.
- Narrowing is performed inside the memory 50 (303) having the information narrowing function by using the designation data of both the relative address designation 55 and the data designation 53 for designating the data 52, and the address where the narrowed result matches. 56 is output.
- This address designates phoneme 5 and is recognized by pattern match 9 of phoneme 5 itself.
- the above five repeated matchings are parallel processing and can be processed at an extremely high speed, and if an array is repeated five times per sequence up to 50, pattern matching 9 is performed sequentially, and an ambiguous pattern match 13 can be realized.
- the pattern match 9 for the entire 50 sequences is introduced.
- the pattern match 9 or 13 is not performed for all the sequences 8 as above in the statistical probability, and the required number, for example, about half is pattern match. 9 and 13 are sufficient.
- the sound generation in the vehicle includes noise of a natural frequency such as engine rotation sound and air-conditioner sound peculiar to the vehicle.
- the data 52 of the array 8 of the natural frequency of the external noise at that time is used.
- phoneme recognition with high reliability becomes possible.
- This dynamic pattern matching can be achieved with high-speed hardware pattern matching.
- the overall recognition rate is improved by combining with the vocabulary pattern matching described below.
- FIG. 56 shows an example of vocabulary pattern matching.
- the pattern matching method can also be applied to the matching of the vocabulary 6 determined by the phoneme arrangement, where the combination of the detected phonemes is a word or vocabulary.
- the utterance of “speech” is a phoneme arrangement pattern such as “o-n-s-e-i”, and the phoneme arrangement forms a vocabulary (word) of a minimum unit as a word.
- one vocabulary is arranged as an array 8 of 16 phonemes 5 and allocated to a memory 50 (303) having an information narrowing function, and phonemes 5 are stored and registered as data 52 at absolute addresses 51. It is a thing.
- phoneme 5 is conditionally input as data 52 to array number 15.
- the vocabulary 6 is detected simply by reading the absolute address 51 that matches the query vocabulary 20 with the pattern match 9.
- the 16 array conditions are pattern matched at once. It is extremely simple and fast vocabulary detection, and no complicated algorithm or other data table is required.
- This method has a major feature that it is possible to store the vocabulary from any vocabulary, the order of which does not matter, and it is not necessary to re-arrange the vocabulary that is normally performed every time a vocabulary is added or modified.
- a database for each language is prepared in a separate storage medium and downloaded to the memory 50 (303) having the information narrowing function each time.
- the ambiguous pattern match 13 for vocabulary recognition will be described.
- the phoneme sequence “on-s-e-i” shown earlier appears in the order of “o”-“n”-“s”-“e”-“i” on the time series.
- the feature of the pattern matching by the memory 50 (303) having the information narrowing function is irrelevant even if a part is missing.
- the phoneme sequence described above has a relative address X + 0 “o” ⁇ relative address X + 1 “n” ⁇ relative address X + 2 “s” ⁇ relative address X + 3 “e” ⁇ relative address X + 4 “i” in the information array. ].
- X can be recognized as a relative value from 1 to 16 in this example as a result of the narrowing-down matching.
- pattern matching is guaranteed perfectly, such as when wildcards are used to specify any part of the array, when narrowing the phoneme array from the reverse (reverse lookup), or narrowing down from the middle (decimation) .
- pattern matching is performed by excluding unrecognized phonemes, or even when the phoneme is totally uncertain due to external noise, etc. It is effective.
- the memory 50 (303) having an information narrowing function varies greatly in parallel operation matching processing time depending on the address size and various optional functions.
- 16 The time required for pattern matching of one set of sequences can be reduced to 1 microsecond or less, and the speed is directly related to the accuracy of recognition.
- the ambiguous information pattern matching described above is more effective. Can be realized.
- the phoneme and vocabulary pattern matching described so far is faster, more accurate, and easier than any other pattern matching.
- FIG. 57 is an explanation of image patterns and image pattern matching.
- pattern 1 is a word that expresses the pattern of a fabric or printed matter, and at the same time, it is a term that is widely used to show a specific event or feature of an object.
- these patterns and patterns may be defined as those in which fine colors and brightness are combined and arranged at various positions.
- the temperature pattern 1 and the business pattern 1 are examples of the one-dimensional information pattern 1, and the character string, the DNA sequence, and the computer virus are also examples of this pattern 1.
- a general image is displayed and reproduced based on the image information 5 on the memory, and the image information 5 and the image have a two-sided relationship. 5 is simply expressed as an image 5.
- the figure shows the concept of searching for a pattern designated by a magnifying glass such as a dragonfly. Although not shown in the figure, a specific pattern 1 is selected from the entire range of image information stored on the image 5. This is a situation detected with a magnifying glass such as a dragonfly.
- the pattern 1 by the image 5 is the color 2 information indicated by BL (black), R (red), G (green), O (orange), and B (blue) of the A pattern 1, and B
- the luminance 3 information indicated by 5, 3, 7, 8, 2 of the pattern 1 is combined on the coordinates.
- the image pattern match 17 is established when the color and brightness data of the pattern 1 and the position of the coordinate 4 are relatively matched.
- the above inquiry pattern 1 is based on the will of the person, when the color and brightness and their positions are appropriately combined as the pattern 1, and when a specific pixel and its position are extracted from some other image, the inquiry pattern 1 is obtained.
- the inquiry pattern 1 is obtained.
- the similar image pattern from the perfect match image pattern match 17 is obtained.
- the means can be expanded to match 17.
- FIG. 58 illustrates the principle of image pattern matching using an information narrowing memory.
- the image 5 is representative information of two-dimensional information that is handled as XY-axis biaxial information.
- the number of pixels (pixels) 6 constituting the image 5 is determined on both the XY axes, and the total number of pixels is the total number of pixels.
- the luminance 3 information that is the basis of the image 5 and the color 2 information such as the three primary colors of the color 2 are acquired in units of the pixels 6 and stored in the storage medium.
- the computer memory has a location for storing information and an absolute address 7 for designating the location of the stored information.
- This absolute address 7 is a one-dimensional, linear array and is usually designated by a hexadecimal value from address 0 to address N.
- addresses such as addresses 0 to n are generally expressed as addresses, but in the figure, for the sake of simplification of explanation, an array of pixels 1 to n is shown.
- addresses are assigned in order from the top of the figure.
- the pixel 6 constituting the image 5 stores only one type of data in the memory in the case of luminance 3 information.
- the three primary colors R, G, and B of the normal colors are independently provided. Normally, it is necessary to store three pieces of pixel information per one pixel 6. Therefore, when storing the color 2 information with 3 addresses per pixel 6, the actual memory requires three times as many addresses as the number of pixels 6. Needless to say, if the number (n) of pixels 6 in one line is known, which color 2 information of which pixel 6 on the image 5 is stored in which position on the memory and vice versa can be easily performed. It is possible to convert.
- the above pixel arrangement is not only image frame buffer information, but also bitmap image information, compressed image data such as JPEG and MPEG, and an artificially created image such as a map or animation image CG, that is, a two-dimensional array. This is common to all images and is a basic commitment for handling general images.
- the two image patterns 1 of A and B shown in FIG. 58 are the image pattern 1 including the five pixels 6 and their positions, and the pattern matching conditions are five conditions.
- R red
- G green
- O range
- B blue
- BL black
- ⁇ 0 color 2 information
- B blue
- “5”, “3”, “7”, and “8” are arranged at the pixel positions shown in the drawing with “2” being luminance 3 information as a reference ( ⁇ 0).
- the reference pixel ( ⁇ 0) may be any pixel in the pattern, and the number of target pixels (pattern matching condition) may be large or small.
- the CPU sequentially processes and finds addresses on the memory stored in the array based on such an inquiry pattern, that is, pattern matching by software.
- pattern matching since the majority of information processing called pattern matching is based on the processing of the CPU, the actual situation is that it is far from the essential pattern matching.
- the absolute address 7 is configured to output the memory 51 (303) having an information narrowing detection function according to the inventor's invention, the details of which are introduced in detail in the patent specification.
- the principle is introduced based on the A and B patterns described above.
- the memory 51 (303) having an information narrowing detection function is a memory that matches specified data, further matches the relative position of the arranged information, and can perform matching between the two inside the memory.
- the position from the reference pixel 6 is converted into the pixel 6 position information in a linear array.
- the specified pattern 1 can be narrowed down and detected by selecting an appropriate number of pixels 6 as a sample, and further, the pattern 1 for each part can be combined as a whole.
- a significant feature is that an effective pattern match 17 is performed, such as detecting the first pattern 1. If the target image has enlargement / reduction, further rotation, etc., simple coordinate conversion is performed and pattern matching 17 is performed. If the rate of image enlargement / reduction and the rotation angle are unknown, the number of pattern matching can be reduced to the limit by expanding the range of coordinates to be matched like the query B pattern. I can do it.
- Comparison of the speed of pattern match 17 and hardware pattern match by a normal CPU and memory is as shown in the background art, which corresponds to the fact that pattern matching of 7 conditions (7 pixels in the case of an image) is realized at 34 nS. .
- This hardware pattern match is based on the fact that the circuit scale, which tends to be in parallel processing, can be realized with the smallest possible configuration, and as a result, the large-scale image processing is possible. It is possible to realize a device with an information processing capacity.
- the prototype machine introduced in the background art was a perfect match type pattern match that required high speed. However, as the function was added, the processing time was somewhat reduced and the information processing capacity was reduced, but as shown in the query B pattern. In addition, it is possible to specify the range of the pixel 6 to be subjected to the pattern match 17, and to detect not only a fixed value but also a range by specifying the range of the detected data value and similar images.
- the image character recognition technology is greatly advanced. Details will be described later.
- the image may be divided into several parts and pattern matching may be performed for each division unit. In this case, in order not to be affected by the pattern match 17 of the image of the divided portion, the image is divided so as to overlap the X axis and the Y axis by the size of the image to be pattern matched 17, and the target image is included in any of the divided images. Pattern matching should be done so that pattern matching can be applied.
- the local (relative) address 103 and the global (absolute) address 104 will be described.
- the current digital full high-definition image is an image composed of pixels of horizontal (X axis) 1920 pixels ⁇ vertical (Y axis) 1080, totaling about 2,073,600 pixels, and from 0 pixels to 2,073,599.
- Image information for each pixel is stored in the absolute address 7 linearly arranged up to the pixel.
- the relative position of any two pixels in this image space can be expressed by the distance of the global address 104 of one-dimensional data.
- This is a feature of the memory 51 (303) having an information narrowing function. This is because pattern matching can be performed in parallel (simultaneously) for all pixels on the image (hardware pattern matching). Although it takes time, the method using the two-dimensional array can be realized by normal processing using a CPU and a memory.
- FIG. 59 is an explanatory diagram of exclusive pattern matching, and shows an example in which the region (area) 9 and the contour (edge) 10 of the object 8 are efficiently detected from the pixels 6 of the target image information 5. .
- searching for an object 8 having a specific color 2 or brightness 3 area (area) 9 the background pattern of the object exists infinitely, so pattern matching 17 based on data 54 of various colors 2 and brightness 3 is required. Must be repeated a number of times.
- the exclusive pattern match 59 is effective in such a case.
- This example is an example in the case of an image having three spherical white (W) objects 8 such as balls in the image, for designating a certain region (area) 9 of the white ball 8 having a width of 6 pixels.
- the four white range (W) data 54 and the four non-white data (W (-)) data 54 circumscribing the white range (W) data 54, that is, the white exclusive data 58, are 4 at the boundary between the (W) and (W (-)).
- the contour (edge) 10 of the ball at the location can be detected.
- a white object 8 having a specific size in this example, a white object (ball) having an area with a width of 6 pixels is detected. Even if the horizontal width of white (W) is 5 pixels or 7 pixels, it is possible to detect a very accurate object size.
- the exclusive pattern match 59 is performed between the pixels 6 that are completely adjacent to each other.
- the white object 8 having a slightly changed size can be easily obtained by setting a certain range between (W) and (W (-)). Can be detected.
- the exclusive data (W (-)) can be used regardless of the color of the background pixel 6 other than the ball area other than white, pattern matching is applied to about 8 pixels 6 as in this example. For example, a white ball with a width of 6 pixels can be found very simply.
- such exclusive data 58 of (W (-)) is once negated (inverted) from the (W) output of the associative memory (CAM) function, By rewriting this invert result (W (-)) as a CAM output (inverting the CAM output), it can be used on a very simple principle, so the background image of the object to be searched for may be unspecified and infinite. Very efficient in some cases.
- the exclusive pattern match 59 of one white color it is the case of the exclusive pattern match 59 of one white color, but it becomes possible to detect a complex image with a very small number of pattern matches by combining other colors.
- the number of pattern matching points and their positions may be appropriately selected.
- Pattern matching that is indispensable when recognizing and following a moving object becomes possible.
- the shape of the object may be updated every frame and matched with the object of the next frame.
- Such tracking of moving objects is an indispensable technique for video devices and security devices.
- This technology can be widely used for character recognition, fingerprint, and pattern matching such as one-dimensional information.
- the pattern matching of this method is powerful, and image processing that tends to be large can be performed very simply.
- General characters consist of a certain color and their shape (area), and parts other than the characters are in a specific color, or even if they are a specific pattern or video, the outside of the area is specified with a color other than the characters Therefore, if this exclusive pattern match is used, a very simplified character recognition pattern match becomes possible.
- FIG. 60 shows an example of a typeface for characters.
- the Japanese language shown in this example is a language that combines various types of characters. Of these, the most frequently used kanji are about 2000 common kanji characters and about 3000 characters including complex kanji characters. It is necessary to recognize up to 5000 kinds of character symbols including hiragana, katakana, Arabic numerals, alphabets, symbols other than these. It is said that the kanji currently used in daily life in China with the maximum number of characters is 6000 to 7000 characters. Therefore, in the case of Chinese, it is necessary to recognize a maximum of 10,000 characters.
- 61 is an explanatory diagram of a character pattern sampling point creation example A.
- FIG. In order to recognize a specific Japanese character “a”, two sampling points 61, No1, No2, No3, and No4, an in-region sampling point 61 and an out-of-region sampling point 62, respectively. In this example, four sampling points 60 are assigned on the coordinate 4 of the local address 103.
- the pattern match 17 is performed in the order of No1, No2, No3, No4.
- the order may be from anywhere, but the coordinates of the local address designated as No1 are output as the absolute address 7 of the matched global address.
- these four sampling points 60 are given the local addresses 103 of the X and Y axes as coordinates 4.
- the meaning of these two types of sampling points 60 is to specify whether the sampling points and their vicinity are character region portions or other portions. In the case of a general character, the area area in the coordinate 4 space is smaller than the area outside the area, and the existence probability of all the character areas is 1 ⁇ 2 or less. / 2 or more.
- the probability of the region is high in the center of the coordinate, and the probability of the region is small near the corner of the coordinate. Therefore, it is possible to reduce the probability and improve the recognition rate by using such characteristics positively by using the sampling point in the area near the corner of the coordinate and the sampling point outside the area in the center of the coordinate. It becomes. Needless to say, the greater the number of sampling points, the higher the discrimination ability. However, the more sampling points, the longer the pattern match time, so it is necessary to determine an appropriate number of sampling points.
- the discrimination probability is about 1/1 million
- the discrimination ability is about 1 billion. Even if some of the sampling points cannot be read normally due to character blur or foreign matter due to the quality of the printed characters or paper quality, a configuration in which most of the sampling points are matched may be accepted. Details will be described later.
- FIG. 62 is an explanatory diagram of character pattern sampling point creation example B, in which some typefaces (fonts) of the specific character “A” are superimposed, and in-region sampling points 61 and any typeface (font) This is a case where a total of 30 out-of-region sampling points 62 are given to the out-of-region portion.
- the special typeface (font) 102 It is possible to match patterns other than characters in common. If a plurality of characters are recognized and selected, the sampling points of the characters may be partially corrected.
- FIG. 63 is an example of creating a character pattern sampling point for a specific typeface. Based on the description so far, 30 sampling points are assigned to each character. Such sample points for pattern matching may be created for 5,000 characters in Japanese and 10,000 characters in Chinese. Even if you collect all the characters from all over the world, about 20,000 characters are enough.
- sampling points When creating sampling points, create them with a large font (font) 102, and if they have a small character size, they can automatically reduce the coordinate four values to match the pattern. Therefore, once these sampling points are created, it becomes a property common to humankind that can be used forever.
- FIG. 64 is an example of character recognition of an image with subtitles.
- Subtitles are an integral part of foreign movies.
- the maximum number of subtitles that appear in one scene is about two lines and the number of characters is about 40, and the display time is about 1 to 5 seconds.
- the memory 51 (303) having the information narrowing function is a complete hardware pattern match, one pattern match is possible in 1 microsecond.
- 30 sample points per pattern match per microsecond one character is 30 microseconds, 0.15 seconds for 5,000 characters in Japanese, and 0.3 seconds for 10,000 characters in Chinese. All characters per screen can be pattern matched. Even with 20,000 characters worldwide, all characters per screen can be pattern matched in 0.6 seconds.
- pattern matching can be applied by changing the size of frequently appearing characters. In the case of Japanese, it is sufficient to perform pattern matching with hiragana and 50 characters that appear frequently. If it is determined that the target character is a document or other subtitles of a movie, for example, that character
- the character size is the place where the required number of pattern matching absolute addresses 7 is obtained by applying pattern matching around the size of general characters. It is also possible to prepare special typefaces (fonts) and recognize characters in special typefaces. As for the color of characters, pattern matching is usually performed with black or white, followed by red, blue, green, and colors in the vicinity thereof.
- FIG. 65 shows an example of an information processing apparatus having a real-time OCR function.
- the OCR pattern database 105 for the pattern match 17 of the sampling points 60 from No1 to No30 is registered in each character 101 of Japanese 5,000 types of characters.
- Japanese is taken as an example, but it is also possible to register languages all over the world collectively in English or Chinese.
- the sampling point 60 includes “XY” local address 103 for each character 101, “D”, that is, data 54 specifying color 2 or luminance 3 of the region of character 101, and color 2 or luminance 3 outside that region. Is registered.
- the data 54 and the exclusive data 58 can be specified and registered separately at a time. The minimum requirement is that each sampling point 60 is an in-region sampling point 61 or an out-of-region sampling point 62 of the character 101. It is necessary to clarify that.
- a memory 51 (303) having an information narrowing function is incorporated in this apparatus, and image information 5 to be subjected to character recognition is stored in the memory 51 (303).
- Characters 101 are sequentially specified from the database 105 and pattern matching is performed 5,000 times. At this time, all that is necessary for the pattern match 17 is to grasp the color and size of the character, convert the local address 103 to the global address 104 and apply the pattern match.
- the above absolute address 7 may be read by the CPU and necessary processing may be performed by the CPU. As described above, the CPU does not need to perform any processing relating to the character recognition itself. What is necessary is to supervise the entire character recognition processing and to give a pattern matching instruction to the memory 51 (303) having an information narrowing function. It simply reads the match result (absolute address 7) and performs the necessary processing as a result. In the case of Japanese only, all the pattern matches can be realized in 0.15 seconds with 5,000 characters. In the case of a normal movie, the subtitle character color is white, and the typeface (font) 102 is also fixed and does not change.
- the points to be considered are character noise due to block noise peculiar to digital images. These color or luminance noises are appropriately filtered, and the range of the data 54 may be designated to enable pattern matching.
- these text data can be used as annotation data of the video scene.
- HDD hard disk drive
- T tera
- the recording time exceeds several hundred hours.
- T tera
- the recording time exceeds several hundred hours.
- T tera
- subtitle video scenes often appear at the beginning of programs or important video scenes.
- FIG. 66 is an example of character recognition of a document image.
- pattern matching based on hardware parallel processing which is a feature of this method, is used, the time required for pattern matching is constant regardless of the number of characters included in the image. Therefore, even in the case of the subtitles of the previous movie, even if the document image has several hundred characters, character recognition per screen can be performed in the same time.
- sampling points When a printed document is scanned with a scanner, some sampling points may not be pattern-matched due to character blurring or foreign matter. If pattern matching is not possible even at one of the sampling points, the character may not be recognized.
- the memory 51 (303) having the information narrowing function having the counter function in the memory 51 (303) having the information narrowing function For example, it is possible to use a method of accepting 25 points or more out of 30 sample points using the counter function, and recognizing and outputting the absolute address 7.
- the method using the two-dimensional array can be realized by normal processing using a CPU and a memory.
- the character recognition by pattern matching according to the present invention can drastically eliminate the dead time due to the search by using the memory 51 (303) having an information narrowing function. It can also be realized by sequential processing.
- the inventor of the present application has so far made use of the high-speed pattern matching of the memory 51 (303) having an information narrowing function, and in the prior invention, human recognition is performed through image recognition and speech recognition, and character recognition according to the present invention.
- the necessary information is stored in the memory 51 (303) having one information narrowing function, and necessary character recognition, image recognition, and voice recognition are performed, and new information is recognized at the next moment.
- the main feature is that it can be used in These things are similar to our brain's information processing, and it is difficult for human beings to concentrate their nerves at the same time, usually processing is concentrated on either image, voice, or text. .
- This means that the memory 51 (303) having an information narrowing function can be expressed as a general-purpose brain chip.
- a memory 51 (303) having an information narrowing function transforms a computer smartly and powerfully through collaboration with a CPU.
- Standardization of pattern matching Hereinafter, an example of standardization of pattern matching will be described with reference to FIGS. It should be noted that in the following description, the reference numerals are left as they are for easy understanding of the correspondence with the basic application for which priority is claimed.
- the present invention is based on the fact that, when pattern matching is performed on information in the target sequence information, the candidate data that will be included in the pattern to be searched is first designated and used as reference information.
- the relative relationship between the candidate data and other information for matching can be designated by coordinates or by distance, and is simply a position.
- the pattern matching method is based on the concept of this data and its position, it can be used universally regardless of the number of pattern matching samples from one-dimensional information to multidimensional information. It is the most basic place.
- CAM associative memory
- the ambiguous pattern information is an information array of a collection of information in which the information (data value) has a width (range) and the information to be stored has a width (range) and the information (data value) is stored. Can be defined.
- the current general memory cannot obscure the data value itself or make the storage address ambiguous as described above. It is difficult to create ambiguous information itself because it is necessary to store data values at a plurality of addresses and separately specify a range of data, which requires complicated information processing.
- the information to be stored (data value) and the address to be stored are fixed, and the range (maximum, minimum, above, below) for both the data value of the query pattern 9 and its position when detecting this Vague pattern matching is possible.
- an ambiguous pattern can be handled and an ambiguous pattern can be matched using a general semiconductor memory.
- information (data) stored in the memory and its location (memory address) remain normal, and the arrangement can be realized by a general information arrangement.
- FIG. 67 shows an example of pattern matching of one-dimensional information.
- time series data such as stock price and temperature, character data, DNA data, etc. are typical one-dimensional array data
- the top address for writing information (data value) is determined in the linear array memory, and the address is determined for each address.
- Data values may be stored and stored sequentially. Of course, in this case, it is possible to store information sequentially every two addresses with a space between one address, or sequentially store information every three addresses. It does not matter as long as the definition (array) of information storage is acceptable.
- the information in the database 8 is described as an absolute address 7 and a global address 113, and the position 103 of the pattern match information data 101 is described as a relative address 57 and a local coordinate 112.
- the pattern match 17 of this method selects information 110 as a reference (candidate) as a candidate for the pattern match 17 in advance, and sequentially gives match information 111 as a pattern match target, and does not match the match information 111.
- 110 candidates are screened one after another, and the address that has been won when a certain number of matches is completed is used as the match address 57. Therefore, what is particularly important when creating the inquiry pattern 9 is the sampling point 60 as the first reference. In this case, three data are used as sample points, and the reference information 101 and No1 are samples on the left side, even at the center. Even if the right sampling point is selected, there is no problem at all. However, as described above, it is meaningless unless the data can be expected to exist. Therefore, in this example, the most likely data (the median value of the data) Is the first choice. Further, the possibility of mismatch can be reduced by appropriately selecting the data range of No1. If there is no such data, the pattern match can be cut off.
- the reference information 110 for performing pattern match information processing in this example is No1, and data that matches both No2 and No3 is searched based on this No1.
- searching for the information of No3 it is not based on the relative position with No2, but searching for information that matches No1 and No3 is the origin of the present invention.
- this principle is the most important even if the number of sampling points increases. Therefore, it is wise not to set the position range for the reference sample No. 1 (the data value has no problem even if the range is set).
- pattern match 17 the case where pattern 1 that matches inquiry pattern 1 exists in database 8 (pattern match 17) is illustrated.
- the above example is a pattern match 17 of three sets of information data, but there can be any number of combinations of information data. Further, the data 101 value and the range 102 of these three pieces of information can be arbitrarily set, and the data position 103 and the position 104 need only be arbitrarily set.
- either the data value range 102 or the position range 104 can be set to “0”, and if both are set to “0”, the pattern match is completely matched.
- FIG. 68 shows an example of pattern matching of two-dimensional information.
- Information such as image information and map information is representative two-dimensional information.
- Such two-dimensional information is normally stored (arranged) sequentially in a linear array memory for each line in the X-axis direction, with the X-axis or Y-axis being raster-scanned (turned back).
- the definition of information storage (array) may be defined.
- a specific pattern 1 is selected from the image information 5 of the entire range stored on the image 5 as a dragonfly. It is an image when detecting with a magnifying glass.
- pattern 1 by image 5 has five pixels 6 from No. 1 to No. 5, in this example, luminance value data such as 7, 5, 3, 8, and 2 and their positions, that is, pixels (pixels).
- luminance value data such as 7, 5, 3, 8, and 2 and their positions, that is, pixels (pixels).
- an ambiguous pattern match 107 is set with a range at position 6.
- the ambiguous pattern matches 17 and 107 are established by detecting addresses in which the color and luminance data of the inquiry pattern 9 and the position of the coordinates 4 are relatively matched from the target image information 5.
- Such ambiguous pattern matching of images becomes an indispensable tool for image recognition.
- the pattern matching address 57 exists, if the query pattern 9 specified by the local coordinates 112 is detected as the absolute address 7 of the memory on the information array, each pixel constituting the pattern is detected.
- the pattern 1 can be detected by relatively finding out the position 6, that is, as a block of information.
- the sampling point 60 that is the first reference information 110 is particularly important.
- five data are used as sample points, and the reference information 110 and No1 are the samples at the center of the pattern. Even if another sampling point is selected as the reference information 110, there is no problem.
- the reference information 110 that always performs the pattern match 17 information processing 10 is No1, and with this No1 as a reference, the corresponding data is searched from the range up to No2 and N5. If No2 and No3, No3 and No4 If you match sequentially, the range will gradually expand and diverge. Although it is possible to intentionally give a range to the position of the sampling point No1, it is usually wise not to set the range of the position for the reference sample No1 as in the case of one dimension.
- the pattern match of this method is a sampling point 60 selected from a lot of information included in the pattern 1 range. If the data 101 of the information has 256 values and the data varies evenly, there are two types. The probability that the data is in the intended relative sequence is 1/256, and the probability that the three types of data are in the intended relative sequence is 1 / (256 ⁇ 256), and in the case of four types, the probability is even lower. By selecting some appropriate sampling points, pattern match candidates (reference information 110) are narrowed down stochastically, and a specific pattern is selected (pattern match 17).
- FIG. 69 shows an example of a one-dimensional information pattern matching GUI.
- a GUI Graphic User Interface
- This example is a pattern matching GUI for one-dimensional information.
- the target information in the data array 110, the top address on the database address, and the size of the X axis and Y axis can be set.
- the start address and the X-axis (data size) may be specified.
- both the X-axis and Y-axis sizes may be specified. In either case, matching can be performed at the relative position of the information specified by the local coordinates, and the finally matched address can be found.
- This example is an example of a GUI that allows pattern matching 17 from 1 to 16 samples of the matching information 111 in the matching order 109 indicated by M1 to M16 with respect to the reference information 101.
- the sampling points 60 are set to 16 samples, but the present invention is not limited to this, and the sampling points 60 can be increased or decreased.
- the data value 101 and the range 102 can be input as the reference information 110.
- the 16 match information 111 from M1 to M16 are configured such that the data value 101 and its range 102, the position 103 of the information, and its range 104 can be designated by local coordinates 112, respectively.
- the information processing 10 is executed, and the result is output as the absolute address 7 and the global address 113 as the match address 57.
- a plurality of addresses are output if there are a plurality of addresses to which the pattern match 17 is performed, and if not, they are not output.
- the exclusive data 115 is designated as the data values M1 to M16, and the exclusive pattern match 116 can be performed.
- the reference information 110 may be configured as exclusive data 115.
- a function is provided to allow 114 that some of them cannot be pattern matched. With such a configuration, an ambiguous pattern match can be made to function more effectively.
- a GUI that is easier to use is completed by enhancing optional functions such as converting coordinates to distance.
- the data array 100 does not need to be set unless it is a special array other than a linear array.
- the range of both the data 101 and the position 103 of the data is set to “0” so that the pattern match can be completely matched, or the degree of ambiguity can be freely set by specifying only one of the ranges. Can be set.
- a common GUI can be used for pattern matching of one-dimensional information such as stock price information, temperature information, and character information.
- FIG. 70 shows an example of a two-dimensional information pattern matching GUI.
- the basic configuration is exactly the same as the one-dimensional information, but in the two-dimensional information, the positions of the reference information 110 and the match information 111 are the two-dimensional local coordinates 112 of the X axis and the Y axis.
- the data array of the two-dimensional information can be input for each of the X axis and the Y axis so that the local address 112 can be converted into the global address 113 and the absolute address 7.
- Two-dimensional information such as an image is often enlarged, reduced, or rotated. In such a case, by using the coordinate transformation function 117, one query pattern is transformed into various patterns and pattern matching is performed. It becomes possible to do.
- a common GUI can be used for pattern matching of two-dimensional information.
- FIG. 71 shows an example of a GUI for image information pattern matching.
- An image with color 2 stores color 2 information, R, G, and B separately for each pixel (pixel) 6. Therefore, when one pixel (pixel) 6 is globally addressed 113, one image Each color 2 information can be set for each gribal address 113. By adopting this method, pattern matching can be performed in units of 6 pixels.
- GUIs Three types of GUIs have been introduced based on the definition of pattern matching so far, but it is possible to unify them into one GUI, and various applications such as selecting and using the optimal GUI for the target information are possible. It is.
- FIG. 72 is a conceptual diagram of information processing for pattern matching using this method. Conditions are set for both the data 101 and the range 102 of the inquiry pattern 9 and the position 103 and the range 104 of the data of the inquiry pattern 9, and information processing is executed by the pattern match command 17. These condition setting and information processing can be performed collectively or individually.
- the basic idea is that the data detection process 10 is performed based on the data 101 of the query pattern 9 and its range 102, and the address matching process 10 is performed based on the data position 103 and the range 104 of the query pattern 9, and this is repeated 10
- the pattern match candidates initially set as the reference information 110 are sequentially subjected to the narrowing processing 10 and the remaining absolute address 7 is output as the pattern match address 757.
- the absolute address 7 the pattern is recognized and the position of the pattern is detected.
- the information to be subjected to the pattern matching 17 can be further distributed and the pattern matching 17 can be performed by the parallel processing 10. Be free. Furthermore, it goes without saying that this can also be realized in the memory 51 having an information narrowing function.
- a general database is composed of one-dimensional or two-dimensional information in most cases, and this pattern matching can be used for general purposes. If it is information data that can be freely configured, an effective and efficient pattern matching can be achieved by using an arrangement suitable for the principle of pattern matching. As an example, high-dimensional information can be shared and used as long as it is an array stored by storing the above two-dimensional information.
- the main points of the present invention described above are as follows. First, the most basic is the arrangement of information. Therefore, by designating this arrangement, pattern matching candidates (reference information) included in this arrangement are selected and the matching partner ( By specifying each other's data value and its position in the match information 111), it becomes possible to process information by standardizing the pattern match, and by defining a range for the data value and its position, an ambiguous pattern match Can be realized, and can standardize all types of pattern matching.
- the position of the information can be either a coordinate value or a distance, and is shown in the cited Japanese Patent Application No. 2005-221974 (which is incorporated herein by reference).
- a method of defining the position of information by Euclidean distance, spatial distance of Manhattan distance, or time series distance according to the type of information and its purpose has been proposed.
- any space, time series, mathematical distance, conceptual coordinates and distance can be used after being converted into the position of the present system.
- a set operation represented by words such as search, search, collation, and recognition by a program using a conventional CPU is a process of searching for specific information from set information stored in a memory. This is a method of obtaining a solution of a set operation by sequentially accessing and referencing (original).
- the location (address) of information is implicitly understood and does not appear on the surface.
- the set operation in the information processing of this method can be realized by ignoring the location (address). Absent.
- the match number counter 21 of the memory 302 having the information narrowing function is simply replaced with a general-purpose set arithmetic circuit.
- the address of a complicated address such as an ambiguous pattern match or edge detection is used. It took a lot of time and effort to generalize the concept of set operations including places.
- the user interface is a GUI (graphic user interface) displayed on a computer display.
- GUI graphics user interface
- the user interface is not limited to a GUI, but can be of any type and display form (including non-display). Includes a user interface.
- the above-described example of pattern matching of images, characters, sounds, and the like may be performed with the arithmetic processing in the arithmetic circuit 224 of the memory 303 according to the present embodiment fixed.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computational Linguistics (AREA)
- Databases & Information Systems (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Character Discrimination (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Memory System (AREA)
- Image Processing (AREA)
Abstract
Description
(情報の集合を一括して論理演算することの意義)
Wikipediaによれば、数学における集合とは、大雑把に言えばいくつかの「もの」からなる「集まり」である。集合を構成する個々の「もの」を元という、と紹介されている。この定義を情報の集合に当てはめて考えると、現在の情報の集合演算は全て元を対象として個別になされる情報処理である。論理和、論理積をはじめとする基本的な論理素子はもとより、現在の情報処理の主役をなす中央処理装置(CPU)による集合演算も、情報集合の元を対象とした情報処理である。
(情報処理における「認識」について)
以上のメモリに記憶された集合情報の中から、特定の情報を探し出す処理の中で最も困難な情報処理である「認識」について説明する。
(情報処理における「パターンマッチング」について)
次に情報におけるパターンやパターンマッチの重要性やその概要を説明する。
(本発明の目的)
本発明は、情報の集合を一括演算処理するための、情報の集合演算機能を備えたプロセッサを提供するものである。
(優先権主張の基礎となる特許出願に記載された発明)
本願に先立って、本願発明者は、以下のような優先権主張の基礎となる特許出願を行っている。
「(1)音声のそれぞれの音素から得られる、スペクトラムもしくはケプストラムのパターンを音素別、周波数別に配列データベースとして用意し
(2)発声される音声の音素から得られる、スペクトラムもしくはケスペクトラムベクトルのパターンを、上記配列データベースに問い合わせすることにより、上記条件にパターンマッチする上記配列データベースのアドレスを検出し
以上(1)(2)により問い合わせ条件の音素を検出することを特徴とする音素認識方法。」
である。
「画像のXY配列の大きさが定義された画像において
(1)画像を構成する、画素の画像情報データ値と、その画素のデータ位置と、の双方を適宜組合せして構成される画像問い合わせパターンを作成するステップ
(2)画像検出の対象となる画像に上記画像問い合わせパターンを問合せする事によりこの画像問い合わせパターンにパターンマッチする画素を上記対象となる画像の中から検出するステップ
以上(1)(2)のステップにより画像処理することを特徴とする画像認識方法。」
である。
「(1)画像中の文字の書体を構成する、画素の画像情報データ値と、その画素の位置と、の双方を適宜組合せして構成される画像文字問い合わせパターンを作成登録用意するステップ
(2)画像文字認識の対象となる画像に上記画像文字問い合わせパターンを問い合わせする事によりこの画像問い合わせパターンにパターンマッチする画素を上記対象となる画像の中から検出するステップ
以上(1)(2)のステップにより画像文字認識処理することを特徴とする画像文字認識方法。」
である。
「情報の配列が定義されて記憶された情報のパターンマッチ検出において
(1)情報の配列の定義を指定するステップ
(2)パターンマッチの候補となる情報のデータ値を指定して基準情報とするステップ
(3)以上(3)の基準情報にマッチさせる複数のマッチ情報のそれぞれのデータ値をそれぞれ独立して指定するとともにこのそれぞれ情報の位置をそれぞれ独立して指定するステップ
(4)以上(1)の基準情報、および(2)の複数のマッチ情報を1組みの問い合わせパターンとしてこの問い合わせパターンにマッチする上記(2)の基準情報のアドレスを検出するステップ
以上(1)から(4)のステップにより情報をパターンマッチ検出することを特徴とするパターンマッチの標準化方法。」
である。
(情報処理における「集合」について)
先に述べた通りWikipediaによれば、数学における集合とは、大雑把に言えばいくつかの「もの」からなる「集まり」である。集合を構成する個々の「もの」を元という、と紹介されている。
オイラー図は集合論の考え方を最も理解しやすくするための概念であり、全体集合103の中から、特定の元105やその部分集合104を探し出す際の考え方をまとめる場合などに頻繁に利用されている。
(情報の集合演算における情報の場所の意義)
そもそも集合演算、つまり情報を探し出し出す処理とは、情報を探し出す対象のメモリ自体の物理的な構造が、アドレスとメモリセルの2つの要素のみで構成されていることから、メモリに記憶された何の情報がアドレス上どこにあるか、反対に一定のアドレスに何の情報があるかを特定すること以外のなにものでもない。
これらの集合演算115の情報処理101は通常プログラムに基づくCPUの情報処理行為によってなされるものである。
(本願発明の概念)
本願発明は、
メモリアドレスごとに情報を記憶しその情報を読み出し可能なメモリであって、このメモリは、外部から与えられる、各メモリアドレスに記憶された情報を比較するための第1の入力221と、各メモリアドレス同士を比較するための第2の入力222と、集合演算条件として(1)部分集合、(2)論理和、(3)論理積、(4)論理否定のいずれか若しくはそれらの2以上の組み合わせを選択可能に指定する第3の入力223と、を入力するための入力手段と、第1の入力に基づき、アドレス毎にこのメモリに記憶された情報と比較し判定する手段208,209と、第2の入力に基づき、このメモリに記憶された情報同士を比較し判定する手段210,211と、第3の入力に基づき、前記第1及び第2の入力に基づく判定結果について論理演算する手段224と、この集合演算結果を出力する手段207とを有するメモリである。
(連想メモリ)
本願発明のメモリは、大きな潜在能力を持ちながらその実力を十分発揮することができなかった連想メモリをヒントにしたプロセッサである。したがって、まず、連想メモリについて説明する。
(情報データの値を集合演算する手段(データ比較回路)及び(情報データの場所の集合演算の手段(アドレス比較回路)について)
図5は、情報絞込み機能を備えたメモリのブロック図の例1である。
この情報絞込み機能を備えたメモリ302は、先に説明の連想メモリ(CAM)301の機能に、外部から与えられるアドレス条件222をもとに、データの場所114を検出するためにアドレス比較回路210を加え、更にその結果を累積カウントするための、マッチ回数カウンタ212、そして勝ち抜いたアドレス、つまりマッチアドレス213を出力するためのプライオリテイアドレスエンコーダ207で構成されたメモリである。
(1次元情報のパターンマッチの情報処理例)
図6は、全文検出の例である。
さらに「処、理、情、報」でも、途中の文字を飛ばしても、アドレス203をシフト比較する際の方向とその位置の対応が的確になされていればどのような配列でも可能である。
従って、探し出したい情報の元105の組み合わせ順序のみが分かっていれば、どのような情報でも探し出せることになる。
従来このような検索を高速で実施する場合には、インデックステーブルや頻繁に検索される情報を引き出しやすくする配列方法など特殊なアルゴリズムを考える必要があったが、データが更新される度にこれらのテーブルやアルゴリズムを変更する必要があった、この技術はこれら情報データの事前加工を全く不要にする。
以上の文字列は背景技術で説明のパターン401の代表的なものであり、上記説明の通り、以上のような文字列は通常の検索を行うことなく、連想メモリ(CAM)301機能と、その出力フラグの位置替えを行うようなメモリアドレスの並列操作216はシフトレジスタの数クロックのシフト演算により容易に検出することが出来るようになる。
したがってこのような完全並列の手法による情報の集合全体の集合演算115はCPUのメモリ空間個別の(情報の元)のスキャン(検索)を全く不要にするので、従来の情報処理101に比較にならない程高速な全文検出が可能になる。
先に述べた特許文献検索に話を戻せば、このような高速情報検出技術を利用することにより、例えば類似、同類な用語(例えばシソーラス)の繰り返しの検出も極めて単純に実現できることを意味する。
この例で示す情報絞込み機能を備えたメモリ302は、先に示した情報の場所114の集合演算115を行なうアドレス比較回路210、およびアドレス領域比較回路211を2次元(X軸、Y軸の2軸)のシフトレジスタで構成したもので、2次元配列情報に最適な構成である。
(2次元情報のパターンマッチの情報処理例)
図8から図18は、以上の構成の情報絞込み機能を備えたメモリ302による2次元情報である画像検出の概念を説明するものである。
図8に示すように、画素406の集合102である画像情報405が、情報絞込み機能を備えたメモリ302に配列記憶されている。
通常のメモリに情報を記憶することと全く同様である。
この場合カウンタは0からN―1個がそれぞれのアドレス、つまり画素406毎に配列されている。
図12に示すように、黒画素406で定めた基準情報421が乗っているマスク217を、問い合わせパターンの、黒と赤の画素の位置、つまり座標404、データの位置414に相当する分、マスク217を移動させてみる。
この時、先ほど窓を開けた基準情報421の位置、座標404、データの位置414から赤の画素が覗けるのは、図中の上部の画素1箇所のみである。
つまり、この基準情報421のマッチ回数カウンタ212が「2」にカウントアップされ、勝ち残り、他の2つの基準位置のマッチ回数カウンタ212は「1」のままで、脱落となる。
本例の場合、6画素が検出されている。
このように座標404、データの位置414に領域を持たせることにより、曖昧なパターンマッチ418が可能になる。
このような、絶対的なアドレス領域の濃度検出は、人間の顔や手など皮膚などの色の領域や、特別な色402や輝度403を持った物体の存在を検出するのに最適である。
季節の温度の変化や、景気の動向など時系列の集合情報、つまり1次元情報の中から、問い合わせパターンに、合致(マッチ)する情報を探し出す際の、イメージであり、一般的な文章などのテキストもこれらの一次元情報の仲間である。
図に示すように、それぞれの情報はそのデータ値(図ではDの値)と基準情報421との相対距離、この場合相対アドレス205(図ではXの値)が指定されている。
図21は2次元情報の検出例を示すものである。
図22は3次元情報の検出例を示すものである。
分子や星座などの集合情報、つまり3次元情報の中から、問い合わせパターン408に、合致(マッチ)する情報を探し出す際のイメージであり、その内容は図20と同様である。
図23は、1次元情報の曖昧検出例を示すものである。
図20で示した1次元情報の中から、曖昧な問い合わせ情報に、合致(マッチ)させる曖昧パターンマッチ418のイメージである。
以上のようなパターンマッチは一例として、画像中の人の顔位置、非顔部分の高速な検出や、車のナンバープレートの高速な文字読み取りなどの分野に最適である。
以上のようなパターンマッチは一例として、分子構造の特定や、宇宙空間の星座の特定、気象データの解析などの分野に最適である。
図は図24で示した2次元情報の曖昧検出の概念をさらに拡張したものである。
図に示すように、領域内のいずれかの場所114に、対象となる情報があるかどうかを検出するものである。
(CPUによるパターンマッチ時間)
検証結果の一つとして、高性能なパソコンで2次元配列の画像(BMP形式)の640×480ピクセルの画像を検索対象画像として、1セット合計5ポイントのサンプリングポイント410によるパターンマッチを行ってみた。
完全一致の場合114m秒の時間が必要であった。
(集合全体一括演算によるパターンマッチ時間)
これまでの情報絞込み機能を備えたメモリ302の研究は主としてFPGAによるものであり、ロジックリソースが不十分な回路構成によるパターンマッチであるものの1m秒以下で1セット合計5ポイントの曖昧パターンマッチが可能あることが確認されている。
CPUによるパターンマッチの時間と比較すれば100万倍以上である。
1秒間であれば、200,000回のパターンマッチ409が可能である。
一例として、認識処理機能を持たせたインテリジェンスなカメラの場合、数十ワットクラスのCPUが内蔵されている。
以上の内容は携帯用のバッテリ機器に大きな意味をもつ。
(データ比較回路、アドレス比較回路に加え、集合演算回路を備えたメモリの例)
これまで様々な観点からパターンマッチ409が情報処理101で極めて有効な情報処理101手段であるとともに、情報絞込み機能を備えたメモリ302が、CPUが苦手な情報処理の一つである、情報のパターンマッチ409に有効であることを示してきた。
そもそもパターンマッチとは、メモリ自体の物理的な構造が、アドレスとメモリセルの2つの要素のみで構成されていることから、メモリに記憶されたパターンの情報がアドレス上どこにあるか、反対に一定のアドレスに何のパターンがあるかを特定すること以外のなにものでもない。
図に示すように集合演算機能を備えたメモリ303は情報絞込み機能を備えたメモリ302のマッチ回数カウンタ212の部分を、演算回路224に置き換え、外部から与えられる論理演算条件223に基づく、論理和109、論理積110、論理否定111などの演算を指定された条件で任意に実現できるように構成したものである。
(文献検索の例)
図30は、文献検索の場合を例にとった、グラフィックユーザーインターフェイス(GUI)の例である。
本例のような集合演算の場合、1Mアドレスであろうと100クロック程度の処理で集合演算が可能になるので、熱が問題にならない10n秒のクロックの場合でも1μ秒程度で全体の集合演算が完了できる。
仮に特許検索に、シソーラスの考えを組み込んだ検索ができるようになれば利用者の負担は大幅に軽減され、しかも見落としのない確実な特許検索が可能になる。
(エッジ検出の例)
図35は、集合演算機能を備えたメモリによるエッジ検出の例である。
以上説明の通りエッジ検出はパターンマッチ同様画像認識を行う上で必要不可欠の画像処理でありグレースケールのみならずカラーによる複雑なエッジ検出は従来概念を大きく変える画像処理ツールとなる。
(画像パターンマッチングの例)
以下、画像パターンマッチングの例を、図36~図50を参照して説明する。なお、以下の説明では、優先権主張の基礎出願との対応関係が分かりやすいように参照符号をそのままにして説明していることに留意されたい。
(実施例1-1)
本発明の特徴を利用した物体の認識の例を説明する
図39は画像の排他パターンマッチの説明図であり、対象となる画像情報5、の画素6の中から物体8の領域(エリア)と輪郭(エッジ)を効率的に検出する例を示している。
つまり特定の大きさの白の物体8、本例では横幅が6画素のエリアを持った白い物体(ボール)のみが検出される。
(実施例1-2)
図40は、近傍4画素パターンによるエッジコード符号化の説明図である。
通常の画像情報は輝度や色を表現(表示)する目的として、獲得され記憶されている。
(実施例1-3)
図41は、近傍8画素パターンによるエッジコード符号化の説明図である。
先に説明の図40では上下左右4画素の近傍画素をエッジコード化したものであったが本図は更に4つの、左上、右上、左下、右下のコーナの画素を含み8つの画素でエッジコード12を符号化した例である。この場合には合計256通りのエッジパターンが存在しより細かなエッジを検出することが出来る。
(実施例1-4)
図42は、情報絞込み機能を備えたメモリを用いた画像パターンマッチの情報配列の説明図である。
(実施例1-5)
図43は、物体のエッジコードの応用例の説明図である。
(実施例1-6)
図44は、局部パターンマッチによる無作為および作為パターンマッチの説明図である。パターンマッチ技術を使って画像上の物体を論理的に効率よく見つけ出す手法を説明する。
シャッターを押した瞬間の画像とシャッターが切れる直前の画像の各部分を部分、または局部的にパターンマッチすることにより、画面全体が移動したのか(手振れ)画像の一部が移動したのか(対象物の移動)、その双方の組み合わせなどが極めて簡単に検出できる。
図45は、物体の変化画像検出の説明図である。
図46は、局部パターンマッチによる物体の対応点検出の説明図である。
本図では空間中に赤(R)、B(青)、黄(Y)、緑(G)の4色の風船のような物体8が浮かんでいる画像を左右のカメラの画像としてとらえたものである。両眼カメラで捉えた物体の画像と実際の物体はエピポーラ平面で構成されるので、両眼カメラの距離が分かれば三角測量により物体の奥行きを含むXYZ軸の位置が計測可能である。
(実施例1-8)
図47は、エッジコードによる物体認識の説明図である。
(実施例1-9)
図48は、立体計測を利用した人物認識の例である。
ここで説明の人物認識は画像範囲の中から人物の特徴を認識して個人名を特定することを意味している。
本例では図に示すように画像に表示される顔の眼や鼻、ホクロ、キズなどの特徴をパターンとして捉え両眼視差による対応点をパターンマッチ17させ、その結果から得られた、X、Y、Zの3軸の実寸法つまり眼の大きさ、鼻の高さなどを計測した例である。
言うまでもなく、このような特徴は眼や、鼻、ホクロ、キズなどの顔にかかわらず、口、眉、髭、手、足など人物として固有の特徴が得られるものであれ人体のどの部分でも構わない。色によらない形状や寸法による認識が出来れば人種を越えた人物認識が可能になる。これらの人物を識別するに相応しい特徴をサンプリンし計測できる解像度の立体計測カメラシステムを用いればよい。
以上のような物体の寸法やその距離が容易に求められる事は物体の実寸が分かる事になり、図に示した、3つの物体の大きさから、トラックのような大きさの物体、顔のような大きさの物体、リンゴのような大きさの物体と、物体の範囲を限定することが可能になり、後は色の情報や詳細な情報を基にクラス分けして物体を認識することが可能になる。
(実施例1-11)
図50は、パターンマッチによる物体認識の概念説明図であり、これまでの説明をまとめたものである。
知識処理は物体の様々な特徴が様々なカテゴリーでクラス分けされて登録されており、画像処理から与えられた特徴を基に、データベースに登録された物体を知識処理で探し出す。また反対に知識処理で特徴を指定し画像処理でその特徴に合う特徴があるか否かを探し出すことも同様に行われる。
本発明の請求項の内容を本明細書の内容に照らし合わせてその要点記述する。画像のパターンマッチに関する特許は数多く出願されているが、メモリ上の画像情報のデータ配列そのものに着目し、極めて単純に画素の画像情報データ値と、その画素のデータ位置と、の2つを基にパターンマッチさせる例はない。
画像のXY配列の大きさが定義された画像において
(1)画像を構成する、画素の画像情報データ値と、その画素のデータ位置と、の双方を適宜組合せして構成される画像問い合わせパターンを作成するステップは、情報絞り込み機能を備えたメモリ51(303)上に配列された画像の中から特定のパターンを見つけ出す際の一例として無作為もしくは作為的な問い合わせパターンの作成方法で示した内容である。(パターンマッチングしたデータを検索する行程)
更に
(2)画像検出の対象となる画像に上記画像問い合わせパターンを問合せする事によりこの画像問い合わせパターンにパターンマッチ17する画素を上記対象となる画像の中から検出するステップは、対象となるメモリ画像にサンプリングされたパターンを問い合わせして、パターンマッチしたアドレス(画素)を検出することである。(パターンマッチングしたアドレスを検索する行程)
以上(1)(2)のステップにより画像処理することを特徴とする画像認識方法(様々なパタンーンマッチの組合せ)が本発明の画像認識の特徴である。
(音声パターンマッチングの例)
以下、音声パターンマッチングの例を、図51~図57を参照して説明する。なお、以下の説明では、優先権主張の基礎出願との対応関係が分かりやすいように参照符号をそのままにして説明していることに留意されたい。
本図はある音素の周波数2毎の強度(パワー)4をスペクトラム16計測したものであり、周波数2毎のその強度(パワー)4を配列番号15として配列8したものである。
一方、本図の音素パターン17は高音域に大きな強度(パワ―)4を持った音素パターン17である。図52、図53で示した通り、音素5をスペクトラム16波形は音素のパターン17を意味しているので、このパターンを正しくパターンマッチさせ読みとることが出来れば、確実な音素5認識が可能になる。
本例では、先に説明した、問い合わせするデータに範囲18を設ける場合の例であり、強度(パワー)4のレベルを16レベルとして、与えられたデータ52に最大値10、最小値11の範囲18を持ってパターンマッチ9をかける、あいまいパターンマッチ13の例を示している。
図54で説明の配列8を、情報絞込み機能を備えたメモリ50(303)に配列8記憶させたものである。
以上により検出された音素の組み合わせさえたものが単語、語彙である、音素の配列で定まる語彙6のマッチングにもこのパターンマッチ方法が応用できる。
極めてシンプルで高速な語彙検出であり、複雑なアルゴリズムや他のデータテーブル等全く不要である。
ここで語彙認識のあいまいパターンマッチ13について説明する。
(文字パターンマッチングの例)
以下、文字画像パターンマッチングの例を、図57~図66を参照して説明する。なお、以下の説明では、優先権主張の基礎出願との対応関係が分かりやすいように参照符号をそのままにして説明していることに留意されたい。
このパターン1の色や輝度のデータとその座標4の位置が相対的に合致する事により画像パターンマッチ17が成立する。
画像5はXY軸2軸の情報として扱われる2次元情報の代表的な情報である。どのような画像5も画像5を構成する画素(ピクセル)6の数がXY軸両軸決められており、その積算されたものが総画素数となる。原則的に画像5の基本となる輝度3情報や色2の三原色等の色2情報はこの画素6単位で獲得され記憶媒体に記憶される。
また画像5を構成する画素6は輝度3情報の場合1つの種類のデータをメモリに記憶するだけであるが、色2情報の場合、通常色の3原色R、G、Bをそれぞれ独立して記憶する必要があり通常は1画素6当たり3つの画素情報を記憶する必要がある。従って1画素6当り3アドレスで色2情報を記憶する場合、実際のメモリは画素6数の3倍のアドレス数が必要になる。いうまでもなく1ラインの画素6数(n)が分かっていれば、画像5上のどの画素6のどの色2の情報がメモリ上のどの位置に記憶されているのか、その反対も容易に変換することが可能である。
対象となる画像のサイズが情報絞込み機能を備えたメモリ51(303)の情報処理容量より大きい場合は、画像をいくつかに分割して、分割単位ごとにパターンマッチをかければよい。この場合、分割部分の画像のパターンマッチ17に影響されないよう、パターンマッチ17する画像の大きさの分X軸、Y軸互いにオーバラップするよう分割し、いずれかの分割画像内に対象なる画像がパターンマッチ掛るようにパターンマッチすればよい。
例えば、現在のデジタルフルハイビジョン画像は横(X軸)1920ピクセル×縦(Y軸)1080、合計約2,073,600ピクセルの画素で構成された映像であり、0ピクセルから2,073,599ピクセルまで線形配列された絶対アドレス7に画素毎の画像情報が記憶される。この画像空間の任意の2つのピクセルの相対位置は一次元データのグローバルアドレス104の距離で表現することが出来る。
図59は排他パターンマッチの説明図であり、対象となる画像情報5、の画素6の中から物体8の領域(エリア)9と輪郭(エッジ)10を効率的に検出する例を示している。特定の色2や輝度3の領域(エリア)9を持った物体8を探す場合、その物体の背景パターンは無限に存在するために様々な色2や輝度3のデータ54によるパターンマッチ17を必要回数繰り返しする必要がある。
動画中のある物体が1フレーム毎に少しずつ大きさや形状が変化する場合には1フレーム毎に物体の形状を更新して次の1フレームの物体とマッチングを取ればよい。この様な移動物体の追従は映像装置やセキュリテー装置に不可欠の技術である。
本方式のパターンマッチは強力である、これまで大がかりになりがちな画像処理を極めて単純処理することが可能になる。一般の文字は一定の色とその形状(領域)で成り立つものであり、文字以外の部分が特定の色であったり、特定の柄や映像であっても文字以外の色で領域外を指定することが出来るので、この排他パターンマッチを利用すると極めて簡素化した文字認識のパターンマッチが可能になる。
本例で示す日本語は様々な種類の文字が組み合わされた言語である、そのうち特に数の多い漢字は、常用漢字で2000文字程度、複雑な漢字を含めて3000文字程度、さらにこれらに日常使用される、ひらがな、カタカナ、アラビア数字、アルファベット、これら以外の記号等を含め最大5000種の文字記号を認識する必要がある。最大の文字数を持つ漢字でも現在日常中国使用されている漢字は6000文字から7000文字と言われている。従って、中国語の場合、最大10,000文字を認識する必要がある。
画像中の文字を認識するには大きく2つの方法がある。一つは画像中の文字一つ一つの書体の特徴を抽出して、その特徴を基にその文字が何であるかを問い合わせする場合。この方法による文字認識は本願発明者により出願された特願2012-101352に紹介する画像認識や物体認識によって実現できる。もう一つは文字一つ一つの書体の領域と非領域を識別するために必要な複数のサンプリングポイントをあらかじめ決めておいて、このサンプリングポイントにマッチする画像を文字と認識する場合である。この方は文字の特徴を予めサンプリングしておき、画像上のすべての文字を対象として並列にパターンマッチさせることが出来るので、前者に比較してより効率よく高速で精度よくパターンマッチ指定した文字が認識できる。
本願発明は後者の方法による文字認識方法を対象にしている。
図61は、文字パターンサンプリングポイント作成例Aの説明図である。
特定の日本語文字「あ」を文字認識するために、No1、No2、No3、No4、まで文字の領域部分には領域内サンプリングポイント61と領域外部分には領域外サンプリングポイント62のそれぞれ2つの4つのサンプリングポイント60をローカルアドレス103の座標4上に付与した場合の例である。
これらの2種のサンプリングポイント60の意味するものは、当該サンプリングポイントおよびその近傍が、文字の領域部分であるかそれ以外であるかを特定するためのものである。
一般的な文字の場合、座標4空間中の領域面積は、領域外面積より少なくなり、全ての文字の領域の存在確率は1/2以下となるが、その反対に非領域の存在確率は1/2以上になる。
以上のようにどのような文字においても領域部分には領域内サンプリングポイント61と領域外部分(非領域)には領域外サンプリングポイント62の双方を同一数配置した場合、1サンプルポイントのそれぞれの座標4が文字の領域であるか否か、反対に領域外であるか否かの平均確率は1/2となる。従って以上の4つのサンプリングポイントがマッチする確率は概ね1/(2*2*2*2)=1/16となる。
図62は文字パターンサンプリングポイント作成例Bの説明図であり、特定文字「あ」のいくつかの書体(フォント)を重ね合わせ、領域が一致する部分に領域内サンプリングポイント61、いずれの書体(フォント)の領域外部分に領域外サンプリングポイント62を合計30個所付与した場合である。
万一複数の文字が認識選択される場合には、その文字のサンプリングポイントを部分的に修正すればよい。
外国映画に字幕は付き物である。
映画の字幕の場合、1シーンに現れる字幕は最大2行、文字数40文字程度であり表示時間は1秒から5秒程度である。情報絞り込み機能を備えたメモリ51(303)は完全なるハードウエアパターンマッチなので1マイクロ秒もあれば1回のパターンマッチが可能である。1マイクロ秒1回のパターンマッチで30サンプルポイントの場合、一文字が30マイクロ秒であり、5千文字の日本語では0.15秒、1万文字の中国語の場合でも0.3秒で一画面当たりのすべての文字をパターンマッチさせることが出来る。全世界文字2万文字でも0.6秒で一画面当たりのすべての文字をパターンマッチさせる事が可能である。
図に示すように、日本語5千種文字のそれぞれの文字101にNo1からNo30までのサンプリングポイント60のパターンマッチ17用のOCRパターンデータベース105が登録されている。本例では日本語を例にしているが、英語でも、中国語でも、全世界の言語を一括して登録することも可能である。
以上の説明通りCPUは文字認識そのものにかかわる一切の処理をする必要がなく、必要なのは文字認識処理全体を統括し、情報絞込み機能を備えたメモリ51(303)にパターンマッチの指示を与え、パターンマッチ結果(絶対アドレス7)を読み取り、その結果必要な処理をするだけである。日本語だけの場合、5千文字として0.15秒で全てのパターンマッチが実現できる。通常映画の場合の字幕の文字色は白で、書体(フォント)102も固定されており変化しない。
図66は、文書画像の文字認識の例である。
先に示した通り、本方式の特徴であるハードウエアによる並列処理によるパターンマッチを使用すれば画像の中に含まれる文字の数が幾つであってもパターンマッチに要する時間は一定である。従って先ほどの映画の字幕の場合でも文字数が数百に及ぶ文書画像であっても1画面当たりの文字認識は同じ時間で可能である。
(パターンマッチの標準化)
以下、パターンマッチングの標準化の例を、図67~図72を参照して説明する。なお、以下の説明では、優先権主張の基礎出願との対応関係が分かりやすいように参照符号をそのままにして説明していることに留意されたい。
現在のコンピュータのメモリの配列情報でとり得る曖昧さは、メモリに記憶格納する情報データの曖昧さと、メモリに記憶格納する場所(アドレス)の2つのみである。つまり情報の集合体であるパターンは一定の定義に基づき配列情報として記憶格納されているのでこの二つを曖昧に情報処理することが出来れば人間に極めて近い認識が可能になることである。
この方法によればメモリに記憶する情報(データ)とその場所(メモリアドレス)は通常のままで、その配列も一般的な情報配列で実現することが出来る。
前述のように、温度の時系列データはデータベースに毎月の最高平均気温が定められた定義によって曖昧さを持たずに記憶格納(配列)されている。人間が感じる感触は通常の絶対的な尺度を持たない曖昧なものであり、例えば暑さを示す尺度であれば、極めて暑い、暑い、過ごしやすい、寒い、極めて寒い、などのような5段階から精々10段階のレベルである。
No3の情報を探し出す際もNo2との相対位置によるものでなく、No1とNo3でマッチする情報を探し出すことがこの発明の原点である。
以上の例は3組の情報データのパターンマッチ17であるが、情報データの組合せはいくら多くてもかまわない。またこれらの3つの情報のデータ101値ならびにその範囲102は任意設定可能であり、そのデータの位置103ならびにその位置104も任意設定可能であればよい。
画像情報や地図情報などの情報は代表的な2次元情報である。
この様な2次元情報は通常X軸またはY軸をラスタースキャン方式(折り返し)X軸方向の1ライン毎に線形配列のメモリに順次記憶格納(配列)されている。1次元情報同様に、X軸を左から記憶するのか右から順に記憶するのか、Y軸で折り返すのかなどは任意であり、情報格納(配列)の定義が定められていればよい。
図に示すように、パターンマッチするアドレス57が存在する場合、ローカル座標112で指定された問い合わせパターン9は、情報配列上ではメモリの絶対アドレス7として検出されれば、パターンを構成するそれぞれの画素6の位置を相対的に見つけ出すこと、つまり情報の塊としてパターン1が検出出来る。
本願発明を分かり易く効率的に使用するためには、問い合わせパターン9のデータを入力するGUI(Graphic User Interface)が必要である。本例は1次元情報のパターンマッチ用GUIである。
この様な構成とすることにより曖昧なパターンマッチをより有効に機能させることが可能になる。座標を距離に変換するなどのオプション機能を充実させることにより更に使いやすいGUIが完成される。
以上のような構成で、株価情報や気温情報、さらには文字情報などの1次元情報のパターンマッチに共通のGUIが利用できる。
基本的な構成は1次元情報と全く同様であるが、2次元情報では、基準になる情報110ならびにマッチ情報111の位置がX軸、Y軸の2次元のローカル座標112となっている。また、本例では2次元情報のデータ配列をX軸、Y軸それぞれ入力可能にしてローカル座標112からグローバルアドレス113、絶対アドレス7変換が可能なように構成されている。画像のような2次元情報は、情報が拡大、縮小、回転することがしばしばおこる、このような場合、座標変換117機能を利用することにより、1つの問い合わせパターンを様々に座標変換してパターンマッチすることが可能になる。
色2のついた画像は1画素(ピクセル)6毎に色2情報、R、G、Bが個別に記憶されている、従って1画素(ピクセル)6をグローバルアドレス113する場合には、1つのグリーバルアドレス113毎にそれぞれの色2情報を設定できるようにしたものである。この方式とすることにより、画素(ピクセル)6単位でのパターンマッチが可能になる。
問い合わせパターン9のデータ101とその範囲102と、問い合わせパターン9のデータの位置103とその範囲104と、の双方の条件設定を行い、パターンマッチ指令17によって、情報処理が実行される。これらの条件設ならびに情報処理は一括して行うことも、個別に実行するのも自由である。
情報の位置は座標値とすることも、距離とすることのいずれでも対応可能であり引用した特願2005-212974号文献(この言及により本願明細書に組み込まれることとする)にも示されるように、情報の種類やその目的に応じてユークリッド距離、マンハッタン距離の空間距離さらには時系列の距離で情報の位置を定義する方法が提案されている。
(本願発明の経緯とまとめ)
従来のCPUを使ったプログラムによるサーチや検索、照合、認識などの言葉に代表される集合演算は、メモリに記憶された集合情報の中から、特定の情報を探し出す処理であり、メモリ上の情報(元)を逐次アクセスし、参照し、集合演算の解を求める方法である。
また、上記で説明した画像、文字、音声等のパターンマッチングの例は、本実施形態にかかるメモリ303の演算回路224における演算処理を固定の状態にして実施するようにしてもよい。
102 集合
103 全体集合
104 部分集合
105 元
106 位置
107 領域
108 距離
109 論理和
110 論理積
111 論理否定
112 正論理
113 負論理
114 情報(データ)の場所
115 集合演算
116 情報(データ)の合致
117 情報(データ)の値
201 メモリ
202 メモリセル
203 アドレス
204 絶対アドレス
205 相対アドレス
206 アドレスデコーダ
207 プライオリテイアドレスエンコーダ
208 データ比較回路
209 データ範囲比較回路
210 アドレス比較回路
211 アドレス領域比較回路
212 マッチ回数カウンタ
213 マッチアドレス
214 勝抜きフラグ(FG)
215 領域勝抜きフラグ(FG)
216 メモリアドレスの並列操作
217 マスク
221 データ条件
222 アドレス条件
223 論理演算条件
224 演算回路
301 連想メモリ(CAM)
302 情報絞込み機能を備えたメモリ
303 集合演算機能を備えたメモリ
401 パターン
402 色
403 輝度
404 座標
405 画像情報
406 画素
407 データベース
408 問い合わせパターン
409 パターンマッチ
410 サンプリングポイント
411 データ配列
412 データ
413 データの範囲
414 データの位置
415 データの領域
416 曖昧情報
417 曖昧パターン
418 曖昧パターンマッチ
419 曖昧認識
420 マッチ順序
421 基準情報
425 ミスマッチ許容
426 排他データ
427 排他パターンマッチ
428 座標変換
Claims (50)
- メモリアドレスごとに情報を記憶しその情報を読み出し可能なメモリであって、
このメモリは、
外部から与えられる、各メモリアドレスに記憶された情報を比較するための第1の入力と、各メモリアドレス同士を比較するための第2の入力と、集合演算条件として(1)部分集合、(2)論理和、(3)論理積、(4)論理否定のいずれか若しくはそれらの2以上の組み合わせを選択可能に指定する第3の入力と、を入力するための入力手段と、
第1の入力に基づき、アドレス毎にこのメモリに記憶された情報と比較し判定する手段と、
第2の入力に基づき、このメモリに記憶された情報同士を比較し判定する手段と、
第3の入力に基づき、前記第1及び第2の入力に基づく判定結果について論理演算する手段と、
この集合演算結果を出力する手段と
を有することを特徴とする集合演算機能を備えたメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第3の入力による前記集合演算結果に、新たに与えられる前記第1~第3の入力により集合演算を繰り返し実行する手段を有することを特徴とする集合演算機能を備えたメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記情報の、前記第1~第3の入力による集合演算の少なくとも1つ以上を並列処理で実行する手段を具備することを特徴とするメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1の入力は、
比較する情報を表す値、及び
比較条件として完全一致、部分一致、範囲一致、若しくはこれらの組み合わせの指定、
を含むものであることを特徴とする集合演算機能を備えたメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1の入力による判定を連想メモリ手段で実現するものである
ことを特徴とする請求項1記載の集合演算機能を備えたメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
第2の入力は、
比較する情報の位置、その位置を基準にした一定の領域、若しくはそれらの組み合わせ
を含むものであることを特徴とするメモリ。 - 請求項6記載の集合演算機能を備えたメモリにおいて、
前記第2の入力の前記比較する情報の位置は、
相対的位置、絶対的位置若しくはそれらの組み合わせを含むものであることを特徴とするメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第2の入力による判定を行う手段は、
メモリアドレスを並列操作することによって実行する手段を含むものである
ことを特徴とするメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記入力手段は情報の配列・順序を指定する第4の入力(画像の大きさなど)をさらに入力するためのものであり、
前記第4の入力に指定された情報の配列・順序に基づいて、前記情報の判定を実行させるものである
ことを特徴とする請求項1記載の集合演算機能を備えたメモリ。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第3の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであることを特徴とするメモリ。 - 請求項10記載の集合演算機能を備えたメモリにおいて、
前記問い合わせ情報パターンは、エッジ検出のための問い合わせ情報であることを特徴とするメモリ。 - 請求項10記載の集合演算機能を備えたメモリにおいて、
前記パターンマッチングは、
文字情報を一例とする1次元情報
画像情報を一例とする2次元情報
動画情報を一例とする3次元情報
配列が定義されたN次元情報
のいずれかに対して実行されるものであることを特徴とするメモリ。 - 請求項10記載の集合演算機能を備えたメモリにおいて、
前記パターンマッチングの前記問い合わせ情報パターンにより
視覚認識
聴覚認識
味覚認識
臭覚認識
触覚認識
の少なくとも1つを処理するものであることを特徴とするメモリ。 - CPUを一例とする他の半導体と一体に組み込みされたことを特徴とする請求項1記載の集合演算機能を備えたメモリ。
- 請求項1記載の集合演算機能を備えたメモリを含んだ装置。
- 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
画像のXY配列の大きさが定義された画像において
(1)画像を構成する、画素の画像情報データ値と、その画素のデータ位置と、の双方を適宜組合せして構成される画像問い合わせパターンを作成するステップと、
(2)画像検出の対象となる画像に上記画像問い合わせパターンを問合せする事によりこの画像問い合わせパターンにパターンマッチする画素を上記対象となる画像の中から検出するステップと
により画像処理することを特徴とする画像認識方法。 - 請求項16記載の方法において、
前記画素の画像情報は
(1)物体の色または輝度情報
(2)物体の輪郭(エッジ)または領域(エリア)情報
(3)物体の奥行き情報
の少なくとも1つの画素の画像情報を基に画像処理することを特徴とする画像認識方法。 - 請求項17記載の方法において、
前記、物体の輪郭(エッジ)または領域(エリア)情報は、画像上の任意の1画素と、その近隣画素と、の比較結果によるコードを画面全体の画素に符号化するとすることを特徴とする、画像認識方法。 - 請求項16記載の方法において、
(1)単眼カメラで撮像された静止画像または動画像
(2)複眼カメラで撮像された静止画像または動画像
(3)CGで作成された静止画像または動画像
の少なくとも1つの画像の認識をすることを特徴とする画像認識方法。 - 請求項16記載の方法において、
(1)複眼カメラで撮像された静止画像の認識ステップと、
(2)複眼画像に表示される物体の対応点をパターンマッチで検出するステップと、
(3)以上で検出された物体を立体計測するステップと
を有し、画像物体の形状の認識をすること特徴とする画像認識方法。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
上記問い合わせ情報パターンは、画像を構成する、画素の画像情報データ値と、その画素のデータ位置と、の双方を適宜組合せして構成されるものであり、
(1)画像全体の中から無作為にパターンを採取し、採取されたパターンを問い合わせ情報パターンとしその特徴に合致する物体を検出するステップ、
(2)予め認識する物体を特定しその物体の特徴を検出するための問い合わせ情報パターンを作成し、画像全体の中からそのパターンに合致する物体を検出するステップ、
(3)以上(1)(2)の組み合わせによるパターンに合致する物体を検出するステップ、
の少なくとも1つのステップにより物体を認識する物体認識方法。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
(1)音声のそれぞれの音素から得られる、スペクトラムもしくはケプストラムのパターンを音素別、周波数別に配列データベースとして用意し
(2)発声される音声の音素から得られる、スペクトラムもしくはケスペクトラムベクトルのパターンを、上記配列データベースに問い合わせすることにより、上記条件にパターンマッチする上記配列データベースのアドレスを検出し
以上(1)(2)により問い合わせ条件の音素を検出することを特徴とする音素認識方法。 - 請求項22記載の方法において、
前記パターンの変化許容する範囲を設定しパターンマッチすることを特徴とする音素認識方法。 - 請求項22記載の方法において、
登録された、前記周波数別のスペクトルもしくはケプストラムのデータの配列データベースの中から必要とする周波数のみをパターンマッチ条件とすることを特徴とする音素認識方法。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
(1)言葉のそれぞれの語彙を、語彙別に音素の配列データベースとして用意し
(2)発音される音声の音素の配列を、上記配列データベースに問い合わせすることにより、上記条件にパターンマッチする上記データベースのアドレスを検出し
以上(1)(2)により問い合わせ条件の語彙を検出することを特徴とする音声認識方法。 - 請求項25記載の音声認識方法において、
前記音素の配列の変化を許容するパターンマッチすることを特徴とする音声認識方法。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
(1)画像中の文字の書体を構成する、画素の画像情報データ値と、その画素の位置と、の双方を適宜組合せして構成される画像文字問い合わせパターンを作成登録用意するステップと
(2)画像文字認識の対象となる画像に上記画像文字問い合わせパターンを問い合わせする事によりこの画像問い合わせパターンにパターンマッチする画素を上記対象となる画像の中から検出するステップと
以上(1)(2)のステップにより画像文字認識処理することを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
前記その画素の位置は
(1)前記文字書体の領域内を示すサンプリングポイント
(2)前記文字書体の領域外を示すサンプリングポイント
以上(1)(2)の画素の位置を基に画像文字認識処理することを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
前記サンプリングポイントは座標変換によって
(1)拡大縮小された文字
(2)回転された文字
以上(1)(2)の画像文字認識処理することを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
前記サンプリングポイントを範囲パターンマッチによって画像文字認識処理することを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
前記画素の画像情報データ値は
(1)前記領域内を示すサンプリングポイントの画素情報は前記文字書体の色データ値もしくは輝度データ値
(2)前記領域外を示すサンプリングポイントの画素情報は前記文字書体の色データ値もしくは輝度データ値以外のデータ値
以上(1)(2)の画素の画像情報データ値を基に画像文字認識処理することを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
複数種類の前記文字書体を1文字毎に重ね合わせ当該文字の
(1)前記文字書体共通領域内に前記文字書体の領域内を示すサンプリングポイント
(2)いずれの前記文字書体の領域に含まれない部分に前記文字書体の領域外を示すサンプリングポイント
以上(1)(2)のサンプリングポイントを決めることを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
前記画像文字問い合わせパターンは相対アドレスとし、この相対アドレスを絶対アドレスに変換してパターンマッチすることを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
前記文字は
単一言語文章に必要な文字全てを対象にして一文字毎に、シーケンシャルに画像空間内全区間パターンマッチすることを特徴とする画像文字認識方法。 - 請求項27記載の画像文字認識方法において、
全世界言語文章に必要な文字全てを対象にして一文字毎に、シーケンシャルに画像空間内全区間をパターンマッチすることを特徴とする画像文字認識方法。 - 請求項1記載のメモリを有し、
(1)画像中の文字の書体を構成する、画素の画像情報データ値と、その画素の位置と、の双方を適宜組合せして構成される画像文字問い合わせパターンを登録したデータベースと
(2)画像文字認識の対象となる画像に上記データベースの画像文字問い合わせパターンをシーケンシャルに問い合わせする事によりこの画像問い合わせパターンにパターンマッチする画素を上記対象となる画像の中から逐次検出する手段と
(3)逐次検出されたパターンマッチアドレスから画像上の文字とその位置を認識して、認識された文字とその配列を記憶する手段と
以上(1)から(3)の手段を具備することを特徴とする画像文字認識機能を備えた情報処理装置。 - 請求項36記載の情報処理装置において、
前記認識された文字とその配列による文字列をアノテーションデータとすることを特徴とする画像文字認識機能を備えた情報処理装置。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
情報の配列が定義されて記憶された情報のパターンマッチ検出において
(1)前記第4の入力として情報の配列の定義を指定するステップと
(2)パターンマッチの候補となる情報のデータ値(前記第1の入力)を指定して基準情報とするステップと
(3)以上(2)の基準情報にマッチさせる複数のマッチ情報のそれぞれのデータ値をそれぞれ独立して指定するとともにこのそれぞれ情報の位置(前記第2の入力)をそれぞれ独立して指定するステップ
(4)以上(1)の基準情報、および(2)の複数のマッチ情報を1つの問い合わせ情報パターンとしてこの問い合わせ情報パターンにマッチする上記(2)の基準情報のアドレスを検出するステップ
以上(1)から(4)のステップにより情報をパターンマッチ検出することを特徴とするパターンマッチ標準化方法。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1~第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
情報の配列が定義されて記憶された情報のパターンマッチ検出において
(5)前記第4の入力として情報の配列の定義を指定するステップ
(6)パターンマッチの候補となる情報のデータ値とその範囲を指定して基準情報とするステップ
(7)以上(6)の基準情報にマッチさせる複数のマッチ情報のそれぞれのデータ値とその範囲をそれぞれ独立して指定するとともにこのそれぞれ情報の位置とその範囲をそれぞれ独立して指定するステップ
(8)以上(6)の基準情報、および(7)の複数のマッチ情報を1つの問い合わせ情報パターンとしてこの問い合わせ情報パターンにマッチする上記(6)の基準情報のアドレスを検出するステップ
以上(5)から(8)のステップにより曖昧情報をパターンマッチ検出することを特徴とするパターンマッチの標準化方法。 - 前記(2)(6)のステップを並列にマッチ実行することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 前記(4)(8)のステップを並列にマッチ実行することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 前記(2)(6)の基準情報ならびに前記(3)(7)のマッチ情報の位置を座標で指定することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 前記(2)(6)の基準情報ならびに前記(3)(7)のマッチ情報の位置を距離で指定することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 前記座標は前記情報の配列の次元に合せた次元座標であることを特徴とする請求項42記載の情報のパターンマッチの標準化方法。
- 前記座標は座標変換をすることが可能であることを特徴とする請求項42記載の情報のパターンマッチの標準化方法。
- 画像情報の前記(2)(3)(6)(7)のデータ値およびその範囲を色情報R、G、B、それぞれ独立して指定することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 前記(4)(8)のステップをマッチ実行する際、前記1つの問い合わせパターンの内、幾つかの前記マッチ情報がマッチしない場合(ミスマッチ)を許容することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 前記パターンマッチを分散処理して実施することを特徴とする請求項38又は39記載の情報のパターンマッチの標準化方法。
- 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1、第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
これを指定するパターンマッチのユーザインタフェースにおいて
(9)第4の入力として配列を指定する機能
(10)問い合わせパターンの設定機能は、
(10-1)パターンマッチの候補となる情報のデータ値を指定して基準情報とする機能、
(10-2)以上(10-1)の基準情報にマッチさせる複数のマッチ情報のそれぞれのデータ値をそれぞれ独立して指定するとともにこのそれぞれ情報の位置をそれぞれ独立して指定する機能
(11)以上(10-1)(10-2)の指定に基づきマッチ指令をする機能
(12)マッチ指令に基づき情報処理されたパターンマッチ結果を表示する機能
以上(9)から(12)のステップにより情報をパターンマッチ検出させること特徴とするパターンマッチ用ユーザインタフェース。 - 請求項1記載の集合演算機能を備えたメモリにおいて、
前記第1、第2の入力は、メモリに記憶された集合情報に対してパターンマッチングする問い合わせ情報パターンを指定するものであり、
これを指定するパターンマッチのユーザインタフェースにおいて
(9)配列を指定する機能
(10)問い合わせ情報パターンの設定機能は、
(10-1)パターンマッチの候補となる情報のデータ値とその範囲を指定して基準情報とする機能
(10-2)以上(10-1)の基準情報にマッチさせる複数のマッチ情報のそれぞれのデータ値とその範囲をそれぞれ独立して指定するとともにこのそれぞれ情報の位置とその範囲をそれぞれ独立して指定する機能
(11)以上(10-1)(10-2)の指定に基づきマッチ指令をする機能
(12)マッチ指令に基づき情報処理されたパターンマッチ結果を表示する機能
以上(9)から(12)のステップにより曖昧情報をパターンマッチ検出させることを特徴とするパターンマッチ用ユーザインタフェース。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2014508027A JP6014120B2 (ja) | 2012-03-28 | 2013-03-28 | 集合演算機能を備えたメモリ及びこれを用いた集合演算処理方法 |
US14/388,765 US20150154317A1 (en) | 2012-03-28 | 2013-03-28 | Memory provided with set operation function, and method for processing set operation processing using same |
Applications Claiming Priority (10)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2012-073451 | 2012-03-28 | ||
JP2012073451 | 2012-03-28 | ||
JP2012-083361 | 2012-03-31 | ||
JP2012083361 | 2012-03-31 | ||
JP2012101352 | 2012-04-26 | ||
JP2012-101352 | 2012-04-26 | ||
JP2012110145 | 2012-05-13 | ||
JP2012-110145 | 2012-05-13 | ||
JP2012121395 | 2012-05-28 | ||
JP2012-121395 | 2012-05-28 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2013147022A1 true WO2013147022A1 (ja) | 2013-10-03 |
Family
ID=49260269
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2013/059260 WO2013147022A1 (ja) | 2012-03-28 | 2013-03-28 | 集合演算機能を備えたメモリ及びこれを用いた集合演算処理方法 |
Country Status (3)
Country | Link |
---|---|
US (1) | US20150154317A1 (ja) |
JP (2) | JP6014120B2 (ja) |
WO (1) | WO2013147022A1 (ja) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015098707A1 (ja) * | 2013-12-23 | 2015-07-02 | 井上 克己 | 情報検索機能を備えたメモリ、その利用方法、装置、情報処理方法。 |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9558825B1 (en) * | 2014-06-25 | 2017-01-31 | Hrl Laboratories, Llc | System and method to discover and encode indirect associations in associative memory |
CN105389566B (zh) * | 2015-11-13 | 2018-09-11 | 广东欧珀移动通信有限公司 | 指纹识别方法、指纹模板的更新方法、装置和移动终端 |
JP6448696B2 (ja) * | 2017-03-22 | 2019-01-09 | 株式会社東芝 | 情報処理装置、方法及びプログラム |
JP7176478B2 (ja) * | 2019-06-14 | 2022-11-22 | トヨタ自動車株式会社 | 画像認識装置 |
CN114387124B (zh) * | 2021-12-22 | 2024-06-07 | 中核武汉核电运行技术股份有限公司 | 一种核电工业互联网平台的时序数据存储方法 |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003036269A (ja) * | 2001-07-23 | 2003-02-07 | Sony Corp | 情報処理装置および情報処理方法並びにこの情報処理のプログラムが記録された記録媒体 |
WO2011102432A1 (ja) * | 2010-02-18 | 2011-08-25 | Inoue Katsumi | 情報絞り込み検出機能を備えたメモリ、このメモリを用いた情報検出方法、このメモリを含む装置、情報の検出方法、メモリの使用方法、およびメモリアドレス比較回路 |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR950034265A (ko) * | 1994-02-10 | 1995-12-28 | 도자끼 시노부 | 연상메모리 |
JP3038649B2 (ja) * | 1997-04-16 | 2000-05-08 | 日本電気株式会社 | 連想検索装置および方法 |
JP2002260389A (ja) * | 2001-03-01 | 2002-09-13 | Kawasaki Microelectronics Kk | 連想メモリ |
US20080285652A1 (en) * | 2007-05-14 | 2008-11-20 | Horizon Semiconductors Ltd. | Apparatus and methods for optimization of image and motion picture memory access |
US7861030B2 (en) * | 2007-08-08 | 2010-12-28 | Microchip Technology Incorporated | Method and apparatus for updating data in ROM using a CAM |
US7643353B1 (en) * | 2007-10-25 | 2010-01-05 | Netlogic Microsystems, Inc. | Content addressable memory having programmable interconnect structure |
US7859878B2 (en) * | 2007-12-03 | 2010-12-28 | International Business Machines Corporation | Design structure for implementing matrix-based search capability in content addressable memory devices |
US7848128B2 (en) * | 2007-12-03 | 2010-12-07 | International Business Machines Corporation | Apparatus and method for implementing matrix-based search capability in content addressable memory devices |
JP4588114B1 (ja) * | 2010-02-18 | 2010-11-24 | 克己 井上 | 情報絞り込み検出機能を備えたメモリ、その使用方法、このメモリを含む装置。 |
-
2013
- 2013-03-28 JP JP2014508027A patent/JP6014120B2/ja active Active
- 2013-03-28 WO PCT/JP2013/059260 patent/WO2013147022A1/ja active Application Filing
- 2013-03-28 US US14/388,765 patent/US20150154317A1/en not_active Abandoned
-
2016
- 2016-09-23 JP JP2016184995A patent/JP2017084349A/ja active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2003036269A (ja) * | 2001-07-23 | 2003-02-07 | Sony Corp | 情報処理装置および情報処理方法並びにこの情報処理のプログラムが記録された記録媒体 |
WO2011102432A1 (ja) * | 2010-02-18 | 2011-08-25 | Inoue Katsumi | 情報絞り込み検出機能を備えたメモリ、このメモリを用いた情報検出方法、このメモリを含む装置、情報の検出方法、メモリの使用方法、およびメモリアドレス比較回路 |
Non-Patent Citations (2)
Title |
---|
DUC-HUNG LE ET AL.: "A Fully-Parallel Information Detection Hardware System Employing Content Addressable Memory", COMMUNICATIONS AND ELECTRONICS(ICCE), 2012 FOURTH INTERNATIONAL CONFERENCE, 3 August 2012 (2012-08-03), pages 447 - 452, XP032245571, DOI: doi:10.1109/CCE.2012.6315948 * |
DUC-HUNG LE ET AL.: "An FPGA-Based Information Detection Hardware System Employing Multi-Match Content Addressable Memory", IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS, COOMUNICATIONS AND COMPUTER SCIENCES, vol. E95-A, 1, 1 October 2012 (2012-10-01), pages 1708 - 1717, XP001578347, DOI: doi:10.1587/transfun.E95.A.1708 * |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2015098707A1 (ja) * | 2013-12-23 | 2015-07-02 | 井上 克己 | 情報検索機能を備えたメモリ、その利用方法、装置、情報処理方法。 |
JP5981666B2 (ja) * | 2013-12-23 | 2016-08-31 | 井上 克己 | 情報検索機能を備えたメモリ、その利用方法、装置、情報処理方法。 |
JP2017016668A (ja) * | 2013-12-23 | 2017-01-19 | 井上 克己 | 情報検索機能を備えたメモリ、その利用方法、装置、情報処理方法。 |
US9627065B2 (en) | 2013-12-23 | 2017-04-18 | Katsumi Inoue | Memory equipped with information retrieval function, method for using same, device, and information processing method |
Also Published As
Publication number | Publication date |
---|---|
JP2017084349A (ja) | 2017-05-18 |
US20150154317A1 (en) | 2015-06-04 |
JPWO2013147022A1 (ja) | 2015-12-14 |
JP6014120B2 (ja) | 2016-10-25 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP2017084349A (ja) | 集合演算機能を備えたメモリ及びこれを用いた集合演算処理方法 | |
Wang et al. | Deep sketch feature for cross-domain image retrieval | |
Das et al. | Automated Indian sign language recognition system by fusing deep and handcrafted feature | |
Karim et al. | Comparative analysis of feature extraction methods in satellite imagery | |
Nguyen et al. | Completed statistical adaptive patterns on three orthogonal planes for recognition of dynamic textures and scenes | |
Bilgin et al. | Road sign recognition system on Raspberry Pi | |
Singh et al. | Modified Haar-cascade model for face detection issues | |
Nachamai | Alphabet recognition of american sign language: a hand gesture recognition approach using sift algorithm | |
CN113468371A (zh) | 实现自然语句图像检索的方法、系统、装置、处理器及其计算机可读存储介质 | |
Vadakkot et al. | Automatic one-hand gesture (mudra) identification in bharatanatyam using eigenmudra projections and convolutional neural networks | |
Muthukumar et al. | Vision based hand gesture recognition for Indian sign languages using local binary patterns with support vector machine classifier | |
Mahmud et al. | Hand gesture recognition using SIFT features on depth image | |
Parra et al. | Automatic gang graffiti recognition and interpretation | |
Seth et al. | A review on content based image retrieval | |
Salau et al. | Image-based number sign recognition for ethiopian sign language using support vector machine | |
Verma et al. | Recent advancement of LBP techniques: A survey | |
Rayeed et al. | Bangla sign digits recognition using depth information | |
Farajzadeh | Exemplar-based logo and trademark recognition | |
Joseph et al. | Content based image retrieval system for Malayalam handwritten characters | |
Boukhriss et al. | Moving object classification in infrared and visible spectra | |
Jiménez et al. | Automatic method for Recognition of Colombian Sign Language for vowels and numbers from zero to five by using SVM and KNN | |
Jiang et al. | Fusing local and global information for scene classification | |
Poornima et al. | Recognition of Indian sign language alphanumeric gestures based on global features | |
Sawat et al. | Pixel encoding for unconstrained face detection | |
Bharathi et al. | Query-based word spotting in handwritten documents using HMM |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 13770013 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2014508027 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
WWE | Wipo information: entry into national phase |
Ref document number: 14388765 Country of ref document: US |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 13770013 Country of ref document: EP Kind code of ref document: A1 |