JP2004157754A - Pattern recognition device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a pattern recognition device which performs hierarchical processing with simple circuit configuration. <P>SOLUTION: This pattern recognition device is provided with: a plurality of detection processing means 1041, 1042... each for detecting one different characteristic with respect to the same input; a plurality of integration processing means 1051, 1052... for spatially integrating the detected plurality of characteristics for each processing result; a plurality of detection memories 1071, 1072... for storing the detection results; a plurality of integrated memories 1011 , 1012... for storing the integration processing results; a shared data line 1030 to which a predetermined detection processing means and the integrated memories are connected in a predetermined timing; and a plurality of local data lines 1061, 1062... to each of which the prescribed detection processing means, the integration processing means and the detection memories are connected. Then, the detection processing results are inputted from the plurality of detection memories to the integration processing means, and the integration processing results are inputted from the integrated memories to the plurality of detection processing means, and ferroelectric materials are used for the detection memories and/or integrated memories. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a circuit configuration of a pattern recognition device that performs pattern recognition, detection of a specific subject, and the like by parallel arithmetic processing such as a neural network.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, in the field of image recognition and voice recognition, a type that sequentially calculates and executes a recognition processing algorithm specialized for a specific recognition target as computer software, or a dedicated parallel image processing processor (SIMD, MIMD machine, etc.). It is roughly classified into the type of execution depending on the hardware used.
[0003]
For example, with respect to a dedicated parallel processing processor, the object identification device disclosed in Japanese Patent Laid-Open No. 6-6793 prepares a plurality of image processing processor units, performs arithmetic processing by a DSP mounted on these processor units, Is transferred to another unit to perform object identification. For example, an image is divided into a plurality of regions, processing of each region is performed in parallel by each processor, and inference of object identification is performed by another processor unit using a neural network or fuzzy control.
[0004]
Further, as a hardware for performing hierarchical parallel processing using a neural network, a hierarchical neural network disclosed in Japanese Patent No. 2679730 discloses a hierarchical structure that enables a single layer of hardware to be multi-layered using time division multiplexing. Structural neural network architecture, formed by connecting multiple neuron models to each other, with the aim of enabling a single layer of hardware to be equivalently multilayered using time division multiplexing. In a neural network, a time-division multiplexed analog signal is integrated with a digital weight data from the outside, a product is integrated by adding the product in a time-division manner through a capacitor, and the A single-layer unit is installed by installing multiple neuron model units that can output A single-layer unit assembling means for forming a set, a feedback means for returning an output of the single-layer unit assembling means to an input portion of the same single-layer unit assembling, and Control means for performing time-division multiplexing of the analog signal and further performing control for time-division multiplexing use of the single-layer unit assembling means via the feedback means. It is configured to form a hierarchically structured neural net equivalently by multiple use.
[0005]
In addition, as hardware using an FPGA (Field Programming Gate Array), there is a processor introduced in US Pat. No. 5,892,962. This processor holds a memory in each FPGA, holds a processing result in the FPGA in a memory, reads a result of the memory, and performs processing.
[0006]
[Problems to be solved by the invention]
In the object identification device disclosed in Japanese Patent Laid-Open No. 6-6793, which has been described as a conventional example of hardware for executing a recognition processing algorithm, an area assigned to each image processing processor unit is further divided. It is possible to perform several stages of processing from a small area to the original area, but to perform hierarchical processing, such as performing parallel processing on multiple results obtained by processing and further processing on multiple other processor units Could not. Also, the result of the processing could not be read. Furthermore, the results of each area could not be spatially integrated.
[0007]
Also, in the hierarchical neural network disclosed in Japanese Patent No. 2679730, there is no means for arbitrarily variably controlling the interlayer coupling, so that the types of processing that can be substantially realized are extremely limited. was there.
[0008]
Further, in the FPGA-BASED PROCESSOR disclosed in US Pat. No. 5,892,962, complicated wiring is required to read an intermediate result held in a memory.
[0009]
Therefore, an object of the present invention is to provide a hierarchical circuit in which a plurality of results obtained by processing by a plurality of processing units are processed in parallel by a plurality of processing units with a simple circuit configuration without using complicated wiring. An object of the present invention is to provide a pattern recognition device capable of performing processing, spatially integrating processing results of each processing, and easily reading the results of each processing.
[0010]
[Means for Solving the Problems]
Therefore, in order to realize the above object, according to the present invention, a pattern recognition device for detecting a predetermined pattern included in an input signal is provided with a plurality of patterns for detecting one different characteristic for the same input. Detection processing means, a plurality of integration processing means for spatially integrating features detected by the plurality of detection processing means for each processing result, and a plurality of detection means for holding processing results of the detection processing means. A memory, a plurality of integrated memories for holding processing results of the integrated processing means, a predetermined detection processing means and a shared data line to which the predetermined integrated memory is connected at a predetermined timing; Means, a plurality of local data lines connected to the integrated processing means and the detection memory, and the integrated processing means is provided with the detection processing means stored in the plurality of detection memories. Read and input the processing result, and read the processing result of the integrated processing means held in the integrated memory, and operate to input the processing result to the plurality of detection processing means. The memory is a memory configured from an element using a ferroelectric material.
[0011]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0012]
(First Embodiment)
FIG. 1 is a diagram showing the configuration of the present embodiment.
[0013]
1, reference numeral 1000 denotes a control means, 1010 denotes an integrated memory controller, 1011 to 1014 denotes an integrated memory, 1015 denotes an auxiliary memory, 1020 denotes an integrated address line, 1021 to 1024 denotes an integrated memory control signal, and 1025 denotes an integrated memory control signal. An auxiliary memory control signal, 1030 is a global data line, 1041A to 1044C are detection processing means, 1051A to 1054C are integration processing means, 1061 to 1064 are local data lines, 1071 to 1074 are detection memories, and 1080 is a detection memory. Reference numerals 1081 to 1084 denote detection memory control signals, 1090 denotes a detection memory controller, 1111A to 1114C denote detection processing unit control signals, and 1121A to 1124C denote integrated processing unit control signals. The detection processing unit control signal and the integration processing unit control signal are not shown in the figure, but only the detection processing unit control signals 1111A to 1111C are shown. Reference numerals 1130 and 1161 to 1164 denote external I / Fs.
[0014]
Hereinafter, the function of each component in FIG. 1 will be described.
[0015]
The control unit 1000 controls the entire circuit, and communicates with and uses the integrated memory controller 1010, the detection processing units 1041A to 1044C, the integrated processing units 1051A to 1054C, and the detection memory controller 1090 to be described later. The recognition operation is performed by controlling each of these units based on the recognition algorithm.
[0016]
The unified memory controller 1010 controls the memories of the unified memories 1011 to 1014 and the auxiliary memory 1015, outputs data from these memories to the global data line 1030, and outputs data on the global data line 1030 to the unified memory 1011 to 1014. Write to 1014 or auxiliary memory 1015. More specifically, an integrated memory control signal 1021 to 1024 such as a chip select signal for outputting an address to the integrated address line 1020 and selecting a memory to be operated, a write enable signal for distinguishing between writing and reading, and an auxiliary memory control signal The above operation is performed by controlling 1025. When a plurality of addresses are generated, for example, the start / end address and the number of steps are set in the integrated memory controller 1010 by the control means 1000, and the counter in which the start address value is set is counted up to the last address by the number of steps. It can respond by doing. In other words, by changing the generation of the address in various ways, data can be read and written in an arbitrary area of the integrated memories 1011 to 1014 and the auxiliary memory 1015, so that a wiring problem can be avoided.
[0017]
The integrated memories 1011 to 1014 are memories that hold processing results of integrated processing units 1051A to 1054C described later. These are connected to the global data line 1030, respectively, and output the results held based on the integrated memory control signals 1021 to 1024 to the global data line 1030, and the integrated processing units 1051A to 1051A to 105 The processing result of 1054C is fetched.
[0018]
The auxiliary memory 1015 is a memory that temporarily stores signals to be recognized such as images. This memory is also connected to the integrated address line 1020 and the global data line 1030, and outputs a signal held based on the auxiliary memory control signal 1025 from the integrated memory controller 1010 to the global data line 1030, The data on 1030 is taken.
[0019]
Signals indicating the addresses of the integrated memories 1011 to 1014 and the auxiliary memory 1015 are output from the integrated memory controller 1010 to the integrated address line 1020. By changing this address, it is possible to avoid the wiring problem and easily read out the results of the respective processes held in the integrated memories 1011 to 1014, and combine the results in the respective detection processing units 1041A to 1044C. Processing can be performed on signals.
[0020]
The unified memory control signals 1021 to 1024 are signals for selecting the unified memories 1011 to 1014 and for discriminating and controlling writing and reading. By selecting the memory in a time-division manner, the data of the integrated memories 1011 to 1014 can be output to the global data line 1030 in a time-division manner at the time of reading, and the integrated processing means control signals 1121A to 1124C at the time of writing. , The processing result of each integrated processing unit output on the global data line 1030 can be held in each of the integrated memories 1011 to 1014.
[0021]
The auxiliary memory control signal 1025 is a signal for performing selection of the auxiliary memory 1015 and distinction / control of writing / reading.
[0022]
The global data line 1030 is connected to the integrated memories 1011 to 1014, the detection processing units 1041A to 1044C, the integrated processing units 1051A to 1054C, and the auxiliary memory 1015. Therefore, the data of the integrated memories 1011 to 1014 are input in parallel to the respective detection processing units 1041A to 1044C, and the processing results from the integrated processing units 1051A to 1054C are stored in the respective integrated memories 1011 to 1014 by time division processing. Written in divisions.
[0023]
Each of the detection processing units 1041A to 1044C is in charge of each processing (for example, edge detection and the like) necessary for the recognition processing. That is, in the apparatus of the present invention, the recognition processing is performed as a whole by combining the processing of the respective detection processing means.
[0024]
The detection processing units 1041A to 1044C process data input via the global data line 1030, and output the results to the local data lines 1061 to 1064, respectively. The processing results of the detection processing units 1041A, 1041B and 1041C are output to the local data line 1061, and the processing results of the detection processing units 1042A to 1042C are output to the local data line 1062. ing. Hereinafter, the same applies to other detection processing means.
[0025]
The same signal is input to the detection processing units 1041A to 1044C via the global data line 1030. Each detection processing means performs different processing on the input signal. If the result is, for example, the detection processing means 1041A, the result is output to the local data line 1061 and the result is stored in the detection memory 1071. Similarly, the processing results of the respective detection processing units are stored in different detection memories 1071 to 1074. Here, which detection processing unit operates is indicated by detection processing unit control signals 1111A to 1114C from the control unit 1000. The detection processing unit control signal 1111A is connected to the detection processing unit 1041A, the detection processing unit control signal 1111B is connected to the detection processing unit 1041B, and so on. (Note that FIG. 1 shows only the detection processing unit control signals 1111A to 1111C.)
For example, at a certain timing, the detection processing unit control signals 1111A, 1112A, 1113A, and 1114A are enabled, and as a result, the detection processing units 1041A, 1042A, 1043A, and 1044A operate, and the processing results of the respective detection processing units are output. The data is held in the detection memories 1071 to 1074 via the local data lines 1061 to 1064. At another timing, another detection processing means control signal is enabled.
[0026]
The integration processing units 1051A to 1054C perform integration processing on the data input via the local data lines 1061 to 1064, respectively, and output the result to the global data line 1030. The input to the integration processing means 1051A to 1051C is performed from the local data line 1061, 1052A to 1052C from 1062, 1053A to 1053C from 1063, and 1054A to 1054C from 1064. Further, the integrated processing means 1051A to 1054C to be operated are selected by enabling the processing means select signal of the integrated processing means control signals 1121A to 1124C (not shown in FIG. 1) from the control means 1000. The output signal of the integrated processing means control signals 1121A to 1124C controls the timing at which the processing results of the integrated processing means 1051A to 1054C are output to the global data line 1030.
[0027]
That is, for example, at a certain timing, the processing unit select signals of the integrated processing unit control signals 1121A, 1122A, 1123A, and 1124A are enabled, and as a result, the integrated processing units 1051A, 1052A, 1053A, and 1054A operate, and the output is performed. The processing result is sequentially output to the global data line 1030 from, for example, the integration processing unit 1051A based on the reset signal. At that time, by controlling the integrated memory controller 1010 at the same timing, data on the global data line 1030 can be held in order from the integrated memory 1011.
[0028]
The local data line 1061 is connected to the detection memory 1071, detection processing units 1041A to 1041C, and integration processing units 1051A to 1051C. The local data line 1062 is connected to the detection memory 1072, the detection processing units 1042A to 1042C, and the integration processing units 1052A to 1052C. The same applies to the local data lines 1063 and 1064. Therefore, the processing results from the detection processing units 1041A to 1041C are stored in the detection memory 1071, the processing results from the detection processing units 1042A to 1042C are stored in the detection memory 1072, and the processing results from the detection processing units 1043A to 1043C are stored in the detection memory 1073. Processing results from the detection processing units 1044A to 1044C are stored in the detection memory 1074. The data of the detection memory 1071 is input to the integration processing means 1051A to 1051C, the data of the detection memory 1072 is input to the integration processing means 1052A to 1052C, and the data of each detection memory is input to the separate integration processing means in parallel. ing.
[0029]
The detection memory 1071 is a memory that holds processing results from the detection processing units 1041A to 1041C, and the detection memory 1072 is a memory that holds processing results from the detection processing units 1041A to 1041C. The same applies to the detection memory 1073 and the detection memory 1074. The detection memory 1071 is connected to the local data line 1061, 1072 is connected to 1062, 1073 is connected to 1063, and 1074 is connected to 1064, and the results held based on the detection memory control signals 1081 to 1084 are stored in the respective local data lines. The data is output to the local data lines 1061 to 1064 and the processing results of the detection processing units 1041A to 1044C on the local data lines 1061 to 1064 are captured.
[0030]
Signals indicating the addresses of the detection memories 1071 to 1074 are output from the detection memory controller 1090 to the detection address line 1080. By changing this address, the wiring problem is avoided, the result of each process at an arbitrary position held in the detection memories 1071 to 1074 is easily read out, and the integrated processing means 1051A to 1054C process the result for each region. Can be performed.
[0031]
The detection memory control signals 1081 to 1084 are signals for selecting the detection memories 1071 to 1074 and for discriminating and controlling write / read.
[0032]
The detection memory controller 1090 controls the memories of the detection memories 1071 to 1074, outputs data from these memories to the local data lines 1061 to 1064, and writes data on the local data lines 1061 to 1064 to the memory. Specifically, by outputting an address to the detection address line 1080 and controlling integrated memory control signals 1081 to 1084 such as a chip select signal for selecting a memory to be operated and a write enable signal for distinguishing between writing and reading. Perform the above operation.
[0033]
The detection processing unit control signals 1111A to 1114C are used for communication between the detection processing units 1041A to 1044C and the control unit 1000. Among the detection processing means 1041A to 1044C, a processing means select signal for selecting a processing means to be operated, an output signal indicating permission to output a processing result to the local data lines 1061 to 1064, and each of the detection processing means 1041A to 1044C Is composed of an end signal indicating the end of the processing at step (1).
[0034]
The integrated processing means control signals 1121A to 1124C are used for communication between the integrated processing means 1051A to 1054C and the control means 1000. Among the integrated processing units 1051A to 1054C, a processing unit select signal for selecting an operating processing unit, an output signal indicating permission to output a processing result to the global data line 1030, and an output signal for each integrated processing unit 1051A to 1054C. It is composed of an end signal indicating the end of the processing.
[0035]
The external I / Fs 1130, 1161, 1162, 1163, and 1164 are connected to the global data line 1030 and the local data lines 1061, 1062, 1063, and 1064, respectively. The processing results of the processing units 1051A to 1054C and the processing results of the detection processing units 1041A to 1044C are taken out during operation of the processing units, or the intermediate processing results held in the integrated memories 1011 to 1014 and the detection memories 1071 to 1074 are taken out. Can be done.
[0036]
Next, the operation of the configuration shown in FIG. 1 will be described for a case where a neural network for performing image recognition is formed by parallel hierarchical processing. First, the processing content of the neural network will be described in detail with reference to FIG. This neural network hierarchically handles information related to recognition (detection) of an object or a geometric feature in a local region in input data, and its basic structure is a so-called Convolutional network structure (LeCun, Y. and Bengio, Y., 1995, "Convolutional Networks for Images Speech, and Time Series" in Handbook of Brain Theory, New York, NY, Nepal. The output from the last layer (top layer) is the category of the recognized target as the recognition result and the position information on the input data.
[0037]
The data input layer 101 is a layer for inputting local area data from a photoelectric conversion element such as a CMOS sensor or a CCD element. The first feature detection layer 102 (1, 0) includes local low-order features (geometric features such as specific directional components and specific spatial frequency components, as well as color components) of the image pattern input from the data input layer 101. May be included in a local area around each position on the entire screen (or a local area centered on each predetermined sampling point over the entire screen) at a plurality of scale levels or resolutions at the same location. The number of feature categories is detected.
[0038]
The feature integration layer 103 (2,0) has a predetermined receptive field structure (hereinafter, the receptive field indicates a coupling range with the output element of the immediately preceding layer, and the receptive field structure indicates a distribution of the coupling load). It performs integration (operations such as sub-sampling by local averaging, maximum output detection, etc.) of a plurality of neuron element outputs in the same receptive field from the feature detection layer 102 (1, 0). This integration process has a role of allowing positional displacement, deformation, and the like by spatially blurring the output from the feature detection layer 102 (1, 0). Each receptive field of the neurons in the feature integration layer has a common structure among neurons in the same layer.
[0039]
The subsequent feature detection layers 102 ((1, 1), (1, 2),..., (1, M)) and the feature integration layers 103 ((2, 1), (2, 2) ),..., (2, M)), the former ((1, 1),...) Detects a plurality of different features in each feature detection module and performs the latter ( (2, 1),...) Integrates detection results for a plurality of features from the preceding feature detection layer. However, the former feature detection layer is connected (wired) so as to receive the cell element output of the former feature integration layer. The sub-sampling, which is a process performed in the feature integration layer, is for averaging the output from a local region (local receptive field of the feature integration layer neuron) from the feature detection cell population of the same feature category. .
[0040]
Further, an operation in a process of detecting an eye from an input image will be described as a specific example with reference to FIGS.
[0041]
FIG. 3 is a flowchart for detecting an eye from an input image. In step S301, an image is input to the auxiliary memory 1015. This corresponds to the data input layer 101. Subsequently, in step S302, a primary feature amount is detected. The primary feature amounts in the eye detection are, for example, those shown in FIG. That is, features in a specific direction such as vertical (4-1-1), horizontal (4-1-2), diagonally upward (4-1-3), and diagonally downward (4-1-4) are extracted.
[0042]
Note that, as described above, the secondary feature amounts are a right empty V-shaped (4-2-1), a left empty V-shaped (4-2-2), a circle (4-2-3), and 3 The next feature amount is the eye (4-3-1). The detection processing units 1041A to 1044C are configured to detect these respective characteristic amounts, and the detection processing unit 1041A sets the vertical (4-1-1) of the primary characteristic amount, and 1042A sets the horizontal (4-1-1). 1-2), 1043A detects an upward slant (4-1-3), and 1044A detects a downward slant (4-1-4). Similarly, the detection processing means 1041B has a right empty V-shape (4-2-1) of the secondary feature amount, 1042B has a left empty V-shape (4-2-2), and 1043B has a circle (4-2-3). ) Is detected. The detection processing unit 1041C is configured to detect the eye (4-3-1).
[0043]
In the eye detection used in this example, the primary feature amount is four, the secondary feature amount is three, and the tertiary feature amount is one. Therefore, the detection processing units 1041A to 1044A and 1041B to 1043B , 1041C, but not 1044B and 1042C-1044C.
[0044]
The primary feature amount detection in step S302 corresponds to the feature detection layer 102 (1, 0), and each detection processing unit corresponds to the feature f detection module 104. The unified memory controller 1010 controls the auxiliary memory control signal 1025 to read local data centering on a certain position of the image from the auxiliary memory 1015 (this local area corresponds to the receptive field 105), and to the global data line 1030 Output. Then, they are input in parallel to the detection processing units 1041A to 1044A, respectively, and detect the respective primary feature amounts described above.
[0045]
At this time, only the processing means select signals 1111A to 1114A of the detection processing means control signals are enabled. Then, upon seeing the end signal indicating the end of the processing, the control unit 1000 enables the output signal, and the detection processing units 1041A to 1044A output the processing results to the local data lines 1061 to 1064. At the same time, the detection memory controller 1090 outputs an address to the detection address line 1080 and controls the detection memory control signals 1081 to 1084 to hold the data on each local data line in the detection memories 1071 to 1074.
[0046]
In the processing by each detection processing means, the input data and the connection weight are used. For example, when detecting the vertical (4-1-1) of the primary feature amount, the size of the receptive field is determined. Conceptually, the receptive field structure shown in FIG. 5 (hereinafter, “receptive field” indicates a coupling range between the output element of the immediately preceding layer and “receptive field structure”) is 3 * 3 and the coupling weight is 0 or 1. Means the distribution of the connection weight). In this step S302, by changing the center point of the local area read from the auxiliary memory 1015 to each point of the entire screen or each point of a predetermined sampling point over the entire screen, the primary feature amount is changed in the entire screen. Is detected. As described above, the process of moving the local region and performing the process over the entire screen is the same in the subsequent integration process and the detection of the secondary and tertiary feature amounts.
[0047]
Subsequently, in step S303, the primary feature amounts are integrated. This corresponds to the feature integration layer 103 (2, 0) and integrates data held in the detection memories 1071 to 1074 (a plurality of neuron elements in the same receptive field from the feature detection layer 102 (1, 0)). Output integration (equivalent to sub-sampling by local averaging, maximum output detection, etc.)). Each integration processing means corresponds to the integration module 106 of the feature f. The detection memory controller 1090 outputs an address on the detection address line 1080, controls the detection memory control signals 1081 to 1084, reads local data of the detection memories 1071 to 1074, and outputs the local data via the local data lines 1061 to 1064. , Each local data is input to the integration processing means 1051A to 1054A.
[0048]
In the detection process in step S302, the data input to the detection processing units 1041A to 1044A are the same, but in the integration process in step S303, the data input to the integration processing units 1051A to 1054A are: Each is different. However, since the position and the size of the receptive field of the integration processing in the input image are common to all the integration processing means 1051A to 1054A, the detection memory controller indicating the position of the data in each of the detection memories 1071 to 1074. The address from 1090 can be the same.
[0049]
That is, when local data is read from the detection memories 1071 to 1074, different addresses are not output to each of the detection memories but data is read from each of the detection memories in parallel with one address output. Also, the integration processing is performed in parallel by each of the integration processing means 1051A to 1054A. As described above, each integration processing means performs processing such as averaging of input data and detection of the maximum value.
[0050]
At this time, only the processing means select signals 1121A to 1124A of the integrated processing means control signals are enabled. Then, upon seeing the end signal indicating the end of the processing, the control means 1000 sequentially enables the output signal, and the integrated processing means 1051A to 1054A output the processing result to the global data line 1030 in a time-division manner.
[0051]
At the same time, the unified memory controller 1010 outputs an address to the unified address line 1020 and controls the unified memory control signals 1021 to 1024 to sequentially hold the data on the global data lines in the unified memories 1011 to 1014. The output of the integrated processing means 1051A can be held in the integrated memory 1011 by matching the enable timing of the output signals of the integrated processing means control signals 1121A to 1124A and the enable timing of the memory select signal of the integrated memory control signal, The output of 1052A can be held at 1012, the output of 1053A can be held at 1013, and the output of 1054A can be held at 1014.
[0052]
In the steps so far, the integrated memory 1011 holds a result obtained by integrating the results of detecting the vertical direction of the primary feature amount, 1012 holds a result obtained by integrating the results of detecting the horizontal direction, and 1013 denotes an obliquely upward right The result obtained by integrating the results of the detection of the directions is held, and reference numeral 1014 holds the result of the integration of the results of the detection of the diagonally right-down direction.
[0053]
In step S304, secondary feature amount detection is performed. This corresponds to the feature detection layer 102 (1, 1). The secondary feature amounts here are a V-shape (4-2-1, 4-2-2) and a circle (4-2-3) as shown in FIG. 4, and the V-shape is within the receptive field. Detection of two primary features in two oblique directions (4-1-3, 4-1-4) and their positional relationship are possible, and a circle indicates detection of all primary features in the receptive field. And its positional relationship.
[0054]
That is, a secondary feature amount can be detected by combining a plurality of types of primary feature amounts. The detection processing of these secondary feature amounts is performed by the detection processing units 1041B to 1043B. The unified memory controller 1010 outputs an address to the unified address line 1020 and controls the unified memory control signals 1021 to 1024 so that the unified memory 1011 to 1014 outputs the integrated local feature amount stored in the unified primary feature amount. The data is read and output to the global data line 1030.
[0055]
At this time, by sequentially changing the enable of the memory select signals of the integrated memory control signals 1021 to 1024, the output of the integrated primary feature amount is sequentially performed from the integrated memories 1011 to 1014. That is, the global data line 1030 is used in a time-sharing manner.
[0056]
As in step 302, these data are input in parallel to the detection processing units 1041B to 1043B, respectively, to detect the above-described secondary feature amounts. At this time, since there are three types of secondary feature amounts to be detected, only the processing means select signals of the detection processing means control signals 1111B to 1113B are enabled.
[0057]
Then, upon seeing the end signal indicating the end of the processing, the control unit 1000 enables the output signal, and the detection processing units 1041B to 1043B output the processing results to the local data lines 1061 to 1063. At the same time, the detection memory controller 1090 outputs an address to the detection address line 1080 and controls the detection memory control signals 1081 to 1083 to hold the data on each local data line in the detection memories 1071 to 1073.
[0058]
Subsequently, in step S305, the secondary feature amounts are integrated. This corresponds to the feature integration layer 103 (2, 1) and integrates the data held in the detection memories 1071 to 1073. The detection memory controller 1090 outputs an address on the detection address line 1080, controls the detection memory control signals 1081 to 1083, reads local data of the detection memories 1071 to 1073, and outputs the local data via the local data lines 1061 to 1063. , Each local data is input to the integration processing means 1051B to 1053B. As in step S303, each integration processing means performs processing such as averaging of input data and detection of the maximum value.
[0059]
At this time, only the processing means select signals 1121B to 1123B of the integrated processing means control signals are enabled. Then, upon seeing the end signal indicating the end of the processing, the control unit 1000 sequentially enables the output signal, and the integrated processing units 1051B to 1053B output the processing result to the global data line 1030 in a time-division manner. At the same time, the integrated memory controller 1010 outputs an address to the integrated address line 1020 and controls the integrated memory control signals 1021 to 1023 to hold data on the global data lines in the integrated memories 1011 to 1013.
[0060]
In step S306, tertiary feature amount detection is performed. This corresponds to the feature detection layer 102 (1, 2). The tertiary feature amount here is the eye (4-3-1) as shown in FIG. 4, and for that purpose, all the secondary feature amounts (V-shaped (4-2-1) in the receptive field are used. , 4-2-2) and the circle (4-2-3)) and their positional relationships may be seen. That is, a tertiary feature amount can be detected by combining a plurality of types of secondary feature amounts. The detection processing of these tertiary feature values is performed by the detection processing unit 1041C.
[0061]
The unified memory controller 1010 outputs an address to the unified address line 1020 and controls the unified memory control signals 1021 to 1023 to output the unified secondary feature amount stored in the unified memory 1011 to 1013 from the unified memory 1011 to 1013. The local data is read and output to the global data line 1030. At this time, similarly to step S304, by sequentially changing the enable of the memory select signals of the integrated memory control signals 1021 to 1023, the output of the integrated secondary feature amount is sequentially performed from the integrated memories 1011 to 1013. , The global data line 1030 is used in a time-sharing manner. These data are input to the detection processing unit 1041C, and the above-described tertiary feature amount is detected.
[0062]
At this time, since there is only one type of tertiary feature to be detected, only the processing means select signal 1111C of the detection processing means control signal is enabled. Then, upon seeing the end signal indicating the end of the processing, the control unit 1000 enables the output signal, and the detection processing unit 1041C outputs the processing result to the local data line 1061. At the same time, the detection memory controller 1090 outputs an address to the detection address line 1080 and controls the detection memory control signal 1081 to hold the data on the local data line in the detection memory 1071.
[0063]
Subsequently, in step S307, the tertiary feature amounts are integrated. This corresponds to the feature integration layer 103 (2, 2) and integrates data held in the detection memory 1071. The detection memory controller 1090 outputs an address on the detection address line 1080, controls the detection memory control signal 1081, reads out local data of the detection memory 1071, and outputs the tertiary feature amount via the local data line 1061. The local data is input to the integration processing means 1051C.
[0064]
The integration processing means performs processing such as averaging of input data and detection of the maximum value. Then, upon seeing the end signal indicating the end of the processing, the control means 1000 sequentially enables the output signal, and the integrated processing means 1051C outputs the processing result to the global data line 1030. At the same time, the integrated memory controller 1010 outputs an address to the integrated address line 1020 and controls the integrated memory control signal 1021 to hold data on the global data line in the integrated memory 1011.
[0065]
Then, the result held in the integrated memory 1011 is the final result of eye detection. Note that the result stored in the detection memory 1071 may be used as the eye detection result without performing step S307.
[0066]
As described above, according to the present embodiment, the detection process of a certain feature and the integration process of the detection result can be easily performed in parallel when performing with a plurality of features, and the processes can be performed hierarchically. It is also easy to do. Also, by storing each detection result and integration result in temporary memory, and then specifying the address of that memory and outputting the result to the data line and inputting it to each processor, even in the processing of complex receptive field structures, It is possible to prevent the wiring from becoming complicated. Furthermore, it is also possible to read out the results of each of these processes, and it is also possible to read out the results at any position held in the memory by addressing.
[0067]
The processing in the detection processing means and the integration processing means of the present embodiment can be performed by digital processing using a DSP or the like, or can be performed by analog processing in which an analog circuit converts a potential, a pulse width, and the like. Further, the detection memory and the integrated memory may be digital memories such as DRAM and SRAM, or analog memories such as FRAM.
[0068]
In other words, the processing in the detection processing means and the integrated processing means may be performed by digital processing, and the detection memory and the integrated memory may be configured by a digital memory, so that an all-digital configuration is possible. Further, the processing in the detection processing means and the integrated processing means may be performed by analog processing, and the detection memory and the integrated memory may be configured by analog memories, thereby forming an all-analog configuration. Further, the processing in the detection processing means and the integration processing means is constituted by analog processing, the detection memory and the integrated memory are constituted by digital memories, and the processing means and the A / D conversion means and the D / A conversion means between the memories are processed by the processing means. It is also possible to adopt a configuration in which the processing result obtained in step (1) is A / D converted and stored in the memory, and the result stored in the memory is D / A converted and used by the processing means.
[0069]
FIG. 6 shows an example in which the processing in the detection processing means and the integration processing means is configured by analog processing, and the detection memory and the integrated memory are configured by analog memory. In this example, a ferroelectric capacitor is used as an analog memory, and the value used in the processing means is represented by a pulse width. Therefore, a PWM conversion means for changing the amount of potential of the analog memory to a pulse width is also provided.
[0070]
6, reference numeral 6000 denotes a control means, 6010 denotes an integrated memory controller, 6211 denotes an integrated sub memory, 6020 denotes an integrated address line, 6021 denotes an integrated memory control signal, 6030 denotes a global data line, and 6041 denotes a detection processing. 6051, a local data line, 6071, a detection sub memory, 6080, a detection address line, 6081, a detection memory control signal, 6090, a detection memory controller, and 6111, a detection processing unit. A means control signal, 6121 is an integrated processing means control signal, 6220 is an integrated address decoder, 6230 is an integrated result PWM conversion means, 6240 is a detection result PWM conversion means, and 6250 is a detected address decoder.
[0071]
The configuration diagram of FIG. 6 is a part of the configuration diagram of FIG. 1, and only one part of the integration processing means, detection processing means, integrated memory, detection memory, and the like is illustrated. One integrated memory 6011 includes a plurality of integrated sub memories 6211,..., And one detection memory 6071 includes a plurality of detected sub memories 6271,.
[0072]
The operation of this circuit is basically the same as the operation of FIG. 1 described above. In FIG. 6, however, the integrated sub-memory 6211 is selected by the integrated address decoder 6220 based on the address of the integrated address line 6020 and the signal of the integrated memory controller 6010, and the detection sub-memory 6271 is selected by the detected address decoder 6250. Is performed based on the address of the detection address line 6080 and the signal of the detection memory controller 6090.
[0073]
Further, the integration result PWM conversion means 6230 converts the integration result held in the integration sub memory 6211 into a PWM signal. Further, the detection result PWM conversion means 6240 converts the detection result held in the detection sub-memory 6271 into a PWM signal. Note that PWM (Pulse Width Modulation) is a modulation method in which information is given to the width of a pulse waveform, and has digital characteristics resistant to noise (binary information of High level and Low level in the voltage direction) and It has both analog characteristics (continuous information in the time direction) that can represent continuous information with one pulse.
[0074]
FIG. 7 is a diagram showing the configuration of the integrated sub-memory 6211 and the integrated result PWM conversion means 6230. 7, 7001 is a ferroelectric capacitor, 7002 and 7003 are MOS transistors, 7004 is a comparator, and 7005 is a selector. When writing an analog value to the ferroelectric capacitor 7001, the integrated address decoder 6220 turns on the MOS transistor 7002 based on the address of the integrated address line 6020 and the signal of the integrated memory controller 6010, and the potential corresponding to the integrated result becomes strong. The value is applied to the dielectric capacitor 7001 and held.
[0075]
When reading the value, the integrated address decoder 6220 turns on the MOS transistor 7003 based on the address of the integrated address line 6020 and the signal of the integrated memory controller 6010, and the comparator 7004 causes the potential of the ferroelectric capacitor 7001 and the reference The signals are compared, and a High signal is output to generate a PWM signal until the reference signal becomes higher. Then, the PWM signal is output to the sub global data line constituting the global data line 6030 selected by the selector 7005.
[0076]
FIG. 8 is a diagram showing the global data line 6030. As described above, the global data line includes a plurality of sub global data lines 6031, 6032,. This is because the detection processing unit 6041 described below needs to input the values of a plurality of integrated sub memories to the detection processing unit 6041 in parallel in order to process the whole or part of the receptive field in parallel. . For this purpose, the integrated address decoder 6220 operates when the integrated sub-memory 6211 is in the receptive field (or a part of the receptive field) with respect to the center address of the receptive field indicated by the address of the integrated address line 6020. I have.
[0077]
In this way, for one address of the integrated address line 6020, the values of the plurality of integrated sub memories are read in parallel, and the read values are input to the detection processing means 6041 in parallel. Use multiple sub-global data lines. Although not shown in FIG. 6, since there are a plurality of detection processing units, each of which is connected to the global data line 6030, the values read in parallel are input to the detection processing unit 6041 in parallel. At the same time, it is also input to other detection processing means, and another feature is detected in parallel using the same value in the integrated sub-memory.
[0078]
FIG. 9 is a diagram showing the configuration of the detection processing means 6041. .., A2,..., A capacitor 9001, a sample-and-hold circuit 9002, and a MOS transistor 9003 as a switch. .. Have a PMOS transistor M1 functioning as a constant current source and a PMOS transistor M2 functioning as a switch. Similarly, the multiplication circuits B1,... Have an NMOS transistor M3 functioning as a constant current source and an NMOS transistor M4 functioning as a switch.
[0079]
.. Perform multiplication with the output value of the preceding neuron when the synapse load value is positive, and the multiplier circuits B1,. Performs multiplication with the output value of the neuron. At this time, the synapse load values input to the multiplication circuits B1,... Are input as absolute values. And the multiplication circuits B1,... Are paired, and the PWM signal from the global data line 6030 is commonly input to the transistors M2 and M4. At this time, in the transistors M1 and M3 operating as constant current sources, a gate voltage is applied such that one of the MOS transistors having a different sign of the weight coefficient does not operate.
[0080]
, B1,... Realize multiplication of the multiplied value P and the multiplied value Q: (P × Q). Note that P and Q satisfy the following conditional expressions.
[0081]
P ≧ 0, Q ≧ 0
Here, the multiplied value P and the multiplied value Q are the PWM signals created from the values of the integrated sub-memory as described above in order to realize the multiplication in the multiplication circuits A1,..., B1,. Vin and an analog voltage Vw, respectively.
[0082]
The PWM signal Vin converted from the multiplied value is input to the input terminal P, and the analog voltage Vw converted from the multiplied value is input to the input terminal Q.
[0083]
The voltage of the multiplication value Vw input to the input terminal Q is applied to the gate terminal of the transistor M1. The PWM signal Vin is inverted and input from the input terminal P to the gate terminal of the transistor M2.
[0084]
The PWM signal has a low level set to 0 V and a high level set to the power supply voltage Vdd (3.3 V in this embodiment). Here, when the PWM signal is at the High level, that is, when the power supply voltage Vdd is applied to the source terminal of the transistor M1, the analog voltage Vw is set to an appropriate voltage range so that the transistor M1 operates in the saturation region. , PWM signal is High level, transistor M1 can be operated as a constant current source.
[0085]
At this time, the amount of current flowing through the transistor M1 is determined by the gate-source voltage, that is, (Vdd-Vw). At this time, the pulse width of the PWM signal Vin is converted so as to be proportional to the multiplied value P. The analog voltage Vw is converted so that the amount of current determined by (Vdd-Vw) is proportional to the multiplication value Q.
[0086]
Therefore, the transistor M1 allows the current determined by the above (Vdd-Vw) to flow only while the PWM signal is at the High level, so that the amount of charge flowing through the transistor M1 is proportional to P × Q. Although the above description has been given by taking the example of the multiplying circuits A1,... Using the PMOS, the operation is similarly the same in the multiplying circuits B1,. You can think.
[0087]
Next, how the calculation results of the multiplication circuits A1,..., B1,.
[0088]
First, an operation in which the operation results of the multiplication circuits A1,... Are held in the capacitor 9001 will be described. Since a plurality of multiplying circuits A1,... Performing independent calculations are connected to the bus, respectively, the calculation results executed in parallel by the respective multiplying circuits A1,. Are stored in the capacitor 9001 through the bus and added.
[0089]
Further, the operation in which the operation results of the multiplication circuits B1,... Are held in the capacitor 9001 is the operation result executed in parallel by the multiplication circuits B1,. This is an operation of being extracted from the accumulated charge. That is, since the product-sum operation result with respect to the negative weight coefficient becomes negative, the subtraction is realized by extracting the charge held in the capacitor 9001. Note that subtraction is not performed in a state where no charge is held in the capacitor 9001. Therefore, a predetermined amount of charge is stored in the capacitor 9001 in advance. The predetermined amount may be, for example, an amount corresponding to the absolute value of the minimum value of the sum of products of the expected output value of the neuron at the previous stage and the synapse load. In this case, when the final charge amount (potential) of the capacitor 9001 is the predetermined charge amount (potential), the product-sum operation result becomes zero.
[0090]
As described above, the total charge amount stored in the capacitor 9001 indicates the total value of the operation results of the multiple multiplier circuits A1,..., B1,. ing.
[0091]
When the product-sum operation described above is completed, the control means 6000 turns on the MOS transistor 9003 of the switch, outputs the potential of the capacitor 9001 to the local data line 6061 via the sample-and-hold circuit 9002, and 6271.
[0092]
Regarding subsequent integration processing, the detection sub-memory 6271 may have the same configuration as the integration sub-memory 6211, the local data line 6061 may have the same configuration as the global data line 6030, and the integration processing unit 6051 may have the same configuration as the detection processing unit 6041. .
[0093]
In the above description, the configuration using ferroelectric capacitors for the integrated sub-memory 6211 and the detection sub-memory 6271 has been described, but other configurations are also conceivable. For example, FIG. 10 shows a configuration of an integrated sub-memory 6211 using a transistor type ferroelectric memory (FeRAM). 1001 is a transistor type ferroelectric memory, 1002 and 1003 are MOS transistors, 1004 is a comparator, and 1005 is a selector.
[0094]
When writing an analog value to the transistor type ferroelectric memory 1001, the integrated address decoder 6220 turns on the MOS transistor 1002 based on the address of the integrated address line 6020 and the signal of the integrated memory controller 6010. Then, the transistor-type ferroelectric memory 1001 is selected by a word line, and a value corresponding to the integration result is applied to the source line, thereby retaining the value.
[0095]
When reading the value, the transistor type ferroelectric memory 1001 is selected by a word line, and the integrated address decoder 6220 turns on the MOS transistor 1003 based on the address of the integrated address line 6020 and the signal of the integrated memory controller 6010. Then, the capacitor 1006 is charged with the drain current, the comparator 1004 compares the potential of the capacitor 1006 with the reference signal, and outputs a High signal until the potential of the capacitor becomes higher to generate a PWM signal. Then, the PWM signal is output to the sub-global data line constituting the global data line 6030 selected by the selector 1005.
[0096]
As described above, when an analog memory using a ferroelectric material is used, an A / D converter and a D / A converter are not required as compared with using a digital memory. This has the effect of reducing power consumption.
[0097]
In the configuration shown in FIG. 6, the integrated sub memory 6211 and the detection sub memory 6271 can be used as digital memories. In this case, the PWM signal created by the integration result PWM conversion means 6230 and the detection result PWM conversion means 6240 is a signal indicating the presence or absence of a feature.
[0098]
Next, a case where focusing on a specific subject, color correction of the specific subject, and exposure control are performed by mounting the pattern detection (recognition) device according to the configuration of the present embodiment on the imaging device will be described with reference to FIG. I do. FIG. 11 is a diagram illustrating a configuration of an example in which the pattern detection (recognition) device according to the embodiment is used for an imaging device.
[0099]
In FIG. 11, an imaging device 5101 includes an imaging optical system 5102 including a photographing lens and a drive control mechanism for zoom photographing, a CCD or CMOS image sensor 5103, an imaging parameter measurement unit 5104, a video signal processing circuit 5105, a storage unit 5106, A control signal generating unit 5107 for generating a control signal for controlling an imaging operation, controlling an imaging condition, etc .; a display display 5108 also serving as a finder such as an EVF; a strobe light emitting unit 5109; a recording medium 5110; The detection device is provided as a subject detection (recognition) device 5111.
[0100]
The imaging apparatus 5101 performs detection (detection of the position and size) of a face image of a person registered in advance from, for example, a captured video by using a subject detection (recognition) apparatus 5111. Then, when the position and size information of the person is input from the subject detection (recognition) device 5111 to the control signal generation unit 5107, the control signal generation unit 5107 outputs the person based on the output from the imaging parameter measurement unit 5104. , A control signal for optimally performing focus control, exposure condition control, white balance control, and the like.
[0101]
By using the above-described pattern detection (recognition) device in the imaging device in this way, it becomes possible to perform optimal detection control and shooting based on the person detection.
[0102]
Hereinafter, features of the present invention according to the above embodiment will be summarized.
[0103]
Features 1.
In a pattern recognition device that detects a predetermined pattern included in an input signal,
A plurality of detection processing means for detecting one different feature from the same input,
A plurality of integrated processing means for spatially integrating the features detected by the plurality of detection processing means for each processing result;
A plurality of detection memories for holding processing results of the detection processing means;
A plurality of integrated memories holding processing results of the integrated processing means,
A shared data line to which the predetermined detection processing means and the predetermined integrated memory are connected at a predetermined timing;
A plurality of local data lines each connected to the predetermined detection processing means, the integration processing means, and the detection memory,
The integrated processing means, read and input the processing results of the detection processing means held in a plurality of the detection memory,
Read the processing result of the integrated processing means held in the integrated memory, operates to input to the plurality of detection processing means,
A pattern recognition device, wherein the detection memory and / or the integrated memory is a memory configured from an element using a ferroelectric material.
[0104]
Features2.
The pattern recognition device according to Feature 1, wherein the detection memory and / or the integrated memory includes a ferroelectric capacitor.
[0105]
Features 3.
The pattern recognition device according to Feature 1, wherein the detection memory and / or the integrated memory is configured by a transistor-type ferroelectric memory.
[0106]
Features 4.
A plurality of holding units; an address line indicating an address corresponding to the plurality of holding units; a plurality of processing units for processing a signal based on a value held by the plurality of holding units; the plurality of holding units; In a pattern recognition device having a data line connected to a plurality of processing means,
For one address of the address line, a value held in a part or all of the plurality of holding units is read out in parallel, and a signal based on the value is transmitted through the data line. The same signal is input in parallel to the processing means to perform pattern recognition processing, and the same signal is input in parallel to some or all of the plurality of processing means connected to the data line. A pattern recognition device, wherein another pattern recognition process is performed in parallel.
[0107]
【The invention's effect】
As described above, according to the present invention, by using a shared data line and a local data line, and a memory, a simple circuit configuration without using complicated wiring can be performed by a plurality of processing units. There is an effect that hierarchical processing such as parallel processing by a plurality of processing means can be performed in parallel with respect to the plurality of obtained results.
[0108]
In the detection processing, parallel operation is possible by inputting the same data to a plurality of predetermined detection processing means, and when reading a value from the integrated memory, a plurality of values are assigned to one address. The processing time can be reduced by reading in parallel and inputting it to the detection processing means in parallel.
[0109]
Furthermore, in the integration processing for spatially integrating the processing results of each detection processing, an address indicating a position in each memory is shared by a plurality of detection memories each holding a result of a different feature detection. In this case, the integration processing can be performed in parallel. Further, there is an effect that these detection / integration processing means can be repeated any number of times.
[0110]
Further, by using an element using a ferroelectric material for a memory, there is an effect that a circuit area and power consumption can be reduced.
[Brief description of the drawings]
FIG. 1 is a diagram showing a configuration of a first embodiment.
FIG. 2 is a diagram illustrating a Convolutional network structure.
FIG. 3 is a flowchart illustrating the operation of the first embodiment.
FIG. 4 is a diagram showing a feature value;
FIG. 5 is a diagram illustrating an example of feature amount detection.
FIG. 6 is a diagram showing a configuration when analog processing means and an analog memory are used.
FIG. 7 is a diagram showing a configuration of an analog memory.
FIG. 8 is a diagram showing a configuration of a global data line.
FIG. 9 is a diagram illustrating a configuration of a detection processing unit.
FIG. 10 shows a configuration of an analog memory.
FIG. 11 is a diagram illustrating a configuration of an example in which the pattern recognition device according to the embodiment is used for an imaging device.

Claims (1)

In a pattern recognition device that detects a predetermined pattern included in an input signal,
A plurality of detection processing means for detecting one different feature from the same input,
A plurality of integrated processing means for spatially integrating the features detected by the plurality of detection processing means for each processing result;
A plurality of detection memories for holding processing results of the detection processing means;
A plurality of integrated memories holding processing results of the integrated processing means,
A shared data line to which the predetermined detection processing means and the predetermined integrated memory are connected at a predetermined timing;
A plurality of local data lines each connected to the predetermined detection processing means, the integration processing means, and the detection memory,
The integrated processing means, read and input the processing results of the detection processing means held in a plurality of the detection memory,
Read the processing result of the integrated processing means held in the integrated memory, operates to input to the plurality of detection processing means,
A pattern recognition device, wherein the detection memory and / or the integrated memory is a memory configured from an element using a ferroelectric material.

Images