KR20110037183A - Power controllable computer system combining neuro-fuzzy system and parallel processor, method and apparatus for recognizing objects using the computer system in images - Google Patents

Abstract

PURPOSE: A computer system which can control power combined to a neuro-fuzzy system and a parallel processing processor are provided to perform only in a processor which needs a parallel process of limited data among input data applied to a neuro network technique and a fuzzy technique. CONSTITUTION: A neuro-fuzzy system(110) includes at least two among a neural network block(111), a fuzzy logic block(112) and a neuro-fuzzy block(113). A parallel processor(120) includes a plurality of processing units(121). A network on chip(130) is connected between the neuro-fuzzy system and the parallel processing processor. The network on chip performs data communication among the neuro-fuzzy system, the parallel processing processor and a power supply unit(160).

Description

POWER CONTROLLABLE COMPUTER SYSTEM COMBINING NEURO-FUZZY SYSTEM AND PARALLEL PROCESSOR, METHOD AND APPARATUS FOR RECOGNIZING OBJECTS USING THE COMPUTER SYSTEM IN IMAGES}

The present invention relates to a parallel processing computer system and a technique for recognizing an object in an image using the same.

Object recognition is a key technology in recent advanced vision applications such as autonomous vehicles, intelligent robot vision systems, and alarm systems. When 2-D image data is given as input, the object recognition Finding the feature point and generating a feature vector (descriptor vector), and compares with the object database which is a vector set of the object registered in advance to determine the nearest object.

In order to perform the object recognition, it is important to extract feature points from the input image and how to describe them as vectors. To this end, a large amount of image data, such as filtering and histogram, require complex and various operations. The most commonly used parallel processing is scale invariant feature transform (SIFT), which requires a large amount of computation, making it difficult to process in real time such that even a 2GHz high-performance CPU can perform as much as 0.5 frames per second.

In order to overcome this problem, when performing object recognition using a parallel processor including a large number of processing units, the frame rate can be improved due to the parallel processing, but in this case, power consumption increases in proportion to the number of processing units. There's a downside to doing that. It will be difficult to apply to low-power real-time applications such as mobile robots and cell phones.

An object of the present invention is to perform parallel processing only for processors requiring limited data among input data by applying neural network technology and fuzzy technology.

In addition, the present invention increases the object recognition processing speed by extracting a region-of-interest (ROI) from an input image and performing parallel processing only on processors requiring only data of the extracted region of interest. It aims to recognize an object in real time by reducing its power consumption.

The computer system according to the present invention comprises at least two of neural networks blocks, fuzzy logic blocks, and neuro-fuzzy blocks in which neural networks and fuzzy logic are combined. A neuro-fuzzy system, a parallel processor including a plurality of processing units, a power supply for controlling the power supplied to the parallel processor according to a control signal of the neuro-fuzzy system, and a neuro A network on chip coupled between the purge system and the parallel processor for data communication between the neuro-fuge system, the parallel processor and the power supply.

It is preferable to further include a memory for storing the data extracted from the neuro-fuzzy system and the intermediate data of the operation process of the parallel processor.

The neuro-fuzzy system further includes a task scheduler, and the task scheduler preferably generates a schedule for distributing output data of the neuro-fuzzy system to the parallel processing processor.

The plurality of processing units are divided into one independent power domain for each specific number, and the power supply includes a plurality of power regulators, each of which controls each of the power domains according to a control signal of a neuro-purge system. It is desirable to.

An object recognition apparatus according to the present invention includes a neuro-neural network including a cellular neural networks visual attention engine, a fuzzy motion estimator, a neuro-fuzzy classifier and a task scheduler. A fuzzy system, a parallel processing processor including a plurality of processing units, a power supply for controlling the power supplied to the parallel processing processor according to a control signal of a neuro-purge system, and a feature vector are compared with vectors in a database. An object determination unit and a neuro-fuge system that recognizes an object corresponding to the vector having the closest distance, and a network on chip for data communication between the parallel processing processor, the object determination unit and the power supply unit, and the fuzzy motion measuring unit is a continuous Dynamic motion vectors between captured imageframes and The concentrator extracts the static features such as intensity, color, and direction and accumulates them together with the dynamic motion vector to generate a feature map, and the neuro-fuzzy classifier extracts seed points based on the feature map and based on the seed points. The region-of-interest (ROI) of each object is extracted in tile units by determining homogeneity through region expansion, and the task scheduler converts the ROI tile into a ROI tile task and converts the transformed ROI. The tile task is distributed to the parallel processor, and the parallel processor performs a single-instruction-multiple-data (SIMD) parallel operation on the tile region of interest to generate a feature vector of the feature point and the feature point of the object, and then asks the feature vector. Deliver to the signing government.

The ROI tile task is a starting address representing the data of the leftmost point of the ROI tile extracted from the entire image data, an X-direction coordinate that is a two-dimensional coordinate value of the leftmost point of the ROI tile of the whole image, and Y. And a tile size describing the width and height of the region of interest tile, the starting coordinates composed of the direction coordinates, and the task scheduler preferably distributes and manages the transformed region of interest tile task to a plurality of processing units of the parallel processor. .

The neuro-fuzzy system preferably predicts the amount of computation to be processed by the parallel processor based on the number of ROI tiles extracted by the neuro-fuzzy classifier.

The plurality of processing units are divided into one independent power domain for each specific number, and the power supply includes a plurality of power regulators, each of which controls each of the power domains according to a control signal of a neuro-purge system. Do.

In the object recognition apparatus according to the present invention, an amount of computation prediction method to be processed by a parallel processing processor includes a first step of measuring the number of ROI tiles extracted by a neuro-fuzzy classifier, comparing the number of measured tiles with the distribution reference number A second step of selecting a number of power domains, a third step of supplying power to only the selected power domains, and a distribution criterion of the second step according to an operation result of the parallel processor included in the powered power domains A fourth step of updating the count.

In the object recognition apparatus according to the present invention, an independent power domain number determining method including one or more processing units includes a first step of displaying a graph in which the number of power domains is represented by the X axis and the power reduction effect according to the number of power domains is represented by the Y axis; Cross point where the slope of the second and first graphs decreases while the slope of the second and first graphs is reduced, indicating that the X-axis is the number of power domains and the additional area required for the power regulator according to the number of power domains is the Y-axis. Determining a number of power domains.

In the object recognition method using the object recognition apparatus according to the present invention, a first step of extracting a region of interest of an object in units of tiles by a neuro-fuzzy system and a starting address of the corresponding tile data in the entire image data In the second step of converting a two-dimensional coordinate value of the starting position of the tile and the size of the tile in the entire image into a region of interest tile task, the neuro-fuzzy system predicts the amount of computation that the parallel processing processor must handle A third step of transmitting a control signal to the supply device, a fourth step of each power regulator controlling each power domain according to the control signal, and a converted region of interest tile task having a plurality of processing units arranged in parallel through a network on chip The fifth step distributed to the processing processor, each processing unit of the parallel processing processor distributed interest Against Tiles task and a seventh step of recognizing the object corresponding to a vector having the shortest distance as compared with the vector in the sixth step and the object determining additional databases for generating a feature point of an object and a feature vector.

In the first step, the neuro-fuzzy system may generate a dynamic motion vector, extract a static feature, and then accumulate the dynamic motion vector and the static feature together to classify the ROI and express the ROI using a basic tile having a constant size. .

According to the present invention, a neural network technique and a fuzzy technique may be applied to perform parallel processing only for a processor requiring limited data among input data.

In addition, according to the present invention, by extracting the region of interest from the input image and performing parallel processing only for the processor that needs only the data of the extracted region of interest, the object recognition processing speed is increased, and the power consumption of the object recognition apparatus is reduced, thereby real-time. Can recognize objects.

Computer systems

1 is a block diagram of a computer system 100 according to an embodiment of the present invention. Computer system 100 includes a neuro-fuzzy system 110, a parallel processor 120, a network on chip 130, a memory 140, and a power supply 160.

The neuro-fuzzy system 110 refers to a system including at least two of neural network block 111, fuzzy logic block 112, and neuro-fuzzy block 113 in which neural network and fuzzy logic are combined.

The neuro-fuzzy system 110 is a system that implements a human learning ability (neural network) and an approximate thinking process (fuzzy logic). The function of the neural network and the fuzzy logic should be combined. To this end, the first case may be composed of only the neuro-fuzzy blocks 113, but this is not the only case. In the second case, the neural network block 111 and the fuzzy logic block 112 may implement the above functions. In the third case, the neuro-fuzzy block 113 and the neural network block 111 may be combined. As described above, the same function may be implemented. In a fourth case, the neuro-fuzzy block 113 and the fuzzy logic block 112 may be combined. In addition, in the fifth case, the combination of the neuro-fuzzy block 113, the neural network block 111, and the fuzzy logic block 112 can implement the above functions. It is defined as a system that includes all cases.

The neural network block 111 is used in a wide range of fields such as data mining, network management, machine vision, signal processing, etc. The existing computer model is excellent in computational computation by continuous instructions, but learning ability to generalize and learn phenomena through experience. Based on this lack of fact, in order to simulate the learning ability of humans, a computer-based algorithm that models the neural network of the human brain is applied.

The algorithm modeling the neural network of the human brain is composed of basic units designed to model neural network activity of the brain. The above unit combines the data inputs into one output, which is called the activation function of the unit. The activation function is divided into two parts. The first part is to consolidate all the inputs into one value. Each input to a unit has its own weight. The most common combination function is a weighted sum, where each input is multiplied by a weight and these are added together. Other combination functions are also sometimes useful and include weighted input maximum, minimum, or logical AND, OR values. The second part of the activation function is the transfer function, given the name given to the fact that the value is transferred from the coupling function to the output of the unit. The previous functions include the sigmoid, linear, and hyperbolic tangent functions.

Fuzzy logic block 112 is widely used in many intelligent control applications, such as plant control, motor control, and airplane control, where fuzzy logic is derived from binary logic that an analysis subject belongs to or does not belong to, and that each analysis subject is gathered. The degree of belonging is expressed mathematically by the membership function. A block implemented with fuzzy logic that handles ambiguous values through such a membership function is called fuzzy logic block 112. Fuzzy logic block 112 simulates the approximate and non-numerical process of thinking in the human brain. Fuzzy logic block 112 may be implemented using various membership functions known for the purposes of computer system 100.

The neuro-fuzzy block 113 combines the neural network technology and the fuzzy logic technology to simulate the inference process occurring in the human brain by combining the learning function of the neural network and the features of fuzzy logic dealing with ambiguous values.

The parallel processor 120 includes a plurality of processing units 121 and calculates the data transmitted from the network on chip 130 in parallel.

The network on chip 130 is connected between the neuro-fuzzy system 110 and the parallel processor 120. The network on chip 130 transmits the data collected or extracted by the neuro-fuzzy system 110 to the parallel processor 120, and the power control signal of the neuro-fuzzy system 110 is transmitted to the power controller 160. do. It also enables communication of data or signals between the neuro-fuzzy system 110, the parallel processor 120, the power supply 160, and the memory 140.

The memory 140 stores data extracted from the neuro-fuzzy system 110 and intermediate data of a calculation process of the parallel processor 120.

The power supply device 160 controls the power supplied to the parallel processor 120 according to the control signal transmitted from the neuro-fuzzy system 110 to the network on chip 130. That is, power is supplied only to the parallel processor 120 that requires data operation, and power supply to the unnecessary parallel processor 120 is cut off. Therefore, the present invention can efficiently control the power consumed by the parallel processing processor 120 according to the amount of processed data, so that the computer system 100 can be driven in real time even when processing massive data.

In addition, the computer system 100 to which the neuro-fuzzy technology is applied does not process or compute all the data inputted for a certain purpose, but the neuro-purge only a certain amount of data necessary to achieve the purpose of the computer system 100. Technology selects or aggregates, and computer system 100 processes or computes only the selected or aggregated data. As a result, processing speed, power consumption, etc. are improved even if the same purpose is achieved compared to the conventional computer system without neuro-purge technology. In addition, even when processing massive data, the computer system 100 to which the neuro-fuzzy technology is applied may be driven in real time.

In addition, the neuro-fuzzy system 110 included in the computer system 100 shown in FIG. 1 may further include a task scheduler (not shown), and the task scheduler may transmit data output from the neuro fuzzy system to the parallel processing processor. Distribute efficiently For example, the task scheduler can efficiently allocate several data to the processing unit 121 for high processing speed, and allocate new data as each processing unit 121 finishes each data operation.

Object recognition device

An object recognition apparatus and an object recognition method among various embodiments using the computer system 100 of FIG. 1 will be described with reference to the accompanying drawings.

Object recognition is an object database that is a vector set of pre-registered objects by finding the feature points of an object in the image and generating a descriptor vector for it when 2-D image data is given as input. It is the process of determining the nearest object compared to the (object database). In order to perform such object recognition, feature points should be extracted from the input image and described as vectors. This requires complex and various operations on a large amount of image data. Currently, the most commonly used parallel processing operation is scale invariant feature transform (SIFT), but it requires a large amount of computation, and it is difficult to process in real time such as 0.5 frames per second even with a 2GHz high-performance CPU.

Application of the computer system 100 proposed in the present invention to the object recognition apparatus not only reduces the amount of data that the parallel processing processor 120 needs to process, but also converts the power consumed by the parallel processing processor into the amount of processed data. According to the present invention, an object recognition requiring a large amount of computation can be performed at a lower power than in the prior art, and an object recognition can be performed in real time by improving processing speed.

2 is a block diagram of an object recognition apparatus according to an embodiment of the present invention.

The object recognition apparatus includes a neuro-fuzzy system 210, a parallel processor 220, a network on chip 230, a memory 240, an object determiner 250, and a power supply 260.

The neuro-fuzzy system 210 includes a neural network block, a fuzzy logic block, a neuro-fuzzy block in which the neural network and fuzzy logic are combined, and a task scheduler 217.

The neuro-fuzzy system 210 receives the entire image 215, extracts the regions of interest 211, 212, and 213 of the object, and then extracts the extracted regions of interest 211, 212, and 213 from the two-dimensional tile task 214. ) And output the converted tile task 214. For example, in the example image 215 of 640x480 pixels, the ROIs 211, 212, and 213 of each object may be extracted by the neuro-fuzzy system 210 in 40 × 40 basic tile units 216.

In addition, the parallel processor 220 estimates the amount of processing to be processed and transmits a control signal for supplying power only to the specific processing units 221 to the power supply device 230 through the network house 230.

The extracted region of interest tile is converted into a 12-byte tile task 214 composed of a start address, a start coordinate, and a tile size of each region of interest base tile 216 by the task scheduler 217, thereby processing 16 processing units ( 221 is assigned to each processing unit 221 of the parallel processing processor 220.

The parallel processor 220 includes a plurality of, for example, sixteen processing units 221 and performs SIMD parallel operation on the tile task 214 of the extracted region of interest tiles 211, 212, 213 through SIMD parallel operation. Generate feature points and feature vectors.

The network on chip 230 is connected between the neuro-fuzzy system 210 and the parallel processor 220, and transmits the ROI tile task 214 or a control signal which is an output of the neuro-fuzzy system 210 to the parallel processor. To 220. In addition, data communication is performed with the neuro-fuzzy system 210, the parallel processor 220, the memory 240, the object determiner 250, and the power supply 260.

The memory 240 stores the entire input image 215 and stores data of the neuro-fuzzy system 210, the object determiner 250, and the processing units 221.

The object determiner 250 determines the type of the object by finding and voting the closest vector from a database previously created for the target object based on the feature point and the feature vector generated by the parallel processing processor 220. If there are several objects, the matching process is repeatedly performed to determine the types of the objects for the various objects.

According to an embodiment of the present invention, the parallel processing processor 220 converts a tile task in which tiles of the regions of interest 211, 212, and 213 extracted from the neuro-fuge system 210, instead of the entire input image 215, are converted. Perform SIMD operation only for 214. The power supply 260 also supplies power only to some of the processing units 221 of the parallel processor 220 according to the number of ROI tiles output from the neuro-fuzzy system. By doing so, the object recognition device proposed in the present invention can recognize an object with low power as compared with the prior art, and the processing speed can be improved to recognize an object in real time.

FIG. 3A is a diagram illustrating a configuration required for the neuro-fuzzy system 210 to extract the ROIs 211, 212, and 213 of an object in the object recognition apparatus illustrated in FIG. 2.

The neuro-fuzzy system 210 includes a fuzzy motion estimator 310, a cellular neural networks visual attention engine 320, a neuro-fuzzy classifier 330, a task. The scheduler 340 and the controller 350 are included.

The fuzzy motion measurer 310 of FIG. 3A uses fuzzy logic to find a motion vector for two temporally consecutive image frames. The motion vector of each point is found through the correspondence problem of finding which point in one frame is which point in another frame. Using full search to find all image areas in this process requires too much computation, so it is necessary to reduce the search area using fuzzy inference. Looking at the operation process in detail, the pixels to be compared are first extracted from the input image frame, and the difference values are accumulated and stored in the memory device, and then the values are fuzzy through the membership function. By determining the similarity of the two image pixels according to the distribution of the fuzzy values, the size of the motion vector is determined.

The cellular neural network visual concentrator 320 of FIG. 3A implements a visual attention phenomenon occurring in the visual cortex of the human brain in hardware using a neural network-based visual focus algorithm. It efficiently performs static feature extraction of intensity, color, direction, etc. from the entire input image using a 2-D connected neural network, and the motion vector and cellular neural network generated by the fuzzy motion measurer 310 A intensity map is accumulated by accumulating all the intensity, color, and directions extracted from the visual concentrator 320. FIG. 3B is a diagram illustrating the cellular neural network visual concentrator 320. Referring to FIG. Each cell 321 is implemented using a cellular neural network 322 consisting of a 2-D array. Each cell 321 of the 2-D array is mapped to one pixel of the input image to perform feature extraction of intensity, color, and direction. This increases the efficiency of feature extraction of the input image. The cellular neural network 322 performs an operation such as image filtering using a one-to-one relationship between the arrangement of each cell 321 and the pixels of the input image. This speeds up processing time such as image filtering and reduces the power consumption of the cellular neural network concentrator 320. Each cell 321 includes a memory device 323 for storing image information, a shift register 324 for exchanging data with neighboring cells, and an arithmetic device 325 for computing cell data. Each cell 321 communicates with neighboring cells 321 through the control unit 350 to perform unique image processing.

Returning back to FIG. 3A, the neuro-fuzzy classifier 330 combines the neural network and fuzzy logic to each object in the input image 215 shown in FIG. 2 based on the feature map obtained in the cellular neural network visual concentrator 320. Serves to extract the regions of interest 211, 212, and 213.

3C is a block diagram illustrating a neuro-fuzzy classifier 330. The neuro-fuzzy classifier 330 determines whether any pixel enters the region of interest 211, 212, 213 of any object, through homogeneity criteria, and the cellular neural network visual concentrator 320. After extracting the seed point, which is the most characteristic part, from the feature map generated in, the area of interest of each object is derived by gradually expanding the area through classification of neighboring pixels. In this process, fuzzy logic is used to measure the similarity between the region of interest of the object and the target pixel for homogeneity testing. Intensity, saliency, distance from the seed point, etc. are considered as objects of the similarity measurement. Fuzzy logic is used to measure similarity between humans and similarity. The similarity (μ1 to μn) is measured through a Gaussian fuzzy logic 331 as shown in Equation (1).

… (1)

… (One)

The measured similarities μ1 to μn are multiplied by the weights ω1 to ωn as in Equation (2) through the neural network 332, and the multiplied values are summed in the summation unit 334. Thereafter, the homogeneity determination unit 335 determines whether the combined values, as shown in Equation (3), are homogeneous pixels finally included in the corresponding area by comparing with the threshold value b.

… (2)

… (2)

… (3)

… (3)

After the determination, the weights ω1 to ωn of each connection are changed by learning as shown in Equation (4) through the weight update path 333 included in the neural network 332.

… (4)

… (4)

FIG. 3D illustrates a circuit 360 and a result waveform 370 in which a Gaussian function, which is a representative example of the fuzzy membership function used for similarity measurement, is implemented as a CMOS device. As shown in FIG. 3D (b), the output Iout 362 of this circuit 360 is output to the Gaussian function 370. Changing the value of the Xseed 361, which is the input of the circuit, may move the center point 371 of the resulting waveform 370. When the target value 373 is input to another input, Xtarget 363, the Gaussian function value 372 according to the input value 373 is output from the output terminal Iout 362.

In order to accurately implement the Gaussian function 370 digitally, more than 1000 instructions need to be performed, but when the analog circuit 360 is implemented, the Gaussian function 370 is less than 100 ns with only a few MOSFET elements M1 to M11. ) Can be implemented.

Referring back to FIG. 3A, the task scheduler 217 is a region of interest tile 211, 212, 213 extracted by the fuzzy motion measurer 310, the cellular neural network visual concentrator 320, and the neuro-fuzzy classifier 330. Is transformed into the region of interest tile task 214 and the region of interest tile task 214 is assigned to the plurality of processing units 221 in parallel, the parallel processing processor 220 simultaneously executes tiles of several regions of interest ( 211, 212, and 213. In assigning the tile task 214 to the plurality of processing units 221, the task scheduler 340 generates a task scheduling table to process which region of interest task 214 is processed in which processing unit 221 and which processing unit. 221 manages the availability. For example, task scheduler 340 assigns several ROI tile tasks 214 to as many processing units 221 as possible for high processing rates, and each processing unit 221 completes new tasks as they complete their individual tasks. It is possible to assign the region of interest tile task 214. However, this may be one scheduling method, and various scheduling methods are possible by the task scheduler. In addition, the size of the region of interest to be allocated is also not constant, it is possible to set the tile size as a multiple of the base tile size arbitrarily.

The controller 350 controls the operations of the fuzzy motion measurer 310, the cellular neural network visual concentrator 320, the neural-fuzzy classifier 330, and the task scheduler 217.

4 illustrates the format of a 12 byte region of interest tile task 214. The tile task 214 is composed of a 32-bit start address 411, a 32-bit start coordinate 412, and a 32-bit tile size 413.

The 32-bit starting address 411 means the address of the data of the leftmost point 415 of the ROI tiles 211, 212, and 213 among the entire image 215 data stored in the memory. The 32-bit start coordinate 412 refers to the two-dimensional coordinate values (X, Y) of the upper left point 415 of the ROI tiles 211, 212, and 213 in the image 215. The start coordinates 412 are further divided into 16-bit X-direction coordinates and Y-direction coordinates, respectively. The tile size 413 describes the size of the tile 414 based on the start coordinates 412, and the width W and the height H of the tile are described by 16 bits. If only the above three pieces of information (411, 412, 413) and the data of the entire image 215 width, the processing unit 221 of Figure 2 can know the location and size of the image 414 tile that it has to process This can be downloaded from the memory 240 and processed. Since the width of the entire image is a value that does not change, it may be set in each processing unit 221 at the program start stage.

FIG. 5A illustrates a parallel processing processor 220 of the object recognition apparatus described with reference to FIG. 2.

The parallel processing processor 220 includes a plurality of, for example, 16 processing units PE 1 to PE 16, 221, wherein the 16 processing units PE 1 to PE 16 each have one independent power every four. It is divided into domains. That is, the first to fourth processing units PE 1 to PE 4 share the first power domain 222, and the fifth to eighth processing units PE 5 to PE 8 share a second power domain (not shown). ), The ninth through twelfth processing units PE 9 through PE 12 share the third power domain (not shown), and the thirteenth through sixteenth processing units PE 13 through PE 16 share the fourth power. Share domain 223.

FIG. 5B is a diagram illustrating the power supply device 260 of the object recognition apparatus described with reference to FIG. 2.

Power supply 260 includes a plurality of, for example, four power regulators 261-264, each power regulator 261-264 having a respective power domain 222, 223,... ) And one-to-one. That is, the first power regulator 261 controls the power of the first power domain 222 of FIG. 5A, and the second power regulator 262 controls the power of the second power domain (not shown) of FIG. 5A. The third power regulator 263 controls the power of the third power domain (not shown) of FIG. 5A, and the fourth power regulator 264 controls the power of the fourth power domain 223 of FIG. 5A.

By separating the power domains 222, 223,..., The power is supplied only to the necessary parallel processing processor 220 when the amount of computation (interest region task) that the parallel processing processor 220 needs to process is not large. Power supply to the parallel processor 220 is cut off. That is, the neuro-fuzzy system 210 predicts the amount of computation to be processed by the parallel processor 220 and transmits a control signal to the power supply 260 to supply power only to the required power domains 222, 223,... do. Therefore, according to an embodiment of the present invention, it is possible to recognize an object in low power and real time.

How to estimate how much processing a parallel processor should handle

FIG. 6 is a flowchart illustrating an operation amount prediction method to be processed by a parallel processor executed by a neuro-fuzzy system of an object recognition apparatus according to an embodiment of the present invention.

First, the neuro-fuzzy system measures the number of ROI tiles extracted by the neuro-fuzzy classifier (S601).

Thereafter, a specific number of power domains is selected by comparing the measured number of tiles with the distribution reference number (S602). That is, the first power domain is selected when the number of ROI tiles is the smallest, and when the next step is selected, the first and second power domains are selected. Is selected when the first, second and third power domains are selected, and when the number of ROI tiles is the largest, all of the first to Nth power domains are selected.

Thereafter, the power supply supplies power only to the selected power domain (S603).

Thereafter, the distribution reference number of the step S602 is updated according to the calculation result of the parallel processor included in the power domain to which power is supplied (S604).

How to determine the number of power domains

FIG. 7 is a diagram illustrating a method for determining the number of power domains in the object recognition apparatus shown in FIG. 2 according to an embodiment of the present invention.

As the number of processing units 221 included in each of the power domains 222, 223,..., That is, the greater the number of power domains 222, 223,..., The power reduction effect 720 of the object recognition apparatus increases. . On the other hand, the number of power regulators 261 to 264, ... required increases with the number of power domains 222, 223, ..., and the area required for the power regulators 261 to 264, ... increases. However, in the case of the power reduction effect 720, the increasing slope decreases as the number of power domains 222, 223, ... increases. In the case of the additional area cost 730, the slope increases as the number of power domains 222, 223, ... increases. Therefore, the number of power domains 222, 223,... Is determined by comparing the cost 730 and the power reduction effect 720 according to the additional area.

First, a graph 740 is drawn with the power domain number 710 as the X axis and the power reduction effect 720 according to the number of power domains as the Y axis. Thereafter, a graph 750 is plotted on the X-axis as the number of power domains 710 and Y-axis as an additional area cost 730 required for the power regulators 261 to 264 required for the number of power domains. Thereafter, the slope of the power reduction effect graph 740 is decreased, and the number of power domains of the intersection 760 at which the slope of the additional area cost graph 750 increases is determined.

For example, in FIG. 7, when the power domains 222, 223,... Increase from two to four, the power reduction effect 720 increases from 56% to 78% and thus additional area cost 730. Increases from 4.5% to 6.4%, and power domains (222, 223,…) increase from 4 to 8, power reduction effect 720 increases from 78% to 88%, resulting in additional area cost. (730) increases from 6.4% to 9.2%. Therefore, the number of power domains 222, 223,... To obtain optimal efficiency should not be more than five. That is, the power reduction effect 720 and the additional area required cost 730 may be determined between two and four.

Object recognition method

8 is a flowchart illustrating an object recognition method performed in an object recognition apparatus according to an embodiment of the present invention.

First, a region of interest of an object is extracted in units of basic tiles by a neuro-fuzzy system (S801).

Thereafter, the extracted region of interest tile is converted into a region of interest tile task including the start address of the tile data in the entire image data, a two-dimensional coordinate value of the start position of the tile in the entire image, and the size of the tile (S802). .

Thereafter, the neuro-fuzzy system predicts the amount of computation to be processed by the parallel processor and transmits a control signal to the power supply device through the network on chip (S803).

Thereafter, each power regulator controls each power domain according to the control signal (S804).

After that, the converted region of interest tile task is distributed to a parallel processing processor including a plurality of processing units through a network on chip (S805).

Thereafter, each processing unit of the parallel processing processor generates a feature point and a feature vector of the object for the ROI tile task distributed (S806).

Thereafter, the object determiner recognizes an object corresponding to the vector having the closest distance compared with the vectors in the database (S807).

In operation S801, the region of interest of each object may be passed through the fuzzy motion detector 310, the cellular neural network visual concentrator 320, and the neuro-fuzzy classifier 330 of the neuro-fuzzy system 210 illustrated in FIG. 3A. 211, 212, and 213 can be extracted.

In the present invention, the computer system 100 is not limited to the above-described embodiments, and may include at least any one of the neural network block 111, the fuzzy logic block 112, and the neuro-fuzzy block 113 in which the neural network and the fuzzy logic are combined. Specifies that the neuro-fuzzy system 110 and the parallel processor 120, which include two, are applicable to all computer systems that process computations.

In the object recognition apparatus according to the exemplary embodiment, the neuro-fuzzy system 210 extracts the ROIs 211, 212, and 213 of each object from the entire image 215, thereby allowing the parallel processor 220 to display the entire image. Since only the data of the ROIs 211, 212, and 213, which are schematic images and not 215, is calculated, the object recognition speed is accelerated. As a result, real-time object recognition of 30 frames or more per second is possible at a low power of 500 mW for the input image 215 having a size of 640x480. For example, as a result of testing using a database that is widely used for experiments of object recognition such as COIL-100, the parallel processing processor 220 has to process the data due to extraction of the regions of interest 211, 212, and 213 of neuro-fuzzy. The image area is reduced by more than 50% on average compared to when no neuro-fuzzy system is applied. In addition, since feature points are extracted only in the required regions of interest 211, 212, and 213, the number of feature points is reduced, thus reducing the amount of computation required in the process of creating a vector for object recognition and matching with a database. This will be possible.

As described above, those skilled in the art will understand that the present invention can be implemented in other specific forms without changing the technical spirit or essential features.

Therefore, the embodiments described above are to be understood in all respects as illustrative and not restrictive, and the scope of the present invention is indicated by the following claims rather than the above description, and the meaning and scope of the claims and All changes or modifications derived from the equivalent concept should be interpreted as being included in the scope of the present invention.

1 is a block diagram of a computer system 100 according to an embodiment of the present invention.

2 is a block diagram of an object recognition apparatus according to an embodiment of the present invention.

FIG. 3A is a diagram illustrating a configuration required for the neuro-fuzzy system 210 to extract the ROIs 211, 212, and 213 of an object in the object recognition apparatus illustrated in FIG. 2.

3B illustrates a cellular neural network visual concentrator 320.

3C is a block diagram illustrating a neuro-fuzzy classifier 330.

FIG. 3D illustrates a circuit 360 and a result waveform 370 in which a Gaussian function, which is a representative example of the fuzzy membership function used for similarity measurement, is implemented as a CMOS device.

4 illustrates the format of a 12 byte region of interest tile task 214.

FIG. 5A is a diagram illustrating a parallel processor 220 of the object recognition apparatus of FIG. 2.

FIG. 5B is a diagram illustrating the power supply device 260 of the object recognition device of FIG. 2.

FIG. 6 is a flowchart illustrating an operation amount prediction method to be processed by a parallel processor executed by a neuro-fuzzy system of an object recognition apparatus according to an embodiment of the present invention.

FIG. 7 is a diagram illustrating a method for determining the number of power domains in the object recognition apparatus shown in FIG. 2 according to an embodiment of the present invention.

8 is a flowchart illustrating an object recognition method performed in an object recognition apparatus according to an embodiment of the present invention.

********** Description of the symbols for the main parts of the drawings **********

110, 210: neuro-fuzzy system

120, 220: parallel processor

130, 230: network on chip

140, 240: memory

160, 260: power supply

217: Task Scheduler

214: Interest tile task

222, 223: power domain

261, 262, 263, 264: power regulator

310: Fuzzy Motion Meter

320: cellular neural network vision concentrator

330 neuro-fuzzy classifier

Claims (13)

A neuro-fuzzy system including at least two of neural networks blocks, fuzzy logic blocks, and neuro-fuzzy blocks in which neural networks and fuzzy logic are combined; A parallel processor including a plurality of processing units; A power supply device controlling power supplied to the parallel processor in accordance with a control signal of the neuro-purge system; And And a network on chip coupled between the neuro-fuge system and the parallel processor and for data communication between the neuro-fuge system, the parallel processor and the power supply. The method of claim 1, And a memory for storing data extracted from said neuro-fuzzy system and intermediate data of a computation process of said parallel processing processor. The method of claim 1, The neuro-fuzzy system further includes a task scheduler, The task scheduler generates a schedule for distributing output data of the neuro-fuzzy system to the parallel processing processor. The method of claim 2, And a task scheduler for generating a schedule for distributing data of said neuro-fuzzy system to said parallel processing processor. The method according to any one of claims 1 to 4, The plurality of processing units are divided into one independent power domain for each specific number, The power supply includes a plurality of power regulators, Each of the power regulators controlling each of the power domains in accordance with a control signal of the neuro-purge system. Neuro-fuzzy systems including cellular neural networks visual attention engine, fuzzy motion estimator, neuro fuzzy classifier, and task scheduler; A parallel processing processor including a plurality of processing units; A power supply device controlling power supplied to the parallel processor in accordance with a control signal of the neuro-purge system; An object determination unit which recognizes an object corresponding to the vector having the closest distance by comparing the feature vector with the vectors in the database; And A network on chip for performing data communication between the neuro-fuzzy system, the parallel processor, the object determination unit, and the power supply device; The fuzzy motion meter generates a dynamic motion vector between successive image frames, The cellular neural network visual concentrator extracts intensity, color, and direction which are static features, accumulates together with the dynamic motion vector to generate a feature map, The neuro-fuzzy classifier extracts a seed point based on the feature map and tiles a region-of-interest (ROI) of each object by determining homogeneity through region expansion based on the seed point. extracted in (tile) units, The task scheduler converts the region of interest tile into a region of interest tile task, distributes the converted region of interest tile task to the parallel processing processor, The parallel processing processor performs a single-instruction-multiple-data (SIMD) parallel operation on the ROI tile task to generate a feature point of the object and a feature vector for the feature point, and transmits the feature vector to the object determiner. , Object recognition device. The method of claim 6, The ROI tile task A start address indicating an address of data of the leftmost point of the extracted ROI tile from all image data; A starting coordinate consisting of an X-direction coordinate and a Y-direction coordinate that are two-dimensional coordinate values of the leftmost point of the ROI tile in the entire image; And A tile size describing a width and a height of the ROI tile, respectively, And the task scheduler distributes and manages the converted region of interest tile task to a plurality of processing units of the parallel processor. The method of claim 6, The neuro-fuzzy system predicts the amount of computation to be processed by the parallel processor based on the number of ROI tiles extracted by the neuro-fuzzy classifier. The method according to any one of claims 6 to 8, The plurality of processing units are divided into one independent power domain for each specific number, The power supply device includes a plurality of power regulators, each power regulator to control each of the power domain according to a control signal of the neuro-purge system. A method of predicting a computation amount to be processed by the parallel processor in an object recognition apparatus according to claim 9, Measuring a number of ROI tiles extracted by the neuro-fuzzy classifier; A second step of selecting a specific number of power domains by comparing the measured number of tiles and the distribution reference number; A third step of supplying power to the selected power domain only; And And a fourth step of updating the number of distribution criteria of the second step according to a calculation result of the parallel processor included in the power domain to which the power is supplied. A method for determining the number of independent power domains including at least one processing unit in an object recognition apparatus according to claim 9, A first step of displaying a graph in which the number of power domains is X-axis and the power reduction effect according to the number of power domains is Y-axis; A second step of displaying a graph in which the number of power domains is X-axis and the additional area cost required for the power regulator according to the number of power domains is Y-axis; And And determining a number of power domains of intersections at which the slope of the first stage graph decreases and the second stage graph slope increases. In the object recognition method using the object recognition device according to claim 9, Extracting a region of interest of an object in units of tiles by the neuro-fuzzy system; A second step of converting the extracted region of interest tile into a region of interest tile task including a start address of the tile data in all image data, a two-dimensional coordinate value of the start position of the tile in the whole image, and a size of the tile; A third step of the neuro-fuzzy system estimating a calculation amount to be processed by the parallel processor and transmitting a control signal to the power supply device; A fourth step of each of the power regulators controlling the respective power domains according to the control signal; A fifth step of distributing the converted region of interest tile task to the parallel processing processor including a plurality of processing units through a network on chip; Generating a feature point and a feature vector of an object with respect to the ROI tile task distributed by each processing unit of the parallel processor; And And a seventh step of the object determining unit recognizing an object corresponding to the vector having the closest distance compared to the vectors in the database. The method of claim 12, The first step, Dynamic motion vector generation, extracting static features; Accumulating the dynamic motion vector and the static feature together to determine a region of interest; And And extracting the region of interest using a basic tile having a predetermined size.

Images