JPH04260979A - Detecting and tracking system for mobile objection - Google Patents

Abstract

PURPOSE:To offer a system for detecting the motion of plural mobile objections included in a continued image frame and tracking an intended specific mobile objection. CONSTITUTION:After calculating an optical flow from the luminance information of respective picture elements in a continued image frame by a simplified MOC filter 104, motion information relating to the boundary of a mobile objection identified by the optical flow is converted by a time space normalizing filter 105 for time spatially averaging and regulating the motion information so as to be concentrated into the center of the mobile objection, projection distribution is applied to the converted data by a feature extracting filter 106 and the distributed data are converted into one-dimensional speed information and then inputted to a competition type neural network 102 having three-layer structure. A box (window) including the specific intended mobile objection is defined from the center information and contour information of the mobile objection is defined in order to track the mobile objection, the shape information of the mobile objection in the box is extracted by a feature extracting filter 108 and inputted to a novel filter neural network 109 having one-layer structure to recognize the shape of the mobile objection.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a system for detecting and tracking moving objects. More specifically, the present invention relates to a system for detecting and tracking moving objects that can detect and track multiple moving objects in real time by applying neural network technology that imitates human visual function. Possible applications include logistics system management and autonomous robot vision.

0002

[Prior Art] Regarding a technology for tracking bright points on a video image, see US Pat. ”(METHOD OF PROCES
SING A VIDEO IMAGE TO TRA
CK A BRIGHT SPOT AND TO P
ROTECT SAID TRACKING FROM
INTERFERENCE FROM OTHER
BRIGHT SPOTS). In conventional technology, a virtual window is created to encompass a nearly circular moving object, and the virtual window is opened in the direction that maximizes the correlation between the two images before and after the movement within this virtual window. Tracking was done by moving the .

0003

[Problem to be Solved by the Invention] Velocity information (optical flow) regarding the contour points of a moving object can be obtained only at the contour points of the object when the brightness of the object is uniform. Velocity information consists of two components, speed and moving direction, but in order to obtain velocity information about a moving object as a single mass, it is necessary to integrate velocity information at contour points of the object by some means. However, in general, for this purpose, a moving object must first be connected to another moving object,
Or it may be necessary to separate it from a stationary object.

[0004] In the above-mentioned prior art, the movement to be tracked is determined by preparing another sub-window to exclude other moving points that enter a pre-provided virtual window. The influence of moving points within the subwindow on points was excluded. For this reason, there is a problem in that as the number of moving points to be tracked increases, the processing time for calculating and setting a new sub-window increases.

An object of the present invention is to provide a moving object detection and tracking system that executes the processing required for tracking in a fixed time regardless of the number of moving objects on the screen, and a system for detecting and tracking moving objects that performs the processing required for tracking in a fixed time regardless of the number of moving objects on the screen. An object of the present invention is to provide a moving object detection and tracking system that is not affected by collision or concealment by other objects.

[0006]

[Means for solving the problem] In order to achieve the above object, first, motion information (optical flow) regarding the outline of a moving object is calculated by applying a simplified MOC filter to two consecutive image frames. do. The simplified MOC filter is an optical flow calculation filter that includes three processing stages (directional edge detection section using a spatial filter, brightness difference detection section using a temporal filter, and product-sum combination section of a spatial filter and a temporal filter). Next, by applying a spatio-temporal normalization filter to the optical flow for each direction of movement, speed information regarding the moving object for each direction of movement is standardized and extracted. After taking the one-dimensional projection distribution of the velocity information regarding the moving object in the direction of movement and in the direction perpendicular to the direction of movement, the direction (direction of movement,
input to the local maximum detection neural network for each projection distribution direction). The local maximum value detection neural network is a competitive neural network with a three-layer structure.

Next, a box containing the moving object is defined from the center and contour information of the moving object detected by the local maximum value detection neural network, and object recognition is performed within this box by the novelty filter neural network.

[0008]

[Action] Simplify M for two consecutive image frames
Motion information (optical flow) regarding the contour of the moving object is calculated by applying the OC filter. The simplified MOC filter is characterized in that velocity information can be calculated without depending on the brightness of the object. By inputting the optical flow information to the spatio-temporal normalization filter for each direction of movement, it is possible to extract normalized speed information regarding the moving object as a single block. The one-dimensional projected distribution waveform of velocity information about a moving object is expressed in the direction (moving direction,
Each moving object is The coordinates of the center of and the direction of movement can be determined. This competitive local maximum value detection neural network sharpens the local maximum value of the external input signal over time.
It acts as a dynamic short-term memory. A box (window) containing the moving object can be defined by using the center information of the moving object obtained by the competitive local maximum detection neural network group and the contour information obtained by the optical flow. A novelty filter neural network is applied to the moving object within this box (window). A novelty filter neural network is a neural network that outputs a novel pattern part for a learned pattern. Due to this property of the novelty filter, even if a collision with another moving object or concealment by another object occurs, if the moving object to be tracked is memorized as a learning pattern, the shape of the moving object to be tracked is can be output associatively. This makes it possible to recognize objects that are not affected by collisions with other moving objects or concealment by other objects.

[0009]

[Embodiment] An embodiment of the present invention will be described in detail below. FIG. 1 shows a block diagram of a moving object detection and tracking system according to the present invention. The sensor 101 provides bitmap images to a motion detection subsystem (IPLS) 102 and a motion tracking subsystem (MTS) 103. A motion detection subsystem (IPLS) 102 performs image information processing to extract motion information using a simplified MOC filter 104 and a spatio-temporal normalization filter 105 on the image obtained by the sensor 101, and then performs feature extraction. 1 filter 106 extracts one-dimensional motion information features, and a local maximum value detection neural network group 107 detects center information of the moving object.

The motion tracking subsystem (MTS) 103 cuts out a box region containing the moving object from the image obtained by the sensor 101 by using the center information of the moving object obtained by the motion detection subsystem (IPLS) 102. , Feature extraction 2 filter 108 from the image within the box area
After extracting shape information regarding the moving object, the novelty filter neural network 109 recognizes the moving object to be tracked, and uses this result to drive the sensor drive unit 110 to capture the intended moving object at the center of the field of view. do.

First, the motion detection subsystem (IPLS)
102 will be explained in detail. The pixel brightness information of continuous image frames obtained by the sensor 101 is expressed as Iij(t1
), Iij(t2), . ．． ．． It is described as follows. Here i, j
indicates the pixel position, and t1 and t2 indicate the time. These pixel brightness information are input to simplified MOC filter 104. The simplified MOC filter 104 has a three-layer structure and calculates the amount of movement in each direction of pixels at the same position in consecutive image frames, that is, the optical flow. This optical flow calculation filter is based on S. Grossber
g, M. “A Neural Archit” by Rudd.
ecture for Visual Motion
Perception* Group Elem
ENT APPARENT MOTION”, Neur
al Network Vol2, pp421-450
(1989), or S. Grossberg, “Th
e Adaptive Brain II”, Else
vir (1987). Simplification M
The output signal of the OC filter 104 is expressed as Uij(k,t). Here, i and j indicate the position of the pixel, k indicates the direction, and t indicates the time. It is assumed that the directional resolution is 8. Simplified MOC
The filter 104 detects the edge of the currently captured image and compares the brightness information of the currently captured image with the image captured immediately before, so that k=0 to 7.
Contour velocity information in eight directions corresponding to , that is, optical flow Uij (k, t) is calculated. The optical flow Uij (k, t) is spatiotemporally averaged and normalized by the spatiotemporal normalization filter 105 and converted into Vij (k, t). The conversion formula is based on the following formula (1).

0012

[Equation 1] εdVij (k, t)/dt = −αVij (k, t
) + (A-Vij(k,t))NEij-Vi
j (k, t) NIij (1) where ε is a time constant, NEij = Uij (k, t), NIij =
ΣGσ(u,v)U(i+u)(j+v)(k,t),
(u, v)≠(0,0) Gσ(u,v)=(1/2πσ)exp(−(u2+v
2)/2σ2).

Equation (1) normalizes the speed so that the value of Vij(k,t) falls between 0 and A. That is, the influence of a moving object having an extremely large velocity component on velocity magnitude information of other moving objects is suppressed. Furthermore, the spatio-temporal normalization filter 105 is the simplified MOC filter 10
It also serves as a very short-term storage memory that performs local spatio-temporal averaging of the output signals Uij(k,t) of 4. The above processing steps are schematically illustrated using FIGS. 2, 3, and 4.

In FIG. 2, three objects exist in the scene, one of which is stationary, and the other two objects are moving to the right at different speeds. There is. Vi
The feature extraction 1 filter 106 calculates the projection distribution of j(k,t) along the movement direction and the direction perpendicular to the movement direction for each of the eight directions, and converts it into Sx,k and Sy,k.

FIG. 4 schematically shows the process of calculating this projection distribution. As a result, for the two moving objects, the center information will be reflected in the maximum values of Sx,k and Sy,k.

FIG. 5 shows the detailed structure of one of the local maximum value detection neural networks 107 that outputs moving object center information. The local maximum value detection neural network is F1, F2, F3
It has a three-layer structure shown in . The input signal Sk is simultaneously input to the F1 and F3 layers. The F2 and F3 layers have a competitive structure in which they receive negative inhibitory inputs from other nodes (neurons) in the same layer and positive excitatory inputs from their own nodes (neurons). The values of these connections within the same layer do not change. Xk and Yk indicate the center coordinates of the moving object and are output to the F2 layer. The node activation value Xj of the F2 layer is determined by the following equation (2) by the connections from the F1 layer to the F2 layer called instars and the connections within the same layer described above.

[0017]

[Math 2]
ndXj/dt = −αXj + (A −
Xj) [ΣWjiSi + f(Xj)] − XjΣ
f(Xi) (2)
i=0
i≠j The coupling value wij from the F1 layer to the F2 layer is expressed by the following formula (3)
It is determined by the instar learning rule given by .

[0018]

[Formula 3] dwij/dt = e(t)(Si-wij)Xj

(3) Here, e(t) is a monotonous decay function that decays over time.

[0019] Through the connection from F2 to F3 layer called the outstar learning rule and the connection within the same layer described above,
The node activation value Yk of the layer is determined by the following equation (4).

[0020]

[Math 4]
ndYk/dt = −αYk + (A − Yk
) [ΣZkjXj + g(Xj) + Sk] −
YkΣg(Yi) (4)
j=0
i≠k The connection value zkj from F2 to F3 layer is determined by the outstar learning rule given by the following equation (5).

[0021]

[Formula 5] dzkj/dt = e(t)(Yk-zkj)Xj

(5) Here, e(t) is the formula (3)
It is a monotonous decay function that decays over time, similar to .

Based on the above equations, the local maximum value detection neural network group 107 learns the motion distribution of each moving object to be tracked. The combination of the F1 layer and the F2 layer operates as a so-called matched filter, sharpening the peak of the signal formed from the node activation values of the F1 layer and removing unnecessary noise components. The F2 layer operates as a short-term memory, holding the peak of the signal Si in the short term, and at the same time operates as a long-term memory through feedback coupling from the F3 layer. As a result, even if the moving object to be tracked is temporarily obscured by another moving object or a stationary object, the center coordinates are stored in the short-term memory formed by equation (2), so that they can be tracked later. If it reappears, you can immediately start tracking it. The local maximum detection neural network is considered to be a subset of the ART1 network. For more information on how the ART1 neural network works, see “The Aug.
Mented ART1 NEural Network
k”, IJCNN-91 (Seattle), proc
eedings, Vol II, pp469-472. It is described in

Next, in FIG. 1, the motion tracking subsystem (MTS) 103 cuts the image obtained by the sensor 101 using the center information Xk, Yk output by the local maximum value detection neural network group 107. In the captured image, features related to the shape of the object are extracted by using the feature extraction 2 filter 108. The novelty filter neural network 109, trained to recognize the shape of an object, associates and outputs novel parts of the shape of the object, thereby preventing collisions with other moving objects, or being obscured by other moving or stationary objects. Even when this occurs, the shape of the intended tracked moving object can be output associatively. This makes it possible to continue tracking reliably. Methods such as edge detection, Fourier transform, and generalized Hough transform are used for shape and contour description.

FIG. 6 shows the process of determining a box containing a moving object centered on the center information Xk, Yk regarding the moving object output from the local maximum value detection neural network group 107. The objects in this figure correspond to those in FIG. Center information 601 and 602 of two moving objects can be separated by boxes 602 and 603 for separating the objects. Although the object 605 is included in the image obtained from the sensor 101, since it is stationary, no information about its movement can be obtained, and a box containing it is not generated.

FIG. 7 shows the configuration of the novelty filter neural network 109. Regarding the principles and applications of novelty filters, see T. “Self-org” by Kohonen
anization and association
Memory” (Springer-Verlag,
1989), or E. “App” by Ardizzore
lication of the Novelty F
Ilter to Motion Analysis”
(INNC 1990, Paris, pp46-49). The novelty filter neural network is a fully interconnected neural network with a single layer structure, and the activation value Xi of each node is
(t) varies according to equation (6).

[0026]

[Formula 6] Xi(t+1) = Xi(t) + ΣWji(t
)Xj(t)
(6)
i≠j The learning rule is given by the following equation (7).

[0027]

[Formula 7] dwij/dt = −αXiXj

(7) From the above equation (7), if the input shape pattern is presented to the network for a long time, the activation value output Xi(t) of each node gradually approaches 0. In the learning mode, the shape pattern to be learned is presented only for a certain period of time (t=t1). . During this time, the weight value wij learns the input shape pattern, and conversely, the activation value output Xi(t) of each node gradually approaches zero. By selecting an appropriate value for t1, learning converges with one pattern presentation. In normal times other than learning mode, the pattern presentation time varies from t1 to t2 (for example, t1
(a positive integer multiple of 1 or more). As a result, if the presented shape pattern is equal to the learned pattern, the activation value output Xi(t) of each node gradually approaches zero. Furthermore, when a pattern with a part missing from a complete learning pattern is presented, the pattern with the missing part is output as an associated novel pattern. Once the learning of the shape of the object to be tracked is completed, tracking becomes possible without being affected by objects overlapping each other.

The principle will be explained using FIG. 801 is an object moving from left to right, and 802 is an object passing in front of the object 801, which partially hides the front part of the object 801. If this overlapping time is very short, the front outline of the object 801 can be output associatively due to the nature of the novelty filter, so it is possible to reproduce the complete shape of the object 801. be.

Next, an implementation example of the moving object detection and tracking system shown in FIG. 1 will be described. In FIG. 9, 901 is a general CCD camera as an image sensor, 902 is a Mei
A board 903 is a board for displaying images built into the M10 casing manufactured by Ko Corporation for capturing images. The captured image frames are sent to a four-frame Meiko via a transducer bus 904.
The data are processed within a transducer network comprised of Inmos T800 transducers on In-Sun transducer boards 905. Since 16 Inmos transducers are mounted on one board, a total of 64 T800 transducers can be used as processing modules. The motion detection subsystem (IP
LS) 102, motion tracking subsystem (MTS) 103
was implemented on the In-Sun transducer network described above. The object tracking results are displayed on the image display board 907
Monitor screen 906 directly connected to Sun4/37
0 monitor screen 908 and Macintosh monitor screen 909. Sun4/370
Regarding the display on the monitor screen 908 and Macintosh monitor screen 909, please refer to the X-Wind
Use ow. Images are input by taking images of two model trains 911 with a CCD camera 901. Collision and concealment time can be controlled by placing bridges and tunnels near the moving model train 911.

Motion Detection Subsystem (IPLS) 102, Motion Tracking Subsystem (MTS), implemented in FIG.
103 shows a module block diagram of 103. The box in the diagram is T8
A compiled code module running on a 00 transducer is shown. For example, code module 1005,
1006, 1007, 1008a, 1008b, etc. were compiled with the standard OCCAM compiler,
Code modules 1002, 1003, and 1004 were compiled with a standard C language compiler. Cylinders 1009a, 1009b, and 1011 represent T800 transducers connected in a pipeline. Arrows connecting each block indicate bidirectional communication paths between each code module.

Cylinder 1001 is a UNIX file system based on SunOS 4.03. CS_HOS
Communication between the transmitter network and the UNIX file system 1001 is performed via the T module 1002. Sun4/370 monitor screen 908
, Macintosh monitor screen 909, UN
CS for communication with IX file system 1001
_HOST module 1002 is used.

The TTY module 1003 is a module for setting parameters for initializing the FG module 1005 so that it can correctly sample a standard video signal. The FG module 1005 is P
Displays any standard video signal such as AL, NTSC, etc.
To correctly sample regardless of the size of a processing screen. TTY module 1003, GFX module 1004
is a control module that complements the functions of the CS_HOST module 1002, and normally input/output between the UNIX file system 1001 and a transfer network consisting of the FG module 1005, FD module 1007, etc. is performed via the CS_HOST module 1002. . In addition, these TTY modules 1
003, the GFX module 1004 is written in C language, but the CS-TOO from Meiko Co., Ltd., which is included as standard, is written in C language.
The LS library allows bidirectional communication with the transfer network and the UNIX file system 1001. The GFX module 1004 is used to switch the graphic mode of the FD module 1007 as necessary. For example, in order to draw an icon of the tracked moving object in order to notify when the tracking is successful, the graphic mode is switched by a command from the GFX module 1004. Note that in order to enable operation in the above two modes (normal display mode, icon drawing mode), the FD module 1007 uses a function to switch the display between the icon image and the icon image while updating the image sent from the FG module 1005. It has a 2-screen switching function.

The FG module 1005 consists of control software for controlling image capture and communication codes for communicating with other modules (1003, 1004, 1006). The FG module 1005 takes in images from the camera and sends them to the BUFF module 100.
Transfer the image to 6. Furthermore, when the system starts operating, various parameter settings for image capture (setting of standard video signal selection, display, processing screen size, etc.) are performed based on commands from the TTY module 1003. When image capture starts, the image is divided into two horizontally and sent to the BUFF module 10 via two channels.
06, the image is transferred. F like other modules
G module 1005 is implemented by a collection of many parallel processes. For example, one of them is a hardware-controlled process for image capture,
Most of the other processes are for efficiently performing data communication with other modules (TTY module 1003, GFX module 1004, BUFF module 1006) in parallel. For example, data communication routines are implemented using standard network protocols such as TCP/IP. Similarly, the BUFF module 1006 has many parallel processes achieving high-speed communication via links. Note that in this text, link and communication channel are used synonymously. The main purpose of the BUFF module 1006 is to buffer the output from the FG module 1005 and send it to the Master1 module (1008a) and the Master1 module (1008a).
The data is divided into two parts and transferred to the ter2 module (1008b). The reason for dividing the processing into two modules is to minimize the overhead time required for image transfer and shorten the overall processing time by executing the same processing in parallel at the same time on as many T800 transducers as possible. It's about doing. As mentioned above, B
The UFF module 1006 divides the image into two equal parts and sends each divided image to the Master1 module (100
8a) and transferred to the Master2 module (1008b). BUFF module 1006 also
Image data and graphic instructions are received from the er1 module (1008a) and the Master2 module (1008b). As mentioned above, the FG module 100
Instructions for switching graphics modes from 5,
Master1 module (1008a), Master
Due to the need to automatically switch and transfer image data from the r2 module (1008b) through the same link,
Strict protocols were set. In this module as well, the above two types of communication were implemented by executing them as parallel processes. The FD module 1007 controls the screen display operation of the image display monitor screen 906.

Master1 module (1008a)
, the Master2 module (1008b) applies each pipeline (1009) of the MOC filter 104 to simplify the image.
The image is divided into as many small block images as the number of T800 transducers in (a, 1009b). In Figure 10, each pipeline actually consists of 20 T800 units.
It consists of a transformer. Master
Since the Master 1 module (1008a) and Master 2 module (1008b) have the same configuration,
For convenience, only the Master1 module (1008a) will be described. Master1 module (1008
In a), the 20-divided image is sent to the simplified MOC filter pipeline module (1009a), and if it is necessary to display it later, the simplified MOC filter processing result image Vij (k, t),n is sent to the simplified MOC filter pipeline module (1009a). Usually, one-dimensional partial projection distribution amounts Sx,k,n and Sy,k,n are received. Here, the subscript n indicates a subscript corresponding to each divided image, reflecting the fact that each partial amount is obtained for each divided image. The one-dimensional partial projection distribution amount is transmitted to B via a link under the control of the Master 1 module (1008a).
The data are aggregated in the UFF2 module 1010 to obtain one-dimensional projection distribution amounts Sx,k and Sy,k. Image data Iij(t) divided into two via this same link is also BUF
It is sent to the F2 module 1010 and used for processing in the novelty filter neural network module 1012. In addition, since all links are bidirectional communication paths, they can be used as transfer paths for graphic commands, output images of the BUFF2 module 1010, the local maximum value detection neural network module 1011, or the novelty filter neural network module 1012. is also used. Simplified MOC filter pipeline module (Simplified MOC filter module (
1009a), simplified MOC filter module (10
09b)) also Master1 module (1008a)
, similar to the Master2 module (1008b),
Modules with the same configuration, each with 20 T80
Consists of 0 transducers. Each T800 transducer has an identical slave module 1009c running on it. Each slave module 1009c is the first master module (Master1 module).
1008a), Master2 module (1008b
)) and the last module are connected to modules on both sides of each other.

Each slave module 1009c is composed of many parallel processes, and in particular has a ring-shaped input/output buffer for buffering input images and processing results Vij(k,t),n. This provides more efficient buffering. The slave module 1009c sequentially performs the simplified MOC filter 104 processing and the spatio-temporal normalization filter 105 processing on the image Uij(k,t)n extracted from the input ring buffer, thereby obtaining Vij(k,t),n. Calculate. Furthermore, by performing feature extraction 1 filter 106 processing on Vij (k, t), n, Sx, k, n, Sy,
Calculate k and n. Each slave module 1009c automatically transfers these processing results to the master module by outputting the output result data to the link of the adjacent module.
1 module (1008a), Master 2 module (1008b)). BUFF2 module 1010 is Master1
Through the module 1009a and the Master2 module 1009b, all one-dimensional partial projection distribution amounts Sx
, k, n, Sy, k, n, one-dimensional projection distribution amounts Sx, k, Sy, k are calculated. Also, Mas
The original image Iij is obtained by integrating the half-divided images sent from the ter1 module 1009a and the Master2 module 1009b.

The BUFF2 module 1010 sends the projection distribution amounts Sx,k and Sy,k to the local maximum value detection neural network module 1011. In the local maximum value detection neural network module 1011, eight outputs from the local maximum value detection neural network 107 corresponding to each movement direction are compared for each projection distribution direction (X, Y), and the maximum value is detected. As a result, center information (moving direction, center coordinates) of the moving object can be obtained. The BUFF2 module 1010 sends the original image Iij to the novelty filter neural network module 1012. In the novelty filter neural network module 1012, a box (window) region containing the moving object is determined from the original image Iij. Further, the feature amount related to the shape obtained by the feature extraction 2 filter 108 within this box (window) area is input to the novelty filter 109. The BUFF2 module 1010 drives the sensor drive circuit 110 by decoding and interpreting the output of the novelty filter 109. In the actual implementation, the local maximum detection neural network module 1011 consists of 16 T800 transducers. Local maximum value detection neural network module 10
11 is composed of 16 local maximum value detection neural networks 107. Novelty filter module 1012 consists of one T800 transformer.

[0037]

[Effects of the invention] By using the local maximum value detection neural network, the moving direction of the center of a moving object can be easily detected, and by using the novelty filter neural network, other stationary and moving objects can be tracked. This has the effect of improving tracking ability by eliminating short-term effects such as collisions and concealment on moving objects.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a moving object detection and tracking system according to the present invention.

FIG. 2 is a block diagram for explaining how a velocity component of a moving object is generated.

FIG. 3 is a block diagram for explaining how a velocity component of a moving object is generated.

FIG. 4 is a block diagram for explaining how a velocity component of a moving object is generated.

FIG. 5 is a diagram showing the configuration of a three-layer neural network that detects local maximum values of input.

FIG. 6 is a schematic diagram showing a process of separating moving objects based on velocity information.

FIG. 7 is a diagram showing the configuration of a one-layer neural network that operates as a novelty filter.

FIG. 8 is a diagram showing a collision between a tracked moving object and another moving object.

FIG. 9 shows a block diagram of a system actually implemented according to the invention.

FIG. 10 is a diagram for explaining FIG. 9 in more detail.

[Explanation of symbols]

101...Sensor 102...Motion detection subsystem (IPLS, Inter
estingPoint Locator Subsy
stem) 103...Motion tracking subsystem module (MTS, M
104...Simplified MOC (Motion Tracking Subsystem) 104...Simplified MOC (Motion Tracking Subsystem)
d Contrast) filter 105...Spatio-temporal normalization filter 106...Feature extraction 1 filter 107...Local maximum value detection neural network group (C
DNN, Centroid Detection Ne
ural Network) 108... Feature extraction 2 filter 109... Novelty filter neural network 110
...Sensor drive circuit 601...Moving object 1 (rectangle) 602...Moving object 2 (circular) 603...Box (window) containing moving object 1 604
…Box (window) containing moving object 2 605…Stationary object 801…Tracking moving object 802…Other stationary or moving object 901…CCD camera 902…Image capture board 903…Meiko M10 housing 904…Meiko Corporation A transformer bus 905 that connects a Sun4/370 (built-in 64 transformers) with a M10 chassis (with built-in image capture board and image display board) made by Manufacturer Co., Ltd....Sun4/370 (64 transformers) 906...image display Monitor screen 907 for...Image display board 908...Monitor screen for Sun4/370 (X-W
indow) 909...Macintosh monitor screen (X-Window) 910...Ethernet 911...Two model trains 1001...Unix file system 1002...Transfer network host (C
S_HOST) module 1003...TTY (terminal screen) module 1004...G
FX (graphics) module 1005...FG (image capture) module 1006...BUFF module 1007...FD (image display) module 1008a...M
aster1 module 1008b...Master2 module 1009a...Simplified MOC filter pipeline 11009b...Simplified MOC filter pipeline 2
1009c...Slave module configuring the simplified MOC filter pipeline 1010...BUFF2 module 1011...Local maximum value detection neural network module (CDNN, Centroid Detection)
on Neural Network) 1012...Novelty Filter Neural Network Module

Claims (4)

[Claims] Claims: 1. A method for determining motion information (optical flow) for each moving direction at a boundary point of a moving object, which is determined by the luminance change of each pixel on an image of consecutive image frames captured by an image input device. one means, a second means for extracting velocity information regarding an object from motion information (optical flow), a third means for taking a projection distribution of velocity information obtained from the second means, a third means In order to output the local maximum value of the projected distribution amount by inputting the projected distribution amount obtained by The fifth means is to extract shape information, the sixth means consists of a single-layer neural network for associative memory of shape information, and the seventh means is to move a specific moving object to the center of the field of view. What is claimed is: 1. A moving object detection and tracking system, characterized by having the system as an object tracking means. Claim 2: Motion information obtained by the first means (
2. The moving object detection and tracking system according to claim 1, wherein said second means inputs said optical flow to said third means after being averaged and normalized spatiotemporally. 3. In the fifth means, image information regarding the moving object is cut out by providing a box containing the moving object centered on the center information obtained by the fourth means. 2. The moving object detection and tracking system according to item 1. 4. The movement according to claim 1, wherein the seventh means includes a driving device for driving an image input device to track a specific object among the moving objects. Object detection and tracking system.