JPH02123877A - Automatic tracking device for mobile object - Google Patents

Abstract

PURPOSE:To attain automatic tracking of a mobile object continuously corresponding to the environmental change by revising the brightness corresponding to the brightness of the mobile object changing for each pickup frame sequentially and generating a binarization image of the mobile object based on the change. CONSTITUTION:Every time a pickup frame changes, the brightness corresponding to the mobile object 18 is decided at every frame point of time by the comparison of brightness histogram between set areas MR and TR set sequentially, a picture signal inputted is converted into a binarization, image to identify the external shape of the mobile object. Moreover, a position calculation device 6 decides the object azimuth of a TV camera based on a geometrical gravity center 9 obtained by the binarization image, a driving base controller 7 controls a camera driving base 2 according to the object azimuth and the TV camera 1 is operated to trace the mobile object. Thus, even if the same brightness as the background exists to part of the mobile object, and the background with less brightness around the vicinity of the mobile object exists, the mobile object is continuously traced automatically.

Description

Detailed Description of the Invention [Industrial Application Field 1] This invention captures an image of a moving object using an imaging device such as a TV camera, calculates the position of the moving object through image processing, and repeats this process to continuously move the object. The present invention relates to an automatic tracking device for a moving object that tracks an object.

[Prior art 1] This invention was originally filed in Japanese Patent Application Laid-Open No. 63-6708 by the applicant.
This relates to improvements to No. 7. That is, JP-A-63
In Patent No. 67087, in order to identify a moving object, a predetermined frame-shaped background area surrounding the moving object in the imaging frame is set, and the brightness of this background area is regarded as the average background brightness of the background. A portion within the background area that has a brightness different from the target background brightness is determined to be a moving object.

[Problems to be Solved by the Invention] For this reason, in the configuration described in JP-A-63-67087, the brightness of the surrounding background and the brightness of the moving object are compared and determined, so that If there is a part other than a moving object that has a brightness that does not exist in the background area, the discrimination may be insufficient.

Therefore, the present invention improves the configuration of the prior application as described above to generate a luminance that corresponds to the luminance of a moving object by updating it sequentially for each imaging frame, and uses the luminance of the moving object itself as a criterion for determining whether the moving object is moving. By identifying the pixels of an object, it is possible to accurately identify moving objects without being affected by background brightness.
This enables more reliable automatic tracking of moving objects.

Further, according to another embodiment of the present invention, even when the area of the moving object in the image is small, the image of the moving object is enlarged to increase the S/N ratio, and the moving object can be automatically automatically detected. The purpose is to perform tracking.

[Means for Solving the Problems] The automatic tracking device for a moving object according to the present invention includes an imaging device that images a moving object and outputs a video signal, and an imaging device that processes the imaging signal from the imaging device to track the moving object. automatic tracking of a moving object, comprising: a control device that generates a control signal for controlling automatic tracking of a moving object; and a drive base that supports the imaging device and controls the imaging direction of the imaging device based on the control signal from the control device. The control device determines the movement of the center of gravity of the moving object in the image obtained in the current imaging frame n based on the background surrounding the object and the final identification brightness of the moving object to be tracked. A brightness histogram HTRI (i) of a rectangular area including the object and its vicinity,
means for generating a brightness histogram HTRI(i) within a prediction area set including the current position in the current imaging frame n and the predicted position of the moving object at the next imaging frame n+1; The method is characterized in that both luminance histograms HMR(i) and HTRl(i) obtained by the means are compared and discriminated to generate the discrimination luminance of the moving object at the time of the next imaging frame n+1.

According to another embodiment of the present invention, the control device enlarges the binary image of the moving object in the current imaging frame n+1, which is determined based on the discrimination brightness of the moving object determined in the previous imaging frame n. The present invention is characterized in that the region including the enlarged moving object image obtained by processing is the rectangular region, and the discriminating brightness of the moving object is generated.

[Example] DESCRIPTION OF THE PREFERRED EMBODIMENTS A description will be given below based on the illustrated embodiments of the present invention.

FIG. 1 shows a block diagram of a device configuration of an embodiment of an automatic moving object tracking device according to the present invention.

In this embodiment, a TV camera 1 is used as an imaging device and is supported by a camera drive base 2 so that the imaging direction can be freely adjusted. A control device consisting of 7 is provided. A video signal 3 obtained from the W camera 1 is input to an image processing device 5 provided at the next stage, and geometric center of gravity data 9 of a moving object in the TV video is calculated.

Furthermore, the TV camera 1 has a built-in zoom mechanism and automatically zooms a moving object to the extent necessary for digital image processing within the image processing device 5. Position calculation device 6
As will be described in detail later, measurement data 8 such as the azimuth angle and angular velocity of a moving object can be calculated from the geometric center of gravity data manually input from the image processing device 5, and can be used as control information for other devices. Further, the position calculation device 6 calculates the coordinates of the geometric center of gravity in one frame of the video signal from the geometric center of gravity data, and outputs the coordinates to the drive platform control device 7. The drive base control device 7 controls the TV camera 1, and uses the human power signal from the position calculation device 6 to send a drive base operation signal 4 to the camera drive base 2 so that the geometric center of gravity is located at the center of the screen. Output. As a result, the azimuth angle of the W camera 1 is controlled so that the image of the moving object is captured at the center of the screen. Further, the driving platform control device 7 outputs an azimuth signal indicating the current imaging direction of the W camera 1 to the position calculation device 6 via the image processing device 5. Thereby, the position calculation device 6 outputs measurement data 8 such as the azimuth angle and azimuthal velocity of the moving object from the azimuth signal of the W camera 1 and the geometric center of gravity coordinates on the video screen.

Next, a detailed block configuration of the image processing device 5 is shown in FIG. This image processing device includes an image memory 16 that stores image data for each frame of an image signal, binary image data,
MR which calculates the geometric center of gravity 9 by manually calculating the imaging device orientation 10 and will be described in detail later.

An image calculation device 15 that calculates and outputs a TRI region, a brightness histogram calculator 14, a brightness determiner, a lookup table 12, and a binary image generator 11 that finally generate binary image data of an image signal. It is equipped.

An image of a moving object is transmitted as an image signal 3 every 1730 sec and for 1760 sec by a TV camera 1, which is an imaging device, and is stored in an image memory 16 and also supplied to a binary image generator 11. Here, in the lookup table 12, it is possible to set an initial value for distinguishing between the background and a moving object at a predetermined threshold level at the time of starting the operation of the device. When the video signal of the first frame is input, the binary image generator 11 converts the image signal 3 into a binary image with the background as "0" and the moving object as "1" according to this threshold level. The value image data is output to the image calculation device 15. This completes the initialization of the image calculation device 15, which calculates the geometric center of gravity of the moving object using the known X-Y axis projection method and the size of the moving object using theta envelope processing. To perform this initialization, as described above, the brightness threshold level suitable for the external conditions is set in the lookup table 12 at the time of startup, and the TV camera 1
This can be done, for example, by manually setting it on a moving object and then starting the calculation of the device.

In this way, normal processing is started with initial data on the geometric center of gravity and size of the moving object being obtained. Here, the signal processing within the device from the image of captured frame n obtained at time t1 to the input of the image of frame n+1 at the next time ζ will be explained based on the time chart shown in FIG. do.

The video signal at time to is stored in the image memory 16 at time t, +1/60 sec, and the image of this image is as shown in FIG. Here, image calculation device 1
5 is a rectangular area 19 (hereinafter referred to as MR) surrounding the vicinity of the moving object 18, as shown in FIG. , together with the expected movement range area 2 of the moving object 18 at time t1 of the next imaging frame n+1.
0 (hereinafter referred to as TR) is determined, and the signal is multiplied by TR and MR and outputted to the brightness histogram calculator 14.

Here, the possible displacement range of the moving object in the image capture screen between one frame will be outlined below.

Amount of change in the position of a moving object within the screen ΔX 20m5pa
Assuming that n moving objects are moving across the screen at a maximum speed of 340 m/s, and this is captured by about 30 pixels on the screen, the deviation per 1/30 sea is 340 m/s x 1/30 sec x 30 pixels. /20m
= 17 pixels, that is, the amount of positional displacement ΔX between one frame
is defined by ΔX≦17 pixels.

t as described above. The predicted movement range position of the moving object within the imaging frame range for each frame from t8 to time t8 is defined, and based on this, the range TR□ can be set with sufficient margin.

In this way, the brightness histogram calculator 14 to which the MR and TR1 area signals are input reads the image data of each designated area from the image memory 16, and the histogram HMR(i)(MR area and HTRI(i)(
TR1 region) is calculated. Each histogram example is
Shown in Figures (a) and (b). Here, in the MR histogram of FIG. 6(a), a peak of the number of pixels appears in the part corresponding to the brightness of the moving object 18, while in the TR histogram of FIG. 6(b), the peak of the moving object 18 and the background appear. There is also a peak in the number of pixels in the area corresponding to the brightness of .

The brightness histogram calculator 14 outputs the two histograms thus obtained to the brightness determiner 13.

This brightness determiner 13 calculates the following from the two input histograms to find the ratio of the number of pixels for each brightness included in each region of MR and TR, and calculates the ratio of the number of pixels for each brightness included in each region of MR and TR. A ratio characteristic like this can be obtained. Furthermore, in the brightness determiner 13, when HMR(i)≠0 and P(i) is larger than a predetermined threshold value 21, most of the pixels having that brightness exist within the MR, i.e. It is determined that the brightness represents the moving object 18, and "1" is set in the corresponding part in the lookup table 12, and "θ'' is set in the other brightness parts. Here, the moving object 18 is The brightness representing the background was set as "1"', and the brightness representing the background was set as "0".

Between the next time t and time t1+1/60 sec, the image signal 3 of the next captured frame n+1 output from the TV left camera is generated by the binary image generator 11 according to the contents set in the lookup table 12. Binarized.

In the image processing device 15, the direction of the TV left camera changes due to tracking of the TV left camera in the direction of the moving object 18, but as shown in FIG.
Based on this, a region TR2 that is spatially the same as TRI is set. Furthermore, the image calculation device 15 operates in the set area TR2.
The geometric center of gravity of the binarized image output from the binary image generator 11 and the size of the object 18 are determined. The information obtained in this way generates the luminance corresponding to the moving object 18 at that time during the period from t, + 1/60 sec to ζ, similarly to the period from t0 + 1/60 sec to tl described above. used for. Thus, t1+1
The luminance corresponding to the moving object 18 updated from /60 sec to ζ is used for discrimination from the next time ζ. In this way, each time the imaging frame changes, by comparing the brightness histograms between the regions MR and TR that are set sequentially,
The brightness corresponding to the moving object 18 at each frame time point is determined, and the input image signal is thereby converted into a binary image to identify the outer shape of the moving object 18. Furthermore, based on the geometric center of gravity 9 obtained from this binarized image, the position calculation device 6 determines the TV left camera target direction, and the drive stand control device 7 controls the camera drive stand 2 according to the target direction. The camera is operated to track the moving object 18. In this way, even if there is a part of the moving object 18 that has the same brightness as the background, or there is a background part that has less brightness in the vicinity of the moving object 18, it is simply a predetermined threshold level. Instead of determining the brightness difference, the brightness corresponding to the brightness of the moving object, which can change for each imaging frame, is updated sequentially, and based on this, the brightness of the moving object is
Since it is configured to generate a valued image, it is possible to automatically track the moving object 18 continuously in response to environmental changes.

Next, another embodiment of the present invention will be described based on FIGS. 7 to 11. FIG. 7 shows a block diagram of the configuration of the image processing apparatus 5 of this embodiment, and the same parts as in the first embodiment are given the same reference numerals and their explanations will be omitted. The configuration shown in FIG. 7 differs from the configuration shown in FIG. 2 in that a binary image memory 22), an object area determination device 23, and an object image memory 24 are added.

In this example, the characteristic parts will be clearly described in the explanation of the operation of the concrete block described below, but in the previous example, the vicinity of the moving object is surrounded by a rectangular area MR, and the area between it and the background is The brightness histograms were compared, and in this example, the disturbance due to the background in the MR is reduced by capturing the outline of the moving object as an area surrounding it by enlarging the binary image.

The operation of the embodiment shown in FIG. 7 will be explained based on the time chart shown in FIG. Time t. At +1/60 sec, time t occurs as in the previous embodiment. The image signal at 9 is stored in the image memory 16 as shown in FIG.
As shown in the figure, the image is stored in the binary image memory 22 at time t. It is assumed that the position and size of the moving object 18 have already been determined within the image processing device 15. The image calculation device 15 has already calculated the time t. moving object 18 in
A rectangular region 19 (MR) surrounding the vicinity of the moving object 18 and a region 20 (TRI) to predict and search for the presence of the moving object 18 at time t1 are determined based on the position and size of the object. It outputs the TR region signal 26 to the determiner 23 and the brightness histogram calculator 14.

Here, the object region determiner 23 selects the MR region pixels from the binary image memory 22 based on the input MR region signal 25.
Perform an operation to determine the target object area by searching and inputting from . This target area determination is performed by enlarging the binary image because there is a possibility that a part of the image of the moving object 18 is missing due to disturbances such as noise in the MR area. This is corrected to create a binary image showing the moving object 18 as the target object. The enlargement processing referred to here is an operation of setting four neighboring pixels to "1" for each contour pixel of the moving object 18 (for each "1" pixel). In this way, a moving object image whose outline is rough in the judgment during binarization as shown at 18' in FIG.
As shown by 18'' in FIG. 10, the outline is stored in the object image memory 24 in a slightly swollen state.

Here, the brightness histogram calculator 14 generates a brightness histogram HMR (i) including pixels determined to be moving objects in the MR region and a brightness histogram HT'' (i) in the TRI region according to an input signal from the object image memory 24. is calculated and transmitted to the brightness determiner 13.The subsequent processing is performed in the same manner as in the previous embodiment, calculates the brightness corresponding to the moving object at that time, and calculates the brightness at time t1 shown in FIG. Use this as calculation data for binary image processing.Repeat this operation sequentially,
Automatically tracks moving objects. In the embodiments shown in FIG. 7 and subsequent figures, the brightness histogram is generated after enlarging the binary image of the moving object, thereby preventing disturbances caused by background noise and Even when the image area to display is small, moving objects can be accurately identified and automatically tracked.

[Effect of the Invention 1 The embodiment of the automatic tracking device for a moving object according to the present invention is as follows. Even if there is a background part with low brightness in the surrounding area, the brightness corresponding to the brightness of the moving object, which can change with each imaging frame, is updated sequentially instead of simply determining the brightness difference based on a predetermined threshold level. Since the system is configured to generate a binary image of a moving object based on this, it is possible to automatically track the moving object continuously in response to environmental changes.

According to another aspect of the invention, a brightness histogram is generated after enlarging a binary image of a moving object, thereby preventing disturbances caused by background noise and displaying the moving object on the screen. Even when the image area is small, moving objects can be accurately identified and automatically tracked.

[Brief explanation of the drawing]

FIG. 1 is a block diagram showing the overall configuration of the embodiment, FIG. 2 is a block diagram showing the configuration of the image processing device shown in FIG. 1, which is a characteristic part, and FIG. A time chart showing the state, FIG. 4 is t. A schematic diagram showing an image based on an image signal at time, FIG. 5 is a schematic diagram of an image at time t1, FIG. 6 (a), (b),
(c) is a diagram of an example of a luminance histogram in the luminance histogram calculator and luminance judger, respectively; FIG. 7 is a block diagram of the image processing device portion according to another embodiment; and FIG. 8 is a timing diagram in the configuration of FIG. 7. FIG. 9 is an example of a binary image at time -, FIG. 10 is an example of a binary image enlarged and stored in the object memory, and FIG. 11 is an image at time t1. It is a diagram. DESCRIPTION OF SYMBOLS 1... W camera, 2... Camera drive stand, 3... Video signal, 4... Drive stand operation signal, 5... Image processing device, 6... Position calculation device, 7...・Drive base control position,
8...Measurement data, 9...Geometric center of gravity data, 10
... Imaging direction signal, 11... Binary image generator, 12
... Lookup table, 13... Brightness determiner,
14... Brightness histogram computing unit, 15... Image computing device, 16... Image memory, 17
...Image, 18...Moving object, 18'...Moving object binary image, 1B''...Moving object binary image after enlargement processing, 19...
Rectangular area MR, 20... area TRI, 20'...
Territory jtTR2, 21...Threshold level, 22
...Binary image memory, 23...Object area determiner,
24...Object image memory.

Claims (4)

[Claims] (1) An imaging device that images a moving object and outputs a video signal, a control device that processes the imaging signal from the imaging device and generates a control signal to control automatic tracking of the moving object, and the imaging device an automatic tracking device for a moving object, comprising: a drive base that supports an image pickup device and controls an imaging direction of an imaging device based on a control signal from the control device; Based on the surrounding background and the final identification brightness of the moving object to be tracked, a brightness histogram H^M^R(i) of a rectangular area including the moving object and its vicinity in the image obtained in the current imaging frame n; Generate a brightness histogram H^T^R^1(i) within a prediction area set including the current position in the current imaging frame n and the predicted position of the moving object at the next imaging frame n+1. and the luminance histogram H^M^ obtained by this generating means.
An automatic tracking device for a moving object, characterized in that it compares and discriminates R^1(i) and H^T^R^1(i) to generate discrimination luminance of the moving object at the time of the next imaging frame n+1. (2) In claim 1, the comparison determination is P(i)=H^M^R(i)/H^T^
An automatic tracking device for a moving object, characterized in that the value of R^1(i) is determined using a predetermined threshold value to generate luminance at each point in time of the moving object. (3) In claim 1, the control device controls the current captured frame n+1 determined based on the discrimination brightness of the moving object determined in the previous captured frame n.
The area surrounding the vicinity of the enlarged binary image of the moving object obtained by enlarging the binary image of the moving object inside is defined as the rectangular area,
An automatic tracking device for a moving object, characterized in that it generates a discrimination luminance for the moving object. (4) In claim 3, the operation of enlarging the binary image includes “1” in the binary data.
1. An automatic tracking device for a moving object, characterized in that the process sets pixels near a pixel "1" to "1".