JP2002259986A - Moving image processing optimizing device and its method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image processing optimizing device and its method for making effective use of a usable calculation source for giving best precision thereto by selecting a processing method while considering a calculation amount. SOLUTION: When inputting image signals from cameras 21, 22...2n, observation parts 41, 42,...4n transmit observation times to a trace part 8, receive predicted values for the number of observed people, observation sizes and positional attitudes calculated from trace models and the observation times from the trace part 8, determine the types of necessary template images, and select optimum approximate templates for the template images from the usable calculation source. Person detection processing is performed for each selected approximate template by template matching, and if a detected person is on a list, observation information is transmitted to the trace part 8, and the trace part 8 transmits the observation information in a non-corresponding area to a discovery part 6.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing optimizing apparatus and a moving image processing optimizing method, and more particularly, to selecting a processing method based on a calculation time available in image processing to obtain an optimum accuracy. The present invention relates to a moving image processing optimizing apparatus and a moving image processing optimizing method.

[0002]

2. Description of the Related Art The present inventor has filed Japanese Patent Application No. Hei 11-30818.
No.7 proposed a moving object tracking device. In this moving object tracking device, images are independently captured by a plurality of cameras, and a feature extraction process is performed on an input image obtained by a corresponding camera in an observation unit. The feature amount obtained by the processing is associated with each tracking target based on the predicted position sent from the tracking unit, and then transmitted to the tracking unit.

[0003] On the other hand, a feature amount that cannot be corresponded is sent to a discovery unit, which uses the uncorresponding point information to detect a new person, and transmits the result of the detection of the new person to the tracking unit. . Then, in the tracking unit, each person information is updated by using the Kalman filter.

[0004]

The image processing represented by the above-described moving object tracking apparatus generally has a high calculation cost, and the tendency becomes more remarkable as the resolution and the number of gradations increase. For example, when processing a square pixel having N pixels on one side, it is generally considered that a calculation cost of O (N ² ) or more is required from the number of pixels referred to.

[0005] On the other hand, as in an online image processing system, there are many cases in which the calculation cost is restricted, such as the need to process a group of images continuously acquired in time series within a limited time. In order to satisfy the above restrictions, simplification has been performed in which the resolution of the image is reduced or a part of the processing is omitted. At that time, as a result of the simplification, the processing accuracy generally decreases. Here, the processing accuracy is assumed to be, for example, the position estimation accuracy in template matching, the correct answer rate of the feature determination process, and the like.

Although simplification of processing is inevitable for realizing online image processing, there is room for improvement in terms of optimal use of computational resources. Conventionally, the degree of simplification of calculation processing has been uniform irrespective of the content of an image or the state of a shooting target, and as a result, dropped frames and excess calculation time have frequently occurred.

Therefore, a main object of the present invention is to select a processing method in consideration of an available calculation time, to effectively use available calculation resources, and to optimize moving picture processing to give the best accuracy. It is an object to provide an apparatus and a moving image processing optimization method.

[0008]

According to the present invention, there is provided a selection means for selecting a processing method based on available calculation time;
And an optimum accuracy providing means for providing an optimum accuracy among calculation resources permitted by the processing method selected by the selecting means.

Further, another invention provides an optimum accuracy among calculation resources allowed by selecting a processing method based on available calculation time.

Further, the present invention is characterized in that an image feature is extracted from a moving image with an optimum accuracy among calculation resources.

Further, the present invention is characterized in that an image feature is extracted from a moving image with an optimum accuracy among calculation resources, and an object is tracked.

[0012]

FIG. 1 is a block diagram of one embodiment of the present invention. The moving image processing optimization device shown in FIG.
The cameras 21, 22,... 2n and the observation units 41, 42,.
, A discovery unit 6 and a tracking unit 8. The observation units 41, 42,... 4n are cameras 21, 22,.
n. Observation units 41, 42 ... 4n
, The discovery unit 6 and the tracking unit 8 can operate independently of each other. For example, they are composed of different computers, and each computer is connected by a local area network LAN.

The symbol A0 indicates the observation information of the corresponding point (tracking target) transmitted from the observation units 41, 42... 4n to the tracking unit 8, and the symbol A1 is transmitted from the observation units 41, 42. A2 indicates observation information of an uncorresponding point (a point which cannot correspond to the tracking target), and a symbol A2 indicates the tracking information from the tracking unit 8 to the observation unit 41,
42n indicates predicted position information transmitted to 4n,
Indicates the position information (initial value) of the new person transmitted from the discovery unit 6 to the tracking unit 8;
Shows the position information (after updating) transmitted to the user.

The observation units 41, 42,... 4n perform feature extraction processing based on input images obtained from corresponding cameras.
Each of the observation units 41, 42,.
The feature amounts (center-of-gravity points and distance conversion values) obtained at 1, 42... 4n are associated with the tracking target based on the predicted position information A2 sent from the tracking unit 8, which will be described later. Is transmitted to the tracking unit 8 together with the information of The information amount for which no correspondence was obtained is transmitted to the discovery unit 6 as observation information A1 of an uncorresponding point.

The discovering unit 6 detects a newly appearing person (new person) in the scene by using the transmitted uncorresponding observation information A1. The detection result of the new person, that is, the position information A3 of the new person is transmitted to the tracking unit 8. Thereby, a new person is added as a new tracking target. Then, tracking is started in the transmission unit 8.

In the tracking section 8, the position information A3 of the new person obtained from the discovery section 6 is used as an initial value, and the observation sections 41, 42
... Using the observation information A0 from 4n as an input value, the position information of the person is updated using the Kalman filter, and the position is predicted based on the observation model. The predicted position information A0 is transmitted to the tracking unit 8. In the discovery unit 6, the position information (after the update) A4 is updated as described later.

FIG. 2 is a diagram showing a human body model used in one embodiment of the present invention. In FIG. 2, X, Y, Z
Indicates three axes of the world coordinate system. The person MAN is modeled by an elliptic cylinder h. The center axis Xh of the person model h is defined as the rotation axis of the person, and the angle r between the normal axis (the axis in the short axis direction perpendicular to the rotation axis Xh) and the X axis is defined as the posture angle of the person. It is assumed that the rotation axis Xh of the person MAN is perpendicular to the floor (the plane defined by the X axis and the Y axis). Camera 2
.., 2n are arranged perpendicular to the rotation axis Xh of the person, that is, the Z axis.

FIG. 3 is a flow chart for explaining the operation of the embodiment of the present invention, and FIG. 4 is a flow chart showing a specific operation of the person detecting process of FIG.

Next, the operation of the embodiment of the present invention will be described with reference to FIGS. Observation units 41, 42 ... 4
n transmits an observation time to the tracking unit 8 in step SP2 when an image signal is input from the corresponding camera in step SP (abbreviated as SP in the figure) 1 shown in FIG. In step SP3, the observation units 41, 42,..., 4n receive, from the tracking unit 8, prediction values such as the number of observed persons, the observed size, and the position and orientation calculated based on the tracking model and the observation time. In step SP4,
The observation units 41, 42,... 4n perform a person area detection process according to the number of observers and the observation size.

More specifically, step SP shown in FIG.
At 41, a required template image type is determined from the number of observers and the respective predicted observation sizes. Then, in step SP42, an optimal approximate template for each template image is selected from available calculation resources. That is, if the required calculation time as the observation size is restricted to, for example, 0.3 second, FIG.
Is selected, and if the required calculation time is restricted to 0.5 second, an approximate template having a medium roughness shown in FIG. 5B is selected, and the required calculation time is relatively large. If set to 9 seconds long, fine
Select the approximate template shown in (c). These approximation templates are stored in the memory in advance.

In step SP43, a person detection process is performed on each selected approximate template by template matching. Here, if the selected approximate template is coarse, the time required for calculating the template matching is short, and if the approximate template is fine, the time required for calculating the template matching is long.
Therefore, the more the time required for the calculation is, the more precise a person image can be extracted.

In step SP44, the detected person image is added to the list. In step SP45, the loop end is determined.
It returns to P43. If it has been completed, step S shown in FIG.
Return to P5.

In step SP5, the observation units 41 and 4
2... 4n determine whether or not there is a human image in the list as the detection result in step SP44. If not, the process returns to step SP1 and returns to steps SP1 to S
The operation of P5 is repeated. If the person image is on the list, the process proceeds to step SP6, where each person region is associated with the tracking model. In step SP7, the observation units 41, 42,..., 4n transmit observation information including the position, the posture, and the person characteristics to the tracking unit 8. In step SP <b> 8, the tracking unit 8 transmits to the discovery unit 6 the position information of the uncorresponding area including the position, the posture, and the person feature.

As described above, by selecting a finer template as the processing calculation time becomes longer, it is possible to obtain highly accurate person information.

Here, assuming that a white Gaussian error occurs in the motion of the target object during tracking, the tracking accuracy was calculated by numerical calculation.

FIGS. 6 and 7 show the relationship between the available calculation time and the image resolution that can be processed when there is only one type of target object. In particular, FIG. FIG. 7 shows the error on the vertical axis and the number of steps on the horizontal axis.

As shown in FIGS. 6 and 7, when the calculation time is shortened and the number of processing steps is reduced, the resolution that can be processed is reduced, and the error is increased so that the processing accuracy is reduced. Conversely, when the calculation time becomes longer and the number of processing steps increases, the resolution that can be processed increases, and the error decreases.

Next, as an example where the available computational resources change, consider the case where the type N of the target object changes. In this case, the calculation time available for each type is 1 / N of that for one type based on the above assumption. For example, when five types of objects are to be tracked, 1/5 is used in a system with a fixed processing method. It is necessary to design a system on the premise of the computational resources. Here, according to the relationship between FIG. 6 and FIG. 7, the processing method (resolution) can be appropriately selected according to the decrease in the available computational resources.

FIG. 8 is a diagram showing the degree of deterioration of the estimation accuracy due to an increase in the number N of types under four different constraint conditions. The vertical axis shows the error, and the horizontal axis shows the number of types. FIG.
, The constraint condition is that the number of processing steps is 400, 60
0, 800 and 1000. It can be seen that the error increases as the number of processing steps decreases as the number of types increases as compared to when the number of types is one.
Therefore, from FIG. 8, it is possible to judge how many types of processing steps should be selected in order to suppress the error and the number of types.

The embodiments disclosed this time are to be considered in all respects as illustrative and not restrictive. The scope of the present invention is defined by the terms of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

[0031]

As described above, according to the present invention, by selecting the processing method based on the available calculation time, the optimum accuracy can be given among the calculation resources allowed, and the available calculation can be performed. Resources can be used effectively and the best accuracy can be given.

[Brief description of the drawings]

FIG. 1 is a block diagram of one embodiment of the present invention.

FIG. 2 is a diagram showing a human body model used in an embodiment of the present invention.

FIG. 3 is a flowchart for explaining the operation of the embodiment of the present invention;

FIG. 4 is a flowchart showing a specific operation of the person detection process of FIG. 3;

FIG. 5 is a diagram illustrating an example of selecting an approximate template.

FIG. 6 is a diagram showing a relationship between available calculation time and a processable image resolution when there is only one type of target object;
The vertical axis indicates the resolution, and the horizontal axis indicates the number of steps.

FIG. 7 is a diagram illustrating a relationship between available calculation time and a processable image resolution when there is only one type of target object;
The vertical axis indicates the error, and the horizontal axis indicates the number of steps.

FIG. 8 shows the number N of types under four different constraints.
FIG. 9 is a diagram showing a degree of deterioration of estimation accuracy due to an increase in the number of the estimation results.

[Explanation of symbols]

21, 22 ... 2n cameras, 41, 42 ... 4n observation units, 6 discovery units, 8 tracking units.

Continued on the front page (72) Inventor Yasuyuki Kado 2-2-2 Kodaidai, Seika-cho, Soraku-gun, Kyoto ATI R & D Co., Ltd. Intelligent Video Communications Laboratory (72) Inventor Kenji Mase Hikari, Seika-cho, Kyoto 2nd-2nd-2nd AT R & Co., Inc. Intelligent Video Communication Laboratory F-term (reference) 5C054 AA01 AA05 CA04 CC02 CH01 EA01 EA03 FA09 FC12 HA18 5L096 BA02 CA02 FA52 FA59 FA67 FA69 HA05 HA09 JA09 JA11 JA13 JA16 LA01

Claims (4)

[Claims] 1. A selecting means for selecting a processing method on the basis of an available calculation time, and an optimum accuracy providing means for providing an optimum accuracy among calculation resources allowed by the processing method selected by the selecting means. A moving image processing optimization device, comprising: 2. A moving image processing optimizing method characterized in that an optimum accuracy is given among calculation resources allowed by selecting a processing method based on available calculation time. 3. The moving image processing optimizing method according to claim 2, wherein an image feature is extracted from the moving image with an optimum accuracy among the calculation resources. 4. The moving image processing optimization method according to claim 2, wherein an image feature is extracted from the moving image with an optimum accuracy in the calculation resources, and the object is tracked.