JP3380937B2 - Method for identifying object in moving image and apparatus for identifying object in moving image - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method of identifying an object in a moving image and an apparatus for identifying an object in a moving image, and an object in a moving image for identifying an object designated by a user while looking at the moving image. And an apparatus for identifying an object in a moving image.

More specifically, in a moving image navigation system or the like that presents information about an object in moving image information specified by a user, an object in a moving image that correctly identifies an object in moving image information specified by a user. And an apparatus for identifying an object in a moving image.

[0003]

2. Description of the Related Art Conventionally, there have been the following methods for identifying an object in a moving image. The conventional method considers a moving image as a three-dimensional space-time consisting of a two-dimensional image and a time axis, sets an area containing the object as management information for each object in the moving image, and The object in the moving image is identified by using the management information to select an object having a position and time pair of the image specified by, that is, an object having a region including the coordinates in the moving image specified by the user (Takano Former, Hiroshi Matoba, Yoshinori Hara: “Navigation method of video and hypermedia”, 8th Symposium on Human Interface, pp
607-612, Oct. 21-23, 1992).

FIG. 10 shows a diagram for explaining a conventional method. 10A shows the time from the time t ₀ to the time t ₄ (t ₀ <t ₄ ) of the triangular object 20 shown in FIG. 10C in the video frame 10 shown in FIG. 10B. elapsed shows a state in which moving image frame 10 with the video frame 10 (coordinates (0 at time t _0,
0,0), (x, 0, t ₀ ), (x, y, t ₀ ),
Five video frames 10 from (quadrangle region having four points of (0, y, t ₀ ) as vertices) to video frame 10 at time t ₄ are arranged in parallel in the time axis (t) direction in ascending order of time. An example of arranging is shown. Object 2 in video frame 10
0 is an object 20 recorded as video information, which is moving from the lower right to the upper left in each video frame 10 as time elapses from time t ₀ to time t ₄ . Further, a quadrangle including the object 20 in each video frame 10 indicates a region in which the object 20 can be identified, and is not recorded as video information but is stored as the management information described above. . FIG. 10D is the same as FIG.
The area in which the object 20 can be identified is shown in each video frame 10 from time t ₀ to time t ₄ shown in 0 (A). This area in which the object 20 can be identified means that when the user points to the object 20 in the moving image while looking at the moving image shown in FIG.
Even if the area is not inside, the area can be identified by the user as having instructed the object 20 if the area within the rectangular area around the object 20 is instructed. Since the position of the object 20 in the video frame 10 changes from the lower right to the upper left with the passage of time, for example, the user instructs the lower right of the object 20 in the video frame 10 at time t ₂ . Even if it is intentional, it takes some time (assuming Δt = t ₃ −t ₂ ) from the time of thinking to instruct to the time of actually instructing. Therefore, when the designated coordinates are input (t ₃ = t ₂ + Δt), the video frame 1 at time t ₃
It may be 0, and the object 20 may not already exist at the specified coordinates in the video frame, and the object 20 intended by the user may not be correctly specified. In such a case, especially the object 20
In order to allow the user to indicate the object 20 as accurately as possible even when the moving speed of the object is fast or the size of the object 20 is small, the actual object for each object 20 in the video frame 10 A range wider than 20 is preset as a region in which the object can be identified.

[0005]

However, the above-mentioned conventional method has the following four problems.

The first problem is that in order for the system to correctly identify the object 20 in the video frame 10 designated by the user, the user must identify each object 20 preset in the system. The point is that you must specify the area within which you can do it. When the user designates coordinates outside even for a region in which the object 20 set in the system can be identified in advance, the system correctly identifies the object 20 as the designated object 20. Therefore, the user must carefully point the object 20.

The second problem is that in the above conventional method, in order to reduce the influence of the first problem, when the number of objects existing in the video frame 10 is small, individual objects are identified. Take as much area as you can
In addition, when the number of objects existing in the video frame 10 is large, it is necessary to make adjustments such as narrowing the area where each object can be identified. Therefore, a large number of steps are required to set the area for each object.

A third problem is that when a fast-moving object is identified, the operation of pointing the object by the user may not catch up with the movement of the object. This is a point that must be set as an area considering the moving speed. Also in this case, many steps are required for setting.

A fourth problem is that the same problem as the third problem occurs even when the moving image reproducing speed is high, so that an object can be identified according to the moving image reproducing speed. The point is that multiple possible areas must be set. This problem also requires many steps for setting like the second problem and the third problem.

The present invention has been made in view of the above points. For each individual object in moving image information, an area in which the object can be identified is set in advance and stored as management information. It is an object of the present invention to provide a method for identifying an object in a moving image and an apparatus for identifying an object in a moving image that can correctly identify an object.

Further, even when the designated coordinates are outside the object or the number of objects in the moving image information is large,
An object of the present invention is to provide a method for identifying an object in a moving image and an apparatus for identifying an object in a moving image that can correctly identify an object intended by a user.

Further, even if the moving speed of the object moving in the moving image information is high, or the moving image information reproducing speed is high, the object intended by the user can be correctly identified. An object is to provide a method for identifying an object in a moving image and an apparatus for identifying an object in a moving image.

[0013]

FIG. 1 shows the principle of the present invention.

The method of identifying an object in a moving image according to the present invention is
The moving image information accumulated in advance is displayed (step 1), and the user indicates the coordinates in the moving image information while looking at the displayed moving image information (step 2).

When the user designates the coordinates in the moving image information, the object is searched from the position information of the object in the moving image information stored in advance (step 3).

The likelihood between the coordinate designated by the user and the object retrieved in step 3 is calculated, and the object in the moving image information corresponding to the coordinate designated by the user is calculated based on the calculated likelihood. Are identified (step 4).

FIG. 2 shows the principle configuration of the present invention.

An apparatus 10 for identifying an object in a moving image according to the present invention.
0 is a moving image while watching the image storage means 110 for storing the moving picture information, the image display means 120 for displaying the moving picture information stored in the image storage means 110, and the moving picture information displayed on the image display means 120. Coordinate pointing means 130 for pointing the coordinates in the image information, position information storage means 140 for storing the position information of each object in the moving image information, and search means 150 for searching the object from the position information storage means 140.
And a likelihood calculating means 16 for calculating a likelihood by associating a real number with a set of each coordinate in the moving image information and position information of an individual object.
A set of 0, the designated coordinate, and the position information of the object searched by the searching means 150 is output to the likelihood calculating means 160 to calculate the likelihood, and the instruction is made based on the calculated likelihood. Identification means 170 for identifying an object corresponding to the coordinate set.

The search means 150 outputs a search area setting means for setting a search area in the moving image information corresponding to each coordinate in the moving image information and the designated coordinates to the search area setting means. In addition, the search area in the search area for the designated coordinates is set, and the search area in the search area in the position information storage means 140 is searched for an object having position information.

The search area setting means sets the search area in the moving image information based on the delay time from the confirmation of the object in the moving image information to the instruction at the designated coordinates.

The likelihood calculating means 160 calculates the distance from the designated coordinates and the position information of the object searched by the searching means 150, and determines the likelihood based on the calculated distance.

[0022]

According to the present invention, by searching for an object based on the designated coordinates in the moving image information and identifying the designated object based on the result of the search, each object in the moving image information is previously identified. It is possible to identify an object without setting and accumulating a region in which the object can be identified.

Further, since the position information in the moving image information is previously stored for each object in the moving image information,
It becomes possible to search for all objects based on the designated coordinates.

Further, by setting a search area for searching an object in the moving image information based on the designated coordinates and searching for the object, it is possible to shorten the search amount and the search time. .

Further, considering the delay time from the confirmation of the object in the displayed moving image information to the actual instruction, a search area for searching for the object in the moving image information is based on the delay time. By setting, fast moving objects,
Even with moving image information or the like having a high reproduction speed, it is possible to search for an object in the moving image information at the time when the coordinates are actually designated.

Further, by calculating the likelihood based on the designated coordinates and the position information of each retrieved object, the object designated by the user can be uniquely identified based on the likelihood.

[0027]

Embodiments of the present invention will now be described in detail with reference to the drawings.

FIG. 3 shows a block diagram of an embodiment of the present invention. The apparatus 100 for identifying an object in a moving image shown in the figure includes an image storage unit 110, an image display unit 120, and a coordinate designating unit 13.
0, position information storage unit 140, search area setting unit 180, likelihood calculation unit 160, and control unit 200.

The image storage unit 110 is a database that stores moving image information, and is specified by the x-axis, y-axis, and t-axis, 0 ≦ x ≦ X, 0 ≦ y ≦ Y, 0 ≦ t ≦. This is moving image information in the range of T, and video frames composed of an x axis and ay axis are accumulated at each time (t).

The image display unit 120 is a display for displaying the moving image information stored in the image storage unit 110.

The coordinate designating unit 130 is a mouse for designating coordinates (x, y, t) in the moving image information displayed on the image display unit 120. While watching the moving image information displayed on the image display unit 120, a certain coordinate (x,
By pressing the button for indicating the position at the position y), the coordinates (x, y, t) can be specified as the coordinates in the moving image information at the time t when the button is pressed. Hereinafter, the coordinates designated by the coordinate designating unit 130 are
Notated as (px, py, pt).

The position information storage unit 140 is the image storage unit 11.
Position information of the center of the object is stored as a representative point for each object in the moving image information stored in 0. The position information of each object is the coordinates (x, y, t) in the three-dimensional space consisting of the two-dimensional coordinates (x, y) in the frame and the time t for each video frame in the moving image information. Is stored as a locus drawn as the moving image information is reproduced.
The position information of the object i in the moving image information stored in the image storage unit 110 is

[0033]

[Equation 1]

The x coordinate and the y coordinate in the moving image information of the object i at time t are expressed as follows.

[0035]

[Equation 2]

The position information of the object i will be described below with reference to the drawings. FIG. 4 is a diagram illustrating the position of the object i at time t according to the embodiment of the present invention. The thick solid line shown in the figure shows the locus of movement of the object i in the video frame from time 0 to time t, and the thick broken line shows the time t of the object i.
The locus of movement after that is shown. The connection point (A) between the thick solid line and the thick broken line indicated by A in the figure is the coordinates (X, 0, t), (X, Y, t), (0, Y, t) at time t,
The position of the object i in the video frame 10 indicated by a quadrangle having four coordinates (0, 0, t) as vertices is shown.
FIG. 5 is a diagram showing a storage example of the position information storage unit of the embodiment of the present invention. The accumulation example of the position information accumulation unit 140 shown in the figure corresponds to the moving image information accumulated in the image accumulation unit 110, and the time, object number, and This is an example of accumulating x-coordinates and y-coordinates as position information of the representative point of the object corresponding to the object number. In the moving image information at time t ₁ , three objects with object numbers 21, 22, and 23 having representative points at the coordinates (3, 7), (12, 7), and (6, 2) are displayed. It has been shown that.

The likelihood calculator 160 is a likelihood calculator L that calculates a likelihood from the coordinates designated by the coordinate designating unit 130 and the position information of the object i.

[0038]

[Equation 3]

Notated as The calculation formula of the likelihood calculator L will be shown in a specific example described later.

The search area setting unit 180 includes a coordinate designating unit 13
A search area setting unit R that sets a search area for searching an object based on the coordinates designated by 0 in the moving image information, and is expressed as R (px, py, pt). The right side of the search area setting unit R will be shown in a specific example described later.

The control unit 200 will be described in detail below with reference to the drawings. FIG. 6 is a block diagram of the control unit according to the embodiment of the present invention. The control unit 200 shown in the figure includes an in-search area search unit 190 and an identification unit 170. The search area search unit 190 reads the moving image information from the image storage unit 110 and displays it on the image display unit 120. Further, when the coordinates in the moving image information are instructed by the coordinate instructing unit 130 while watching the displayed moving image information and the instructed coordinates are input, the instructed coordinates are output to the search area setting unit 180, and the instructed coordinates A search area for searching for an object in the moving image information corresponding to is set. When the search area is set, the object having the position information in the search area in the position information storage unit 140 is searched and output to the identification unit 170.

The identification unit 170 outputs the set of the designated coordinates and the position information of each object output by the search area search unit 190 to the likelihood calculation unit 160 to calculate the likelihood, and the likelihood of the calculated result. The object with the largest value is identified as the object in the moving image information corresponding to the designated coordinates, and the identification result is output.

Below, an apparatus 1 for identifying an object in the moving image is described.
The operation of 00 will be described with reference to the drawings. FIG. 7 is a flow chart showing the operation of the embodiment of the present invention.

(Step 20) The in-search area search unit 190 of the control unit 200 reads out moving image information from the moving image storage unit 110 and displays it on the image table unit 120.

(Step 21) Retrieval area retrieval unit 19
In 0, it is checked whether or not the user has instructed the coordinates in the moving image information while watching the moving image information by the coordinate instruction device 130, and if not instructed, the process proceeds to step 20.

(Step 22) Retrieval area retrieval unit 19
0 is the coordinate (px, py, p) instructed when the user instructed the coordinate in the moving image information in step 21.
t) is output to the search area setting unit 180, and the search area setting unit R sets the search area.

(Step 23) Search Area Search Unit 19
0 is a search area set by the search area setting unit 180, based on the search area, the position information storage unit 160 is searched for an object having position information in the search area, and the position information of the object searched as the search result. Is output to the identification unit 170.

(Step 24) The identification unit 170 of the control unit 200, based on the position information of the object output by the in-search area search unit 190, coordinates (px, p) instructed by the user.
y, pt) and the position information of each object are output to the likelihood calculator 160, and the likelihood calculator L calculates the likelihood.

(Step 25) The identification unit 170 checks whether or not the likelihood calculation of Step 24 has been performed based on the position information of all the objects output from the search area search unit 190 in Step 23. If there is an object for which likelihood calculation is not performed, the process proceeds to step 24.

(Step 26) When the likelihood calculation is completed for all the objects in step 25, the identifying section 170 selects the object having the largest likelihood from the calculated likelihoods of the objects. Then, the selected object is identified as the object corresponding to the coordinates designated by the user, and the identification result is output.

In the following, one embodiment of the present invention will be described.
A specific example of the apparatus 100 for identifying an object in a moving image will be described. In the following description, the area setter R of the area setting unit 180 and the likelihood calculator L of the likelihood calculation unit 160 will be described below.

R (px, py, pt) = {(rx, ry, rt) | 0
≦ rx ≦ X, 0 ≦ ry ≦ Y, rt = pt}

[0053]

[Equation 4]

Further, the apparatus 100 for identifying an object in the main moving image.
While observing the moving image information, at time t ₁ when the moving image information shown in FIG. 8 is displayed, it is assumed that the coordinate indicating device 130 indicates the position of the coordinate indicating cursor 11 indicated by the thick arrow in the figure. , The designated coordinates will be described as (4, 6, t ₁ ). In FIG. 8, r1, r2, and r3 indicate the distance between the instructed coordinate and each of the objects 21, 22, and 23. The value obtained by multiplying the distance by -1, and -r1, -r2, and -r3 are likelihoods. Becomes

Further, it is assumed that the position information storage section 140 stores the position information shown in FIG.

Search area search section 190 of control section 200
However, the moving image information is read from the moving image storage unit 110 and displayed on the image table unit 120 (step 20).

When the in-search area search unit 190 checks whether or not the user has instructed the coordinates in the moving image information while looking at the moving image information by the coordinate instructing device 130, the coordinates of the user are (px, Since py, pt) = (4,6, t ₁ ) has been instructed (step 21, YES), the search area retrieval unit 190 causes the user to instruct the coordinates, (px, py, pt) = (4 , 6, t ₁ ) to the search area setting unit 180, and the search area setting unit R
Where {(rx, ry, rt) | 0 ≦ rx ≦ X, 0 ≦ ry ≦
A search area of Y, rt = t ₁ } is set (step 22).

The in-search area search unit 190 uses the search area set by the search area setting unit 180 as follows: {(rx, ry, rt) | 0 ≦ rx ≦ X, 0 ≦ ry ≦
Y, rt = t ₁ } based on Y, rt = t ₁ }, an object having the position information in the search area is searched from the position information storage unit 160, and the position information of the three objects 21, 22, 23 is searched as the search result,

[0059]

[Equation 5]

The position information thus obtained is identified by the identification unit 17
It is output to 0 (step 23).

The identification unit 170 of the control unit 200, based on the position information of the object output from the search area search unit 190,
The likelihood calculation unit 1 calculates the coordinates (px, py, pt) = (4, 6, t ₁ ) specified by the user and the position information of each object 21, 22, 23.
60, and each of the objects 21, 22, 23 is calculated by the likelihood calculator L.
The following three calculation results are obtained as the likelihood of (step 2
4).

[0062]

[Equation 6]

Since the identifying section 170 has completed the calculation of the likelihood in step 24 based on the position information of all the objects output by the in-search area searching section 190 in step 23 (step 25, YES), The object 21 is selected as the object having the largest likelihood from the likelihoods of the selected objects,
The object 21 is identified as an object corresponding to the coordinates designated by the user (step 26).

In the apparatus 100 for identifying an object in a moving image according to one embodiment of the present invention, the area setting unit R of the area setting unit 180 and the likelihood calculator L of the likelihood calculating unit 160 will be described below. A specific example of another embodiment will be described. The area setting unit R of the area setting unit 180 is set to R (px, py, pt) = {(rx, ry, rt) | 0
≦ rx ≦ X, 0 ≦ ry ≦ Y, rt = pt−Δt}, and the likelihood calculator L of the likelihood calculator 160 is set to

[0065]

[Equation 7]

It is assumed that

Further, the apparatus 100 for identifying an object in the main moving image.
9A while watching the moving image information, at time t ₂ when the image frame 10 ₁ shown by the outer quadrangle shown in FIG. 9A is displayed, the coordinate pointing device 130 causes the broken line in FIG. Position of coordinate indication cursor 12 indicated by arrow (8, 5, t ₂ )
It is assumed that a rectangular object 24 is to be pointed by. However, the objects 23 and 24 shown in the same figure have a high moving speed, and the time from when the user confirms the object 24 in the video frame 10 ₁ of FIG. As a result of the user actually instructing, the result of moving to the right within the delay time (Δt) is shown in FIG.
At time t ₃ (t ₃ = t ₂ + Δt) shown in (B), the position is the position of the coordinate designating cursor 13 indicated by the arrow in the video frame 10 ₃ indicated by the outer quadrangle, and the designated coordinates are (8, 5, t). It will be described as (8, 5, t ₃ ) instead of ₂ ).

Further, it is assumed that the position information storage section 140 stores the position information shown in FIG.

Search area search section 190 of control section 200
However, the moving image information is read from the moving image storage unit 110 and displayed on the image table unit 120 (step 20).

When the in-search area search unit 190 checks whether or not the user has instructed the coordinates in the moving image information while looking at the moving image information by the coordinate instructing device 130, the coordinates of the user are (px, Since py, pt) = (8,5, t ₃ ) is instructed (step 21, YES), the search area search unit 190 causes the search area search unit 190 to specify the coordinates (px, py, pt) = (8 , 5, t ₃ ) to the search area setting unit 180, and the search area setting unit R
Where {(rx, ry, rt) | 0 ≦ rx ≦ X, 0 ≦ ry ≦
A search area of Y, rt = t ₂ } is set (step 22).

The search area in-search section 190 has a search area set by the search area setting section 180: {(rx, ry, rt) | 0 ≦ rx ≦ X, 0 ≦ ry ≦
Y, rt = t ₂ } based on Y, rt = t ₂ }, the object having the position information in the search area is searched from the position information storage unit 160, and the position information of the two objects 23 and 24 is obtained as the search result.

[0072]

[Equation 8]

The position information thus obtained is identified by the identification unit 17
It is output to 0 (step 23).

The identification unit 170 of the control unit 200, based on the position information of the object output by the in-search area search unit 190,
The coordinates instructed by the user, (px, py, pt) = (8, 5, t ₃ ) and the position information of each of the objects 23 and 24 are output to the likelihood calculation unit 160, and each of them is calculated by the likelihood calculator L. The following two calculation results are obtained as the likelihoods of the objects 23 and 24 (step 24).

[0075]

[Equation 9]

Since the identifying section 170 has completed the calculation of the likelihood of step 24 based on the position information of all the objects output by the in-search area search section 190 in step 23 (step 25, YES), The object 24 is selected as the object having the largest likelihood from the likelihoods of the selected objects,
The object 24 is identified as an object corresponding to the coordinates designated by the user (step 26).

When the moving speed of the object in the moving image information is high as in the other embodiments, the coordinates obtained as a result of the user operating the coordinate instructing section 130 to instruct the object 24 have the coordinates intended by the user. coordinates (8,5, t ₂₎ rather than, Delta] t slow, sometimes becomes the coordinates (8,5, t _3).
However, according to the other embodiment described above, even when the user does not catch up with the moving speed of the object and there is a delay time (Δt) from when the user tries to instruct the object until when the user actually instructs the object, the input is performed. It becomes possible to identify the object based on (8,5, t ₂ ) considering the delay time instead of the designated coordinate (8,5, t ₃ ), and it is possible to correctly identify the object intended by the user. . The delay time Δt can be set according to the user, the moving speed of the object in the moving image information, the reproduction speed of the moving image information, and the like, and by using an optimum value in each case, the object Can be correctly identified.

Further, when the reproduction speed (display speed) of the moving image information is changed in the other embodiment, the object can be correctly identified by multiplying Δt by the reproduction speed ratio.

Further, according to the above-described embodiment, even if the user's coordinate pointing operation is delayed, the user can change the likelihood calculator L so that the user's designated coordinates are included in the moving image information. It becomes possible to correctly identify the object in the vicinity of the object.

In the above embodiment, the position information storage unit 14
One representative point of an object is stored as the position information of each object to be stored in 0. However, by setting a plurality of position information for one object, a plurality of objects may overlap each other. However, it is possible to identify correctly.

Further, the search area setting unit 180 of the above embodiment
In the search area setting device R, the example in which the moving image information at a certain time is set as the search area is shown. However, by setting the search area having a width in the time axis direction, the moving image information is moved in a specific direction. It is also possible to selectively identify an object that does.

In the likelihood calculator L of the likelihood checking section 160 of the above embodiment, the likelihood between the designated coordinates and the object is x.
Although an example of calculating based on the distance on the two-dimensional plane composed of the axes and the y-axis has been shown, by calculating based on the distance in the three-dimensional space including the t-axis in addition to the x-axis and the y-axis, Even when a search area having a width on the time axis is set, it is possible to identify the object based on the calculation of the likelihood.

Further, in the above embodiment, the coordinate designating section 130 is used.
As an example, a mouse is used as the image display unit 12.
If it is possible to instruct arbitrary coordinates in the moving image information displayed at 0, the coordinates may be instructed based on the line of sight of the user such as an eye camera.
30 is not limited to mice.

In the above embodiment, the center point of the object is used as the position information of each object. However, if the point is a predetermined point representative of the object, the center point may not be the center point. Of course, the position information is not limited to the center point.

Further, in the above embodiment, the in-search area search unit 190 of the control unit 200 is provided with the function of reading moving image information from the image information storage unit 110 and displaying it on the image display unit 130. It is also possible to provide a moving image information reproducing unit or the like for reading moving image information from the information storage unit 110 and displaying it on the image display unit 130, and it is limited to displaying the moving image information by the search area search unit 190. is not.

Further, in the present embodiment, an example has been shown in which a specific area setter R and likelihood calculator L are set in the area setting section 180 and the likelihood calculating section 160 to identify an object. It is also possible to set a plurality of region setting units R and likelihood calculating units L in the unit 180 and the likelihood calculating unit 160, respectively, and select them appropriately to identify an object. The area setter R and the likelihood calculator L set in the unit 160 are not limited to one.

[0087]

As described above, according to the present invention, the object is searched based on the coordinates designated by the user, and the object corresponding to the coordinates designated by the user is identified from the searched objects. , If the user indicates the outside of the object or the object in the moving image information without setting and accumulating the area in which the object can be identified for each individual object in the moving image information in advance. Even if the number is large, the object corresponding to the coordinates designated by the user can be correctly identified.

Further, since the position information in the moving image information for each object in the moving image information is stored in advance, the object based on the designated coordinates can be searched at high speed and reliably. It is possible to accurately identify the object corresponding to the coordinates designated by the person.

Further, based on the coordinates designated by the user,
By setting a search area for searching an object in the moving image information and performing a search, it is possible to reduce the search amount and the search time of the object.

Also, in consideration of the delay time from when the user confirms the object in the displayed moving image information until when the user actually gives an instruction, the search area for searching the object is moved past the moving image by the delay time. By setting the information, it is possible to correctly identify the object even if it is a fast-moving object or moving image information having a high reproduction speed.

Further, the distance is calculated from the instructed coordinates and the position information of each searched object, and the likelihood is determined based on the calculated distance. An object most likely to be identified as the instructed object can be identified based on the likelihood.

Further, with the above effect, an area for identifying an object in the moving image information is created in advance in consideration of the size of the object, the number of objects, the reproduction speed of the moving image information, the reaction speed of the user, and the like. Since it is possible to omit many steps such as the above, it is possible to reduce the time and cost required for the production.

Further, for a person who uses the apparatus for identifying an object in a moving image, even if the object is a fast-moving object or a small object, it is possible to easily specify the target object without carefully instructing the object. It will be possible.

[Brief description of drawings]

FIG. 1 is a diagram illustrating the principle of the present invention.

FIG. 2 is a principle configuration diagram of the present invention.

FIG. 3 is a configuration diagram of an embodiment of the present invention.

FIG. 4 is a diagram illustrating a position of an object i at time t according to an embodiment of the present invention.

FIG. 5 is a diagram showing a storage example of a position information storage unit according to an embodiment of the present invention.

FIG. 6 is a configuration diagram of a control unit according to an embodiment of the present invention.

FIG. 7 is a flowchart showing the operation of one embodiment of the present invention.

FIG. 8 is a diagram (1) illustrating identification of an object according to an embodiment of the present invention.

FIG. 9 is a diagram (2) explaining identification of an object according to an embodiment of the present invention.

FIG. 10 is a diagram illustrating a conventional method.

[Explanation of symbols]

10 video frames 11 Coordinate pointing cursor 12 Coordinate pointing cursor 13 Coordinate pointing cursor 20 objects 21 objects 22 objects 23 objects 24 objects 30 areas 100 Object identification device in moving image 110 image storage means, image storage section 120 image display means, image display section 130 Coordinate instruction means, coordinate instruction section 140 Position Information Storage Means, Position Information Storage Unit 150 Search method 160 Likelihood calculator, likelihood calculator 170 Identification means, identification section 180 Search area setting section 190 Search area search section 200 Control unit

Front page continuation (72) Inventor Ryoji Kataoka 1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Inside Nippon Telegraph and Telephone Corporation (56) Reference JP-A-5-189153 (JP, A) JP-A-4-352076 (JP, A) JP-A-3-52070 (JP, A) Ryoji Kataoka and 2 others, Multimedia Information System based on Visual Guidance Information Acquisition Model-VideoReality-, Information Processing Society Research Report (94-DBS-97 ), Japan, IPSJ, March 18, 1994, Volume 94, No. 30, PP. 21-30 Seiichi Konya and 3 others, Management model of moving image objects in multimedia information system, Proc. Of the 49th National Conference of IPSJ (6th semester) (3), Japan, IPSJ , September 30, 1994, 331-332 Yoshinori Hara and 7 others, Outline of hypermedia platform "Miyabi", Research Report of Information Processing Society of Japan (92-DBS-90), Japan, Information Processing Society of Japan, September 11, 1992, Vol. 92, No. 71, PP. 29-38 Tetsuji Sato and 2 others, Video Reality: Advanced Information Retrieval Method Using Video, Information Processing Society of Japan Research Report (94-DBS-99), Japan, Information Processing Society of Japan, July 22, 1994, Vol. 94, No. 62, pp. 281-286 (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06F 17/30 G06T 1/00 G06T 7/00 JISST file (JOIS)

Claims (8)

(57) [Claims] 1.In the method of identifying an object in a moving image, A step of displaying the moving image information accumulated in advance, While watching the displayed moving image information, the position in the moving image information
The step of indicating the mark, Is the position information of the object in the moving image information stored in advance?
A search step to search for objects from The designated coordinates and the object retrieved by the retrieval step
The likelihood with the body is calculated, and the finger is calculated based on the calculated likelihood.
An identification step to identify the object corresponding to the indicated coordinates and
HaveThen The search step is Search in the moving image information based on the designated coordinates
Search area setting step to set the area, Position information of the object in the moving image information stored in advance
Search area that searches for objects located in the search area
With internal search step In a moving image characterized by
Method of object identification. 2. The search area has a predetermined width in the time axis direction.
The method for identifying an object in a moving image according to claim 1, which is an area to be held . 3. The search area setting step includes, based on the designated coordinates, the moving image information that is past by a delay time from when the object in the moving image information is confirmed to when the object is designated, Claim set as a search area
1. A method for identifying an object in a moving image according to 1 . 4. The identifying step calculates the likelihood based on a distance between the designated coordinates and position information of an object obtained as a result of the search, and the calculated likelihood becomes maximum. The method for identifying an object in a moving image according to claim 1, wherein the object is identified as an object corresponding to the designated coordinates. 5.In the identification device of the object in the moving image, An image storage means for storing moving image information, An image for displaying the moving image information stored in the image storage means.
Image display means, While watching the moving image information displayed on the image display means,
Coordinate indicating means for indicating the coordinates in the moving image information, Positions for accumulating position information of individual objects in the moving image information
Information storage means, Search means for searching an object from the position information storage means
When, In the set of each coordinate in the moving image information and the position information of each object
On the other hand, a likelihood calculating means that associates a real number with the likelihood, The designated coordinates and the object retrieved by the retrieval means
The pair with the position information is output to the likelihood calculation means to calculate the likelihood.
Calculated and the indicated coordinates based on the calculated likelihood
And an identification means for identifying an object corresponding toThen The search means is Search in the moving image information based on the designated coordinates
Search area setting means for setting an area, The position information is stored in the search area in the position information storage means.
A search area search means for searching an object To do
An apparatus for identifying an object in a moving image, characterized by. 6. The search area has a predetermined width in the time axis direction.
The apparatus for identifying an object in a moving image according to claim 5, wherein the area is a region having a . 7. The search area setting means is configured to perform the search in the moving image information based on a delay time from confirmation of an object in the moving image information to instruction of the specified coordinate. The apparatus for identifying an object in a moving image according to claim 5, wherein an area is set. 8. The likelihood calculating means calculates a distance from the designated coordinates and the position information of the object searched by the searching means, and the distance to the designated coordinates is calculated based on the calculated distance. The apparatus for identifying an object in a moving image according to claim 5, which determines the likelihood.