JP2003099783A - Animation processing method, its device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an animation processing method, its method, and a program for it capable of extracting a moving body such as a human from a photographed/formed animation and the like, tracing the extracted moving body, and associating the coordinate selected from the animation with the moving body. SOLUTION: A frame is extracted from the animation by an image extraction part 31, while the moving body is extracted by a moving body area extraction part 32 on the basis of differential information of one frame or each of a plurality of frames. Related information is set to the moving body by a moving body information registration part 33, and the setting is controlled by a related information management part 34. Then, respective moving bodies are traced by a clickable information management part 35 on the basis of the characteristic quantity of each moving body.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image processing method, apparatus and program, and more particularly to a moving image processing method, apparatus and program for tracking a moving person or object in a moving image such as video or animation. .

[0002]

2. Description of the Related Art In recent years, along with the progress of digitization in cable television, satellite broadcasting, terrestrial broadcasting, etc., the Internet has entered the broadband era, and various moving images can be viewed easily and with high quality. Came.

In addition, as represented by the Internet, there are many things that allow two-way communication between a distributor of information (moving images and the like) and a viewer, and the viewer simply watches the moving images and the like. Instead, there is an increasing number of opportunities to acquire and browse related information.

[0004] For example, at present, web pages describing information relating to distributed dramas, movies, variety programs, etc. are published via the Internet, and viewers can easily obtain favorite information. Is able to.

[0005]

However, at the present time,
Most of the information that has been distributed and released is on a program-by-program basis such as dramas and movies, and even if the viewer wants to obtain information about a specific actor, etc. It is necessary to confirm the actor name and acquire the information about the desired actor again.

[0006] When a viewer wants to directly obtain information on a specific actor or the like, if he / she knows the actor name and the URL where the information is disclosed, he / she can obtain the desired information relatively easily. It is possible, but it becomes difficult if you do not know the actor name.

[0007] Also, if the viewer of the moving image or the like knows what kind of information the viewer wants, the corresponding information can be distributed or the URL where the information is disclosed can be notified. Becomes However, in a program such as a drama or a movie, usually, a plurality of actors appear, and it is difficult to determine which actor the viewer desires.

Therefore, the present invention extracts a moving body such as a person from a moving picture taken such as a drama, movie, variety, or a moving picture created like animation, and tracks the extracted moving body. It is an object of the present invention to provide a moving image processing method, device, and program capable of associating a moving object with a coordinate selected from a moving image.

[0009]

In order to achieve the above-mentioned object, the invention of claim 1 is a moving image processing method for tracking a moving object in a moving image in a moving image captured or created. Difference information is acquired for each predetermined number of frames from a plurality of frames constituting the, and the moving object is specified from the acquired difference information, and the position information in the moving image of the specified moving object is managed. To do.

Further, in the invention of claim 2, in the invention of claim 1, when one or more moving bodies are specified from the difference information, the identity is judged for each moving body, and the judgment result is obtained. The position information in the moving image of the mobile body is managed based on the above.

Further, the invention of claim 3 is characterized in that, in the invention of claim 2, the determination of the identity is performed based on a characteristic amount of the moving body.

The invention according to claim 4 is characterized in that, in the invention according to claim 2, the determination of the identity is performed based on a moving distance of the moving body.

Further, the invention of claim 5 is characterized in that, in the invention of claim 2, the determination of the identity is performed based on a feature amount and a moving distance of the moving body.

The invention according to claim 6 is characterized in that, in the invention according to claim 1, the difference information is acquired for each frame from the moving image.

According to a seventh aspect of the present invention, in the first aspect of the present invention, the position information is managed by associating the related information of the moving body corresponding to the position information with each other, and managing the position of the moving body from the moving image. When the position corresponding to the information is selected, the related information corresponding to the selected position information is provided.

According to the invention of claim 8, in a moving image processing apparatus for performing tracking processing of a moving object in a moving image in a moving image captured or created, a predetermined number of frames from a plurality of frames forming the moving image. A moving body identifying unit that obtains the difference information and identifies the moving body from the obtained difference information, and a position information managing unit that manages the position information in the moving image of the moving body identified by the moving body identifying unit. It is characterized by doing.

Further, in a ninth aspect of the invention, in the eighth aspect of the invention, when the moving body specifying means specifies one or more moving bodies, the moving body judging means for judging the identity of each moving body. Further, the position information managing means manages the position information in the moving image of the moving body based on the judgment result of the moving body judging means.

Further, the invention of claim 10 is characterized in that, in the invention of claim 9, the moving body judging means judges the identity based on a characteristic amount of the moving body.

The invention of claim 11 is characterized in that, in the invention of claim 9, the moving body judging means judges the identity based on a moving distance of the moving body.

Further, the invention of claim 12 is characterized in that, in the invention of claim 9, the moving body judging means judges the identity based on a characteristic amount and a moving distance of the moving body. To do.

The invention of claim 13 is characterized in that, in the invention of claim 8, the moving body specifying means acquires the difference information for each frame from the moving image.

According to a fourteenth aspect of the present invention, in the eighth aspect of the present invention, the relevant information of the moving body corresponding to the position information managed by the position information managing means is managed in association with the moving information from the moving image. When the position corresponding to the position information of the body is selected, the related information management means for providing the related information corresponding to the selected position information is further provided.

According to a fifteenth aspect of the present invention, in a moving image processing program for causing a computer to perform a tracking process of a moving object in a moving image captured or created, difference information is obtained from a plurality of frames forming the moving image every predetermined number of frames. Managing the position information in the moving image of the moving body specified in the second step, and the second step of specifying the moving body from the difference information acquired in the first step. And a third step of

According to a sixteenth aspect of the present invention, in the fifteenth aspect of the present invention, when one or more moving bodies are specified in the second step, the identity is judged for each moving body. The method further comprises steps, wherein the third step manages the position information of the moving body in the moving image based on the determination result of the fourth step.

Further, the invention of claim 17 is characterized in that, in the invention of claim 16, the fourth step determines the identity based on a characteristic amount of the moving body.

The invention of claim 18 is characterized in that, in the invention of claim 16, the fourth step determines the identity based on a moving distance of the moving body.

The invention of claim 19 is characterized in that, in the invention of claim 16, the fourth step determines the identity based on a feature amount and a moving distance of the moving body. To do.

According to a twentieth aspect of the present invention, in the fifteenth aspect of the present invention, the first step is from the moving image.
The difference information is obtained for each frame.

According to a twenty-first aspect of the present invention, in the fifteenth aspect of the invention, a fifth step of managing the position information by associating the relevant information of the moving body corresponding to the position information, And a sixth step of providing related information corresponding to the selected position information when a position corresponding to the position information of the mobile body is selected.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of a moving image processing method, apparatus and program according to the present invention will be described in detail below with reference to the accompanying drawings.

[Embodiment 1] FIG. 1 is a block diagram showing a functional configuration of a moving body tracking device to which the present invention is applied. The moving body tracking device 3 shown in the figure may be configured as dedicated hardware, but may be configured as software that operates on a computer.

As shown in the figure, the moving body tracking device 3 is
The image extraction unit 31, the moving body region extraction unit 32, the moving body information registration unit 33, the related information management unit 34, and the clickable information management unit 3
5, a clickable map generation unit 36, and a moving image distribution unit 37.

The image extraction unit 31 is provided for the video camera 1 and the VT.
The moving image input from R2 or the like is cut out frame by frame, and each frame is converted into a bitmap image represented by each color of RGB. The moving body region extraction unit 32 calculates the difference between the images of two consecutive frames extracted by the image extraction unit 31 and extracts a rectangular region including a region where the difference exceeds a certain threshold as a moving body. However, the calculation of the difference does not necessarily have to be performed for each frame, and may be performed for every several frames.

The moving body information registration unit 33 stores information related to the moving body, for example, a URL (UniformResource).
GUI (Grap) for setting Locator
Hical User Interface). The related information management unit 34 manages a set of the characteristic amount of the moving body and the related information set for the moving body.

The clickable information management unit 35 tracks and manages the area information of the mobile body managed by the related information management unit 34 based on the feature quantity of the mobile body sequentially extracted from the moving body area extraction unit 32. The clickable map generation unit 36 generates a clickable map based on the area information managed by the clickable information management unit 35. The clickable map generated by the clickable map generation unit 36 operates on the client side.

The moving picture distribution section 37 generates a moving picture in which the clickable map generated by the clickable cup generation section 36 is associated with each frame extracted by the image extraction section 31, and is distributed to the clients connected via the network. To deliver.

Next, the operation of the moving body tracking device 3 will be described. First, the registration process of the related information for the moving body will be explained. FIG. 2 is a flowchart showing a flow of registration processing of related information by the mobile tracking device 3.

The moving body tracking device 3 includes an image extracting unit 31 for extracting a moving image input from the video camera 1 or the VTR 2 or the like.
Cuts out each frame, and the moving body extraction unit 32 extracts the moving body from the difference information of consecutive frames (step 101), and acquires the moving body area including the extracted moving body and the feature amount of the moving body. (Step 102). Then, the moving body extraction unit 32 presents the moving body region on a display unit (not shown) (step 103).

Here, when the moving body area presented by the moving body extraction unit 32 is selected by the registrant (YE in step 104).
S), the moving body information registration unit 33 displays a registration screen (GUI) on the display unit (not shown) (step 105). There may be a plurality of moving body regions presented by the moving body extraction unit 32, and the registrant selects a moving body region with a pointing device (not shown) such as a mouse. For example, when the screen 200 as shown in FIG. 3 is displayed on the display unit (not shown) and the moving body area 201 and the moving body area 202 are presented in the screen 200, the registrant moves the pointer 203 to the moving body area 201. It is assumed that the mouse is operated. By this operation, the moving body information registration unit 33 displays the registration screen 204.

Subsequently, when the registrant inputs relevant information such as URL on the registration screen (YES in step 106), the moving body information registration section 33 relates the inputted relevant information with the feature amount of the moving body as a set. Register in the information management unit 34. At this time, if the mobile body is already registered (YES in step 107), the registered information is rewritten (step 108), and if it is not registered yet (NO in step 107). , Is newly registered (step 109).

Information registered in the related information management section 34 will be described below. As the information registered in the related information management unit 34, as shown in FIG. 4, the related information in which the moving body feature amount representing the feature amount of the moving body is input by the registrant is managed as a group, and the moving body group is managed in each group. An identifier is attached.
The moving body identifier is for identifying each moving body and is unique within each moving image. The moving body feature amount represents the feature of the moving body, and is calculated by, for example, color (RGB value), brightness, shape, size, or the like.

Next, the moving body tracking processing by the moving body tracking device 3 will be described. FIG. 5 is a flowchart showing the flow of the moving body tracking processing by the moving body tracking device 3.

When the moving body tracking device 3 starts the moving body tracking process, first, the information registered in the clickable information management section 35 is initialized (step 111). Then, the moving body extracted by the moving body area extraction unit 32 is detected (step 112). Here, if there is an unprocessed moving body detected (YES in step 113), a moving body identifier is acquired from the related information information management unit 34 based on the feature amount of the moving body (step 114), The related information is acquired (step 115). Then, the moving body region extraction unit 3
The moving body area including the moving body is acquired from 2 (step 11
6) The acquired moving body area and related information are paired and additionally registered in the clickable information management unit 35 (step 11).
7). The information registered in the clickable information management unit 35 is as shown in FIG. 6, for example.

The processing from step 114 to step 117 is repeated while there is an unprocessed moving object detected in step 112 (Y in step 113).
ES), if processing is performed on all the detected moving bodies (NO in step 113), the moving body tracking processing is ended.

The moving body tracking processing described here is
It is executed for each one or a plurality of frames of the moving image (every time the moving body region extraction unit 32 extracts the moving body from the difference information). That is, the information registered in the clickable information management unit 35 is updated every time the moving body in the moving image moves.

Next, a moving image distribution process by the moving body tracking device 3 will be described. FIG. 7 is a flowchart showing a flow of moving image distribution processing by the moving body tracking device 3.

In the moving image distribution process, first, the clickable map generator 36 initializes the clickable map (step 121). Then, the clickable map generation unit 36 acquires the information registered in the clickable information management unit 35 (step 122) and creates the clickable map based on the acquired information (step 1).
23). When the clickable map is created, the moving image delivery unit 37 delivers the clickable map to the client together with a part (one or a plurality of frames) of the moving image (step 124).

The clickable map generation unit 36 generates the clickable map for each one or a plurality of frames of the moving image (every time the moving body region extraction unit 32 extracts the moving body from the difference information).
Since the related information such as the URL is included, the client side receiving the clickable map together with the moving image acquires the set related information by selecting a desired moving object in the moving image with a mouse or the like. be able to.

For example, if the moving image distributed to the client is displayed as the screen 250 shown in FIG. 8, the operator (viewer) of the client operates the mouse or the like to move the pointer 251 in the screen 250. When an operation such as clicking is performed according to the moving body, the client acquires a web page or the like based on the related information (URL) set in the moving body, and displays it as a screen 252.

[Embodiment 2] In the above-described Embodiment 1, the clickable map to be distributed together with the moving image operates on the client side. However, in this Embodiment 2, the clickable map to be distributed is distributed to the server (moving object tracking). A case where a device operating on the device side is used will be described.

FIG. 9 is a block diagram showing the functional structure of a moving body tracking device according to the second embodiment of the present invention. The moving body tracking device 4 shown in the figure may be configured as dedicated hardware, but may be configured as software that operates on a computer.

As shown in the figure, the moving body tracking device 4 is
The image extraction unit 41, the moving body region extraction unit 42, the moving body information registration unit 23, the related information management unit 44, and the clickable information management unit 4
5, video distribution unit 46, click area analysis unit 47, HTM
An L document generation unit 48 and an HTML document distribution unit 49 are provided and configured.

The image extracting section 41 is provided for the video camera 1 and the VT.
The moving image input from R2 or the like is cut out frame by frame, and each frame is converted into a bitmap image represented by each color of RGB. The moving body region extraction unit 42 calculates a difference between the images of two consecutive frames extracted by the image extraction unit 41, and extracts a rectangular region including a region where the difference exceeds a certain threshold as a moving body. However, the calculation of the difference does not necessarily have to be performed for each frame, and may be performed for every several frames.

The moving body information registration unit 43 provides a GUI for setting related information in the moving body. Related Information Management Department 4
Reference numeral 4 manages a set of the characteristic amount of the moving body and the related information set in the moving body. The clickable information management unit 45 tracks and manages the area information of the mobile body managed by the related information management unit 44 based on the feature amount of the mobile body sequentially extracted from the moving body area extraction unit 42.

The moving image distribution unit 46 associates the clickable map with the moving image acquired from the image extraction unit 41 and distributes it to the clients connected via the network. The clickable map to be associated here is one that is processed on the server (moving object tracking device 4) side, that is, when there is an operation such as a click on the client side, the coordinates thereof are returned to the server side. Therefore, the clickable maps to be associated are always the same.

The click area analysis unit 47 acquires the value of the coordinate transmitted when the arbitrary coordinate on the moving image (clickable map) is clicked on the client side. H
The TML document generation unit 48 identifies the selected moving body based on the coordinate values acquired by the click area analysis unit 47 and the region information of the moving body managed by the clickable information management unit 45, and identifies the selected moving body. An HTML document for displaying the set related information is generated. HTML document generator 4
The document generated by 8 is a document in which a web page managed by the mobile tracking device 4 or another web server is acquired, or a redirect instruction to the web page is described.
This is for displaying information (information other than URL) directly input as related information.

The HTML document delivery section 49 delivers the HTML document generated by the HTML document generation section 48 to the client.

Next, the operation of the moving body tracking device 4 will be described. The registration processing of the related information and the moving body tracking processing are the same as in the case of the above-described first embodiment (see FIGS. 2 and 5). However, the image extraction unit 3 in the description of the first embodiment
1 and the moving body region extraction unit 32, the moving body information registration unit 33, the related information management unit 34, and the clickable information management unit 35, respectively, the image extraction unit 41, the moving body region extraction unit 42, the moving body information registration unit 23, and the related information management unit 44. , And is replaced by the clickable information management unit 45.

The moving image distribution process by the moving body tracking device 4 is a process in which the moving image distribution unit 46 only distributes the moving image acquired from the image extraction unit 41 to the client in association with the clickable map as described above. .

Next, the process of providing the related information by the moving body tracking device 4 will be described. FIG. 10 is a flowchart showing a flow of related information providing processing in the moving body tracking device 4.

When the click area analysis unit 47 acquires the coordinates selected by the client (YES in step 131), the moving body tracking device 4 sets the acquired coordinates in HTML.
It is passed to the document generation unit 48. The HTML document generation unit 48 checks whether the related information corresponding to the coordinates passed from the click area analysis unit 47 is managed by the clickable information management unit 45. Here, when the related information corresponding to the coordinates exists (YES in step 132), the HTML document generation unit 48 acquires the related information from the clickable information management unit 45 (step 133) and generates the HTML document. (Step 134). Then, the HTML document delivery unit 49 delivers the generated document to the client (step 135).

[Third Embodiment] In the first and second embodiments described above, the feature amount of each moving body is used for tracking the moving body. In this third embodiment, however, the area is moved for tracking the moving body. The case of using the distance will be described. In the third embodiment, a case where a clickable map to be processed on the client side is distributed to a client as a moving image distribution destination (corresponding to the above-described first embodiment) and a case where a clickable map to be processed on the server side is distributed. Both (corresponding to the second embodiment described above) will be described.

FIG. 11 is a block diagram showing the functional arrangement of a mobile tracking device in the case of delivering a clickable map to be processed on the client side in the third embodiment. The mobile object tracking device 5 shown in the figure may be configured as dedicated hardware, but may be configured as software that operates on a computer.

As shown in the figure, the moving body tracking device 5 is
The image extraction unit 51, the moving body region extraction unit 52, the moving body information registration unit 53, the related information management unit 54, the clickable map generation unit 55, and the moving image distribution unit 56 are configured.

The image extracting section 51 is provided for the video camera 1 and the VT.
The moving image input from R2 or the like is cut out frame by frame, and each frame is converted into a bitmap image represented by each color of RGB. The moving body region extraction unit 52 calculates the difference between the images of two consecutive frames extracted by the image extraction unit 51, and extracts a rectangular region including a region where the difference exceeds a certain threshold as a moving body. However, the calculation of the difference does not necessarily have to be performed for each frame, and may be performed for every several frames.

The moving body information registration section 53 provides a GUI for setting related information to the moving body. Related Information Management Department 5
Reference numeral 4 manages a set of coordinates or a region of the moving body and related information set in the moving body. The clickable map generation unit 55 generates a clickable map based on the area information managed by the related information management unit 54. The clickable map generated by the clickable map generation unit 55 is
It works on the client side.

The moving image distribution unit 56 generates a moving image in which the clickable map generated by the clickable cup generation unit 55 is associated with each frame extracted by the image extraction unit 51, and is distributed to the clients connected via the network. To deliver.

FIG. 12 is a block diagram showing the functional arrangement of a mobile tracking device in the case where a clickable map to be processed on the server side is distributed in the third embodiment. The moving body tracking device 6 shown in the figure may be configured as dedicated hardware, but may be configured as software that operates on a computer.

As shown in the figure, the moving body tracking device 6 is
The image extracting section 61, the moving body area extracting section 62, the moving body information registering section 63, the related information managing section 64, the moving image delivering section 65, the click area analyzing section 66, the HTML document generating section 67, and the HTML document delivering section 68 are provided. To be done.

The image extracting section 61 is provided for the video camera 1 and the VT.
The moving image input from R2 or the like is cut out frame by frame, and each frame is converted into a bitmap image represented by each color of RGB. The moving body region extraction unit 62 calculates a difference between the images of two consecutive frames extracted by the image extraction unit 61, and extracts a rectangular region including a region where the difference exceeds a certain threshold as a moving body. However, the calculation of the difference does not necessarily have to be performed for each frame, and may be performed for every several frames.

The moving body information registration unit 63 provides a GUI for setting related information to the moving body. Related Information Management Department 6
Reference numeral 4 manages a set of coordinates or a region of the moving body and related information set in the moving body.

The moving picture distribution unit 65 associates the moving picture acquired from the image extraction unit 61 with the clickable map and distributes it to the client connected via the network. The clickable map to be associated here is one that is processed on the server (moving object tracking device 6) side, that is, when there is an operation such as a click on the client side, the coordinates thereof are returned to the server side. Therefore, the clickable maps to be associated are always the same.

The click area analysis unit 66 acquires the value of the coordinates transmitted when any coordinates on the moving image (clickable map) are clicked on the client side. H
The TML document generation unit 67 specifies the selected moving body based on the coordinate value acquired by the click area analysis unit 66 and the area information of the moving body managed by the related information management unit 64. H to display the set related information
Generate a TML document. The document generated by the HTML document generation unit 67 is directly input as related information when a web page managed by the mobile tracking device 6 or another web server is acquired, a redirect instruction to the web page is described, and related information. It is for displaying information (information other than URL).

The HTML document distribution unit 68 distributes the HTML document generated by the HTML document generation unit 67 to the client.

Next, the operation of the moving body tracking device 5 and the moving body tracking device 6 will be described. The moving image distribution process by the moving body tracking device 5 is the same as the moving image distribution process in the first embodiment (see FIG. 7). Similarly, the moving image tracking process and the related information providing process performed by the mobile tracking device 6 are performed in the second embodiment.
Since it is the same as the moving image distribution process and the related information providing process (see FIG. 10) in FIG.
Only the related information registration processing and the mobile body tracking processing will be described. However, the clickable information management unit 35, clickable map generation unit 36, and moving image distribution unit 37 in the description of the first embodiment should be read as the related information management unit 54, clickable map generation unit 55, and moving image distribution unit 56, respectively. The image extraction unit 41, the clickable information management unit 45, the moving image distribution unit 46, the click area analysis unit 47, the HTML document generation unit 48, and the HTML document distribution unit 49 in the description of Example 2 are respectively an image extraction unit 61 and related information management. Part 6
4, video distribution unit 65, click area analysis unit 66, HTM
The L document generation unit 67 and the HTML document distribution unit 68 are to be replaced.

FIG. 13 is a flow chart showing a flow of registration processing of related information by the mobile unit tracking device 5 and the mobile unit tracking device 6.

When the registrant selects arbitrary coordinates from the moving image input from the video camera 1 or the VTR 2 with the pointing device such as a mouse (YES in step 301), the moving body tracking device 5 (6) moves the moving body. The information registration unit 53 (63) displays a registration screen (G
UI) is displayed (step 302). The registration screen is displayed as a screen 400 as shown in FIG. 14, for example. When the registrant moves the pointer 401 to a desired place in the screen 400 and operates the mouse, the moving body information registration unit 53 (63) displays the registration screen 402.

Subsequently, when the registrant inputs relevant information such as URL on the registration screen (YES in step 303), the moving body information registration section 53 (63) sets the inputted relevant information as a pair with the selected coordinate. And registers it in the related information management unit 54 (64).

Information registered in the related information management section 54 (64) will be described below. Related information management unit 54
As shown in FIG. 15, the information registered in (64) is managed as a set of coordinates or a region of the moving body (coordinates are converted into a region by a tracking process described later) and related information input by the registrant. The moving body identifier is attached to each set. The moving body identifier is for identifying each moving body and is unique within each moving image.

Next, the moving body tracking processing by the moving body tracking device 5 and the moving body tracking device 6 will be described. Figure 1
6 is a flowchart showing the flow of the moving body tracking device 5 and the moving body tracking process by the moving body tracking device 6.

The moving body tracking device 5 (6) includes an image extraction unit 5
1 (61) extracts the moving object region extraction unit 5 from within the frame.
2 (62) detects the moving object extracted (step 31)
1). Here, if there is an unprocessed moving body detected (YES in step 312), the moving body of coordinates or area within a fixed distance from the area of the moving body is registered in the related information management unit 54 (64). It is detected from the stored information (step 313).

As a result, if a coordinate within a certain distance from the detected region of the moving body or a moving body in the area is registered in the related information management unit 54 (64) (YES in step 314), it is extracted from within the frame. The region of the moving body is replaced with the coordinate or region registered in the related information management unit 54 (64) (step 315).

The processing of steps 313 to 315 is repeated while there is an unprocessed moving object detected in step 311 (Y in step 312).
ES), if processing is performed on all the detected moving bodies (NO in step 312), the moving body tracking processing is ended.

The moving object tracking process described here is
For each one or a plurality of frames of the moving image (moving object region extraction unit 52,
62 is executed every time a moving body is extracted from the difference information).
That is, the information registered in the related information management units 54 and 64 is updated every time the moving body in the moving image moves.

Since the moving body tracking processing is performed in parallel with the above-mentioned related information registration processing, the moving body is tracked from the next frame in which the related information is registered.

[Fourth Embodiment] In the third embodiment described above, the moving distance of each moving body is used for tracking the moving body, but in the fourth embodiment, the feature amount and area of the moving body are used for tracking the moving body. The case where the moving distance of is used will be described. The fourth embodiment is the same as the third embodiment including the functional configuration of the movement tracking device except for the moving body tracking processing, and therefore the description other than the moving body tracking processing will be omitted.

First, the information registered in the related information management unit 54 (64) in the fourth embodiment will be described. The information registered in the related information management unit 54 (64) is input by the registrant, the coordinates or area of the moving body (the coordinates are converted into the area by the tracking processing described later), the moving body feature amount, as shown in FIG. The related information thus created is managed as a set, and a moving body identifier is attached to each set. The moving body identifier is for identifying each moving body and is unique within each moving image. The moving body feature amount is the same as that described in the first and second embodiments.

Next, the moving body tracking processing by the moving body tracking device 5 and the moving body tracking device 6 will be described. Figure 1
8 is a flowchart showing a flow of a mobile body tracking process by the mobile body tracking device 5 and the mobile body tracking device 6.

The moving body tracking device 5 (6) includes an image extraction unit 5
1 (61) extracts the moving object region extraction unit 5 from within the frame.
2 (62) detects the moving object extracted (step 51).
1). If there is an unprocessed moving body detected (YES in step 512), the moving body of coordinates or area within a certain distance from the area of the moving body is registered in the related information management unit 54 (64). It is detected from the stored information (step 513).

As a result, if a coordinate within a certain distance from the detected moving object area or a moving object in the area is registered in the related information management unit 54 (64) (YES in step 514), it is extracted from within the frame. The similarity between the feature amount of the processing target mobile unit and the registered feature amount of the mobile unit is compared (step 515).

As a result of comparison, if the difference in similarity between the two is within a certain range (YES in step 516), the area of the moving body extracted from the frame is registered in the related information management unit 54 (64). Alternatively, the area is replaced (step 517).

The processing of steps 513 to 517 is repeated while there is an unprocessed moving object detected in step 511 (Y in step 512).
ES), if the processing is performed on all the detected moving bodies (NO in step 512), the moving body tracking processing is ended.

The moving body tracking process described here is
For each one or a plurality of frames of the moving image (moving object region extraction unit 52,
62 is executed every time a moving body is extracted from the difference information).
That is, the information registered in the related information management units 54 and 64 is updated every time the moving body in the moving image moves.

Since the moving body tracking processing is performed in parallel with the above-mentioned related information registration processing, the moving body tracking is executed from the next frame in which the related information is registered.

[0095]

As described above, according to the present invention, a frame is extracted from a moving image and the extracted frame is extracted.
Alternatively, the moving body is extracted based on the difference information for each of a plurality of frames, and the related information is set in the moving body. After that, based on at least one of the feature amount and moving distance of each moving body,
Since it is configured to track each moving object, the viewer
It is possible to easily acquire the related information set for each mobile body.

[Brief description of drawings]

FIG. 1 is a block diagram showing a functional configuration of a moving body tracking device (embodiment 1) to which the invention is applied.

FIG. 2 is a flowchart showing a flow of registration processing of related information by the mobile tracking device 3.

FIG. 3 is a diagram showing an example of a related information registration screen (Example 1).

FIG. 4 is a diagram showing an example of information registered in a related information management unit 34.

5 is a flowchart showing a flow of a moving body tracking process by the moving body tracking device 3. FIG.

FIG. 6 is a diagram showing an example of information registered in the clickable information management unit 35.

FIG. 7 is a flowchart showing a flow of moving image distribution processing by the moving body tracking device 3.

FIG. 8 is a diagram showing an example of a screen displayed on the client side.

FIG. 9 is a block diagram showing a functional configuration of a moving body tracking device according to a second embodiment of the present invention.

FIG. 10 is a flowchart showing a flow of related information providing processing in the mobile tracking device 4.

FIG. 11 is a block diagram showing a functional configuration of a mobile tracking device when a clickable map to be processed on the client side is distributed in the third embodiment.

FIG. 12 is a block diagram showing a functional configuration of a mobile tracking device when a clickable map to be processed on the server side is distributed in a third embodiment.

FIG. 13 is a flowchart showing a flow of registration processing of related information by the mobile unit tracking device 5 and the mobile unit tracking device 6.

FIG. 14 is a diagram showing an example of a related information registration screen (Example 3).

FIG. 15 is a diagram showing an example of information registered in a related information management unit 54 (64).

FIG. 16 is a flowchart showing a flow of a moving body tracking process by the moving body tracking device 5 and the moving body tracking device 6.

FIG. 17 illustrates a related information management unit 54 (6) in the fourth embodiment.
It is the figure which showed the example of the information registered into 4).

FIG. 18 is a flowchart showing a flow of a moving body tracking process by the moving body tracking device 5 and the moving body tracking device 6.

[Explanation of symbols]

1 video camera 2 VTR 3 Mobile tracking device 4 Mobile tracking device 5 Mobile tracking device 6 Mobile tracking device 31 Image Extraction Unit 32 Moving Object Region Extraction Unit 33 Moving object information registration section 34 Related Information Management Department 35 Clickable Information Management Department 36 Clickable map generator 37 Video Distribution Department 41 Image Extraction Unit 42 Moving Object Region Extraction Unit 43 Moving object information registration section 44 Related Information Management Department 45 Clickable Information Management Department 46 Video Distribution Department 47 Click area analysis section 48 HTML document generator 49 HTML Document Delivery Department 51 Image Extraction Unit 52 Moving Object Region Extraction Unit 53 Moving object information registration section 54 Related Information Management Department 55 Clickable map generator 56 Video Distribution Department 61 Image extraction unit 62 Moving Object Region Extraction Unit 63 Moving object information registration section 64 Related Information Management Department 65 Video Distribution Department 66 Click area analysis section 67 HTML document generator 68 HTML Document Delivery Department 200 screens 201 Moving object area 202 Moving object area 203 pointer 204 registration screen 250 screens 251 pointer 252 screen 400 screens 401 pointer 402 registration screen

Front page continuation (51) Int.Cl. ⁷ Identification code FI theme code (reference) H04N 7/173 610 H04N 7/18 K 5L096 7/18 5/91 LN // H04N 7/08 7/08 Z 7 / 081 F Term (Reference) 5C023 AA06 AA15 AA16 AA18 AA26 BA04 CA01 CA05 DA01 DA08 5C053 FA14 FA29 GB06 GB40 JA03 JA21 JA30 LA01 LA14 5C054 AA02 CA04 CC02 CG02 CH01 DA06 EA03 FE03 FE25 FE13 FB13 FE13 FB13 FE13 FB06 FE13 FB13 FE06 FB13 FE13 FB13 FE06 FB13 FE13 FC12 GD04 HA31 5C063 AB07 AC01 BA12 CA07 CA23 CA40 DA07 DA13 DA20 EB39 EB45 EB50 5C064 BA07 BB10 BC10 BC18 BC20 BC25 BD02 BD08 BD09 5L096 FA69 GA08 HA03 JA11

Claims (21)

[Claims] 1. A moving image processing method for performing tracking processing of a moving object in a moving image on a captured or created moving image, wherein difference information is acquired from a plurality of frames forming the moving image for every predetermined number of frames, A moving image processing method, characterized in that a moving object is specified from the acquired difference information, and position information in the moving image of the specified moving object is managed. 2. When one or more moving bodies are specified from the difference information, the sameness is determined for each moving body, and the position information in the moving image of the moving body is determined based on the determination result. The moving image processing method according to claim 1, wherein the moving image processing method is managed. 3. The moving image processing method according to claim 2, wherein the determination of the identity is performed based on a feature amount of the moving body. 4. The moving image processing method according to claim 2, wherein the determination of the identity is performed based on a moving distance of the moving body. 5. The moving image processing method according to claim 2, wherein the determination of the identity is performed based on a feature amount and a moving distance of the moving body. 6. The moving image processing method according to claim 1, wherein the difference information is acquired for each frame from the moving image. 7. The position information is managed by associating it with related information of a moving body corresponding to the position information, and when the position corresponding to the position information of the moving body is selected from the moving image, the selection is performed. The moving image processing method according to claim 1, wherein related information corresponding to the generated position information is provided. 8. A moving image processing apparatus for tracking a moving object in a moving image captured or created, obtains difference information for each predetermined number of frames from a plurality of frames forming the moving image, A moving image including: a moving body specifying unit that specifies a moving body from the acquired difference information; and a position information managing unit that manages position information in the moving image of the moving body specified by the moving body specifying unit. Processing equipment. 9. The moving body identifying means further comprises a moving body determining means for determining the identity of each moving body when the moving body identifying means identifies one or more moving bodies. 9. The moving image processing apparatus according to claim 8, wherein position information in the moving image of the moving body is managed based on the determination result of the body determining unit. 10. The moving image processing apparatus according to claim 9, wherein the moving body determination unit determines the identity based on a feature amount of the moving body. 11. The moving image processing apparatus according to claim 9, wherein the moving body determination means determines the identity based on a moving distance of the moving body. 12. The moving image processing apparatus according to claim 9, wherein the moving body determination means determines the identity based on a feature amount and a moving distance of the moving body. 13. The moving image processing apparatus according to claim 8, wherein the moving body identifying means acquires the difference information for each frame from the moving image. 14. The related information of the mobile body corresponding to the position information managed by the position information management means is managed in association with each other,
The apparatus further comprises a related information management unit that provides related information corresponding to the selected position information when a position corresponding to the position information of the moving body is selected from the moving image. 8. The moving image processing device according to item 8. 15. A moving image processing program for causing a computer to execute a tracking process of a moving object in a moving image captured or created, a first step of obtaining difference information for each predetermined number of frames from a plurality of frames forming the moving image. And a second step of identifying the moving body from the difference information acquired in the first step, and a third step of managing position information in the moving image of the moving body identified in the second step. A moving image processing program provided. 16. The method further comprises a fourth step of determining the identity of each of the moving bodies when one or more moving bodies are identified by the second step, and the third step comprises: 16. The moving image processing program according to claim 15, wherein position information in the moving image of the moving body is managed based on the determination result of the fourth step. 17. The moving image processing program according to claim 16, wherein in the fourth step, the identity is determined based on the characteristic amount of the moving body. 18. The moving image processing program according to claim 16, wherein in the fourth step, the identity is determined based on a moving distance of the moving body. 19. The moving image processing program according to claim 16, wherein in the fourth step, the identity is determined based on a feature amount and a moving distance of the moving body. 20. The moving image processing program according to claim 15, wherein the first step acquires the difference information for each frame from the moving image. 21. A fifth step of managing the position information by associating the related information of the moving body corresponding to the position information, and the position corresponding to the position information of the moving body is selected from the moving image. The video processing program according to claim 15, further comprising a sixth step of providing related information corresponding to the selected position information.