JPH08123816A - Animation hypermedia processor - Google Patents

Abstract

PURPOSE: To obtain an animation hypermedia processor which can manage an anchor in an optional area moving on an animation without applying much load to a computer. CONSTITUTION: An animation hypermedia processor is provided with an animation storage part 11, a link data storage part 5 which defines an optional area of the part 11 as an anchor and stores both anchor and link information, a user operation means 3 which manages the input of a user, a link management means 1 which manages the anchor and link information stored in the part 5 by means of the means 3, a display attribute storage part 10 which stores the display attributes of the animation and the anchor, and a display means 2 which performs the dynamic control of the anchor display on the animation stored in the part 11 by the display attribute stored in the part 10 and based on the anchor and link information stored in the part 5.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention, in a hypermedia processing apparatus using a moving image, displays an anchor which is a logical unit of information on the moving image, connects the anchor by a link, and connects these anchors on the moving image. The present invention relates to a hypermedia processing device which specifies an anchor and operates a link related to the anchor.

[0002]

2. Description of the Related Art A conventional hypermedia processing apparatus using moving images aims to use video data in a hypermedia environment. October 21, 1992-
"Navigation method of video / hypermedia" (NEC C & C Information Laboratory, Moto Takano, Hiroshi Matoba, Yoshinori Hara) announced at the 8th Human Interface Symposium on 23rd, regarding the use of video in hypermedia Has been considered. The handling of video data has been changed so that the appearing objects can be controlled from scene to scene. The link is not directly connected to the physical structure, and the compatibility with the data of other media is considered. Considering the navigation interface specific to video media.
However, the work of creating a link to these objects appearing in the video is a very difficult work, and it becomes necessary to extract the object appearing for each scene of the video and create the link. When actually trying to perform a link search, an anchor for link search is specified at the same time as the video is played back. However, this specification method needs to consider the characteristics of the video.

[0003]

If a conventional moving image hypermedia is used to create a link, it is necessary to extract an object that appears in each video scene and create a link, which is very troublesome. There was a problem of this. Also, when actually performing a link search, in video hypermedia, the shape, size, position, and content of the anchor specified for the link search may change over time, so this was taken into consideration. Anchor designation method is needed. Furthermore, although the moving image data has only one resolution to be stored, when the user wants to use it by enlarging / reducing it to any size, it is necessary to redefine the anchor information one by one.

The present invention has been made in order to solve the above problems, and provides a moving picture hypermedia processing apparatus which reduces the work amount when extracting an appearing object in a moving picture hypermedia. It is an object.
Further, it is another object of the present invention to enable an anchor designation that can be easily operated by the user when performing a link search on a moving image hypermedia. Further, according to the present invention, when displaying moving picture information in a moving picture hypermedia, it is possible to control display without redefining anchors based on separately stored moving picture information and anchor information. It is an object.

[0005]

The moving picture hypermedia processing apparatus according to the first invention has the following elements. (A) a moving image storage unit that stores moving images, (b) a link data storage unit that stores anchor information and link information using an arbitrary area on the moving image storage unit as an anchor,
(C) User operation means for managing user input, (d)
Link management means for managing link information and anchor information in the link data storage section using the user operation means,
(E) a display attribute storage unit that stores the display attributes of the moving image and the anchor, (f) on the moving image stored in the moving image storage unit,
Display means for dynamically displaying an anchor with the display attribute stored in the display attribute storage unit based on the anchor information and the link information stored in the link data storage unit.

In the moving picture hypermedia processing apparatus according to the second aspect of the present invention, the link data storage section stores anchor information serving as an end point of link search and link information indicating a link destination, and the anchor information is the entire moving picture. Is divided into several areas, and the link operation can be performed using the area as an anchor.

In the moving picture hypermedia processing apparatus according to the third aspect of the present invention, the link data storage section uses the following anchor creating method for each type of moving picture data in the moving picture storage section when registering the anchor information. Is characterized by. (A) Video that moves the entire screen (b) Video that moves part of the screen

In the moving picture hypermedia processing apparatus according to the fourth aspect of the present invention, when registering the anchor information, the link data storage unit allows the user to previously specify the shape and movement of the appearing object desired to be the anchor with the mouse. Is characterized by.

In the moving picture hypermedia processing apparatus according to the fifth aspect of the present invention, the link data storage section is characterized in that the link information dynamically changes the link destination in relation to the elapsed time.

In the moving picture hypermedia processing apparatus according to the sixth aspect of the present invention, when registering the link information, the link management means registers the link information after visually confirming the content of the moving picture at the link destination. Characterize what you do.

In the moving picture hypermedia processing apparatus according to the seventh aspect of the present invention, when the link management means displays the anchor, the created anchor is highlighted so that the user can easily distinguish it from the background moving picture. It is characterized by being displayed.

In the moving picture hypermedia processing apparatus according to the eighth aspect of the invention, the link management means searches the link information when the anchor moves quickly, when the mouse is framed in the moving picture, or when the mouse is clicked. At this time, the reproduction speed is slow.

In the moving picture hypermedia processing apparatus according to the ninth aspect of the present invention, the link management means searches the link information when the anchor moves quickly, when the playback speed is near the anchor, when the mouse is clicked, or when the mouse moves. It is characterized by being slow when it makes a certain movement.

In the moving picture hypermedia processing apparatus according to the tenth aspect of the invention, the link management means, when the link information is searched, if the anchor is small and it is difficult to click, when the mouse is clicked in the moving picture, the designated area of the mouse is clicked. Is enlarged and displayed.

In the moving picture hypermedia processing apparatus according to the eleventh aspect of the present invention, the link management means stops displaying the moving picture once when a certain anchor appears when searching for the link information, and makes a selection prompting the user to input. It is characterized by displaying a dialog.

In the moving picture hypermedia processing apparatus according to the twelfth aspect of the invention, the link management means fixes the position and size of the anchor when searching the link information.

[0017]

In the moving picture hypermedia processing apparatus according to the present invention, the display means displays the moving picture data stored in the moving picture data storage section on the link data storage section according to the user's instruction input through the user operation section. Based on the stored anchor information and link information, the anchor is dynamically displayed with the display attribute stored in the display attribute storage unit. Since the moving image data is managed separately from the anchor information and the link information, a hypermedia mechanism of the moving image can be virtually constructed.

Further, the link data storage section stores anchor information serving as an end point of link search and link information indicating a link destination, and the anchor information divides the entire moving image into several areas, By performing the link operation using the area as the anchor, the load of management of the anchor on the system side can be reduced.

When registering the anchor information, the link data storage unit uses the following anchor creation method for each type of moving image data in the moving image data storage unit to extract an object serving as an anchor. In doing so, the amount of computation on the system side can be reduced. (A) Video that moves the entire screen (b) Video that moves part of the screen

When the anchor information is registered, the link data storage section extracts the object serving as the anchor by the user previously designating the shape and the movement of the appearing object desired as the anchor with the mouse. In this case, the amount of calculation on the system side can be reduced.

Further, the link data storage unit dynamically changes the link destination in relation to the elapsed time of the link information, so that the link destination is changed depending on the situation even for links to the same anchor. It is possible.

Further, when registering the link information, the link management means visually confirms the content of the moving image at the link destination and then registers the link information.
When you want to link to moving image data or an object on a moving image, you can visually check the linked data.

Further, when the link management means displays the anchor, the created anchor is highlighted to distinguish whether the user is the anchor or the moving image which is the background. Can do things

Further, the link management means causes a slow playback speed when the anchor moves quickly when the link information is searched, when the mouse is framed in the moving image or when the mouse is clicked. You can easily select a fast-moving anchor.

Further, the link management means, when searching the link information, when the anchor moves fast, when the playback speed is such that when the mouse is clicked near the anchor or when the mouse makes a certain movement, the playback speed is The slowness makes it possible to easily select a fast-moving anchor.

In addition, when the anchor is small and it is difficult to click when searching for the link information, the link management means makes the designated area of the mouse smaller by clicking the mouse in the moving image. It is possible to easily select an anchor that is difficult to select.

Further, the link management means stops the display of the moving image once and displays a selection dialog prompting the user to input in the moving image when a certain anchor appears when searching the link information. The input dialog can be displayed by stopping the video at any position and time.

Further, the link managing means fixes the position and size of the anchor when searching the link information, so that the user can perform the link operation without making a mistake in the position of the anchor. it can. Also, since the position is fixed, the load on the link operation on the system side is reduced.

[0029]

【Example】

Example 1. The structure and operation of the basic part of the present invention will be described. A moving picture hypermedia processing apparatus according to claim 1 of the present invention will be described with reference to FIG. In the figure, 1 is a link management means for managing links and anchors, 2 is a display device for displaying still images, moving images, figures and the like, 3 is a user operation means such as a keyboard and mouse, and 4 is a user for operation. is there. Reference numeral 5 is a link data storage unit that stores link data, 6 is a moving image storage unit that stores moving images, 7 is link information in the link data storage unit 5, 8 is anchor information in the data storage unit 5, and 9 is scaling, etc. The display control means 10 for controlling the above is a display attribute storage unit for storing the attributes of the display size. Reference numeral 11 is a moving image storage unit for storing moving image data, 12
Is a still image storage unit for storing still image data.

Next, the operation will be described. FIG. 2 is a diagram illustrating the flow of processing. First, the display control means 9
Reads the display attributes stored in the display attribute storage unit 10 (21). Next, the display control means 9 generates a window having a size determined by the display attribute, reads the moving image data from the moving image storage unit 11 and displays it on the display device 2 (22). Next, the link management unit 1 reads the anchor information 8 in the link data storage unit 5 and sends it to the display device 2 via the display control unit 9 to display the anchor (23). While the moving image and the anchor are displayed, the user operation means 3 detects the user's operation (25).
When no user operation is performed, the moving image display is continued (22). When the search is instructed, the link management means 1 performs the link search (26) and displays the moving image (22) or the still image (28). When the screen size change is instructed, the screen size is changed (29), and the display attribute stored in the display attribute storage unit 10 is changed (3).
0). Further, when the link creation is instructed, the link creation is performed (31).

FIG. 3 is a diagram for explaining the operation centering on the screen example. In the figure, an anchor 41a is defined in a moving image 46a being reproduced, and a search request is issued to the anchor 41a to display a still image 42 as a search result.

FIG. 4 is a diagram for explaining the data structure. In the figure, a display attribute 43a indicating the size of the moving image display in each of the vertical and horizontal directions, anchor information 44 indicating at which position the anchor is drawn, the identifier, and link information indicating the link destination for the identifier of the anchor. 45 are defined respectively.

FIG. 5 is a diagram showing an example in which the screen size is changed from FIG. In the figure, the anchor 41b is also subjected to coordinate conversion in the size-changed moving image 46b and is displayed at the same position on the moving image. On the other hand, by issuing a search request, a still image 42 is obtained as the search result as in FIG.

FIG. 6 is a diagram for explaining the data structure after the size change. In the figure, the changed vertical and horizontal sizes are defined in the display attribute 43b.

Example 2. In the system configuration diagram of FIG. 1, the link data storage unit 5 stores anchor information 8 that is the end point of the link search and link information 7 that indicates the link destination for the anchor. In the present embodiment, the stored anchor information 8 as shown in claim 2 is stored as
A method will be described in which the entire moving image is divided into some areas and a link operation is performed using any of the divided areas as an anchor.

FIG. 7 shows an anchor information table in the link data storage unit 5 in this embodiment.
In FIG. 7, 51 is an anchor information table, and 5
2 is an anchor ID, 53 is a start frame, 54 is an end frame, and 55 is an anchor area. Further, FIG. 8 shows the anchor area definition table 56 in which the anchor area 55 is defined. In FIG. 8, 57 is the X coordinate of the anchor, 58 is the Y coordinate of the anchor, 59 is the width of the anchor, and 60 is the height of the anchor. In this case, the moving image data with a resolution of 240x240 is nine rectangular regions of 80x80. Has been divided in advance.

By the definition of the anchor information shown in FIG. 7,
The anchor area 55 is B2 → C3 in accordance with the reproduction of the moving image.
→ B3 → A2 → B2. The situation is illustrated in FIG. In FIG. 9, 61 is a moving image reproduction screen, and 62 is a butterfly appearing in the moving image. Here, the anchor area changes as the butterfly moves. For example, in FIG. 9, when the moving image is reproduced for 15 frames, the user can select a region (for example, (18
0,180)), the system side will select anchor 1
Is determined to have been selected, and according to the instruction from the user,
A link operation such as a link search for the anchor 1 can be performed.

FIG. 10 shows a flow chart in this embodiment. First, when reproducing a moving image, initialization processing such as a timer is performed (63). Next, the anchor area table 56
Is read and the whole moving picture is divided into several areas (64). After that, the anchor information table 51 is read to analyze which of the divided areas becomes an anchor at which timing (6
5). Thereafter, the moving image is actually displayed and the internal timer is advanced (66). The system appropriately waits for an input of a mouse event or the like from the user (67), and when an anchor is selected (68), performs a link operation process (6).
9).

Example 3. In addition, as shown in FIG. 11, the anchor information table 51 in the link data storage unit 5,
It is possible that multiple areas are specified at the same time. Especially, when there are two butterflies, it is necessary to perform processing with the same anchor. FIG. 12 shows a state of the anchor area when FIG. 11 is actually displayed. For example, in FIG. 12, when the moving image is reproduced for 15 frames, the area of B3 becomes the anchor area in addition to the area of C3.

Example 4. In the second embodiment, when the anchor area is defined, the entire area is divided into rectangular areas, but it is also possible to divide the area into arbitrary areas.

Example 5. In the second embodiment, when the anchor area is defined, the entire area is divided into rectangular areas, but it is also possible to overlap the areas by changing the description in the anchor area definition table.

Example 6. FIG. 13 shows an S / W configuration diagram in claim 3 which is an extension of FIG. Here, 101 is a scene management unit that newly divides a moving image stored in the moving image storage unit 6 into scenes and manages information, and 102 is a scene information storage that stores information about the scenes divided by the scene management unit 101. 103 is a fixed anchor creating means for creating an anchor area fixed in a moving image, and 104 is a dynamic anchor creating means for creating an anchor in an object that moves in the moving image. There are two types of moving images: one in which the object of interest hardly moves (referred to as type (a) in FIG. 14) and one in which the object of interest moves without background (referred to as type (b) in FIG. 15). Classify by scene and change the anchor creation method for each type.

FIG. 16 shows a flow chart of the processing. The table shown in FIG. 17 is held in the scene information storage unit 102 by the scene management unit 101. In the case of the (a) type moving image, the fixed anchor creating means 103 designates the rough shape of the object to be the anchor by using the mouse or the keyboard for the user as shown in FIG. Let The specified area information is used as a table shown in FIG.
To hold. In the case of a (b) type moving image, the dynamic anchor creating means 104 uses a conventional moving image analysis method,
Anchor is designated by the method described in claim 5.

During the reproduction of the moving image, the scene information management means 101 monitors the scene switching, and the scene information storage section 102.
The scene type information is read to understand which type of scene is currently being reproduced. When the (a) type moving image is being reproduced, the link management means 1 reads the area information stored in the link data storage unit 5, and executes the link operation when the area is selected by a device such as a mouse. To do. When the (b) type moving image is being reproduced, the link management means 1 executes the link operation by the conventional method or the method described in claim 5.
As a result, in the case of the (a) type moving image, it is only necessary to compare the area specified by the user with the area information stored in the link data management unit 5, and the amount of calculation is drastically reduced.

Example 7. In the system configuration diagram of FIG. 1, the link data storage unit 5 stores anchor information 8 that is the end point of the link search and link information 7 that indicates the link destination for the anchor. In the present embodiment, as described in claim 4, when registering the anchor information 8 in the link data storage unit 5, an anchor registration method in which the shape and movement of the appearing object desired by the user as an anchor is designated by the mouse. Explain.

FIG. 20 illustrates the extraction method of the shape of the appearing object in this embodiment. Figure 2
At 0, 151 is an object to be extracted,
Reference numeral 152 is a contour in which an object to be extracted is designated by a mouse or the like. As shown in FIG. 21, the method of extracting the appearance object shape in a normal moving image is to perform edge extraction for all parts of the image and extract all appearing objects. Here, 153 is an object to be extracted, and 154a and 154b are unrelated objects. In the conventional method, the system side has the edge information of all the extracted objects, and therefore, when the original image is large, calculation of irrelevant object extraction is performed, which is inefficient.

Therefore, as shown in FIG. 20, as a preprocessing for edge extraction, a rough outline 152 of an object to be extracted is designated by a mouse or another pointing device, so that only an edge near the designated portion is edged. Since it suffices to perform extraction processing, the efficiency of object shape extraction is improved. Specifically, the user's input is transmitted to the link management means 1 via the user operation means 3, and the moving picture data stored in the moving picture storage unit 11 is analyzed. Here, the shape of the extracted object is stored in the anchor information table 51 in the link data storage unit 5. In FIG. 20, the butterfly-shaped anchor information 8 is stored by designating the butterfly-shaped with a pointing device.

Next, with reference to FIG. 22, a method of extracting the trajectory of an appearing object in this embodiment will be described. In FIG. 20, reference numeral 151 denotes an object to be extracted, and 155 is a locus in which the object to be extracted is designated by a mouse or the like. As shown in FIG. 23, the method of extracting the locus of the appearing object in a normal moving image is to extract the movement of the specified object from the error between the frame extracted at regular intervals and the previously extracted frame.
Therefore, as shown in FIG. 22, as a pre-process for extracting a trajectory, a rough movement 155 of an object whose trajectory is to be extracted is designated by a mouse or another pointing device, so that only the vicinity of the designated portion is framed. Since it suffices to perform the inter-error extraction processing, the efficiency of object trajectory extraction is improved. Specifically, the user's input is performed via the user operation unit 3.
The moving image data transmitted to the link management means 1 and analyzed in the moving image storage unit 11 is analyzed. Here, the trajectory of the extracted object is stored in the anchor information table 51 in the link data storage unit 5. In FIG. 22, the anchor information 8 of the trajectory of the butterfly is stored by designating the movement of the butterfly with a pointing device.

FIG. 24 shows a flow chart in this embodiment. First, a moving image for which object extraction is desired is displayed (158). Next, the outline of the object to be extracted is specified by a pointing device such as a mouse (159), and edge extraction processing is actually performed.
The shape of the object is extracted (160). Further, the locus of the object to be extracted is specified by a pointing device such as a mouse (161), the interframe error is actually calculated, and the locus of the object is extracted (162). Finally, the anchor information 8 of the object whose shape and trajectory are extracted is saved (163).

Example 8. In the system configuration diagram of FIG. 1, the link data storage unit 5 stores anchor information 8 that is the end point of the link search and link information 7 that indicates the link destination for the anchor. In the present embodiment, a method for dynamically changing the content of the link information as described in claim 5 in relation to the elapsed time will be described.

FIG. 25 shows a link information table in the link data storage unit 5 in this embodiment. In FIG. 25, 201 is a link information table, 2
Reference numeral 02 is a link ID, 203 is an anchor ID, 204 is a start frame, 205 is an end frame, and 206 is a link destination object ID. In FIG. 25, the link 1 and the link 2 are links to the same anchor 1, but the domain is 0 to 20 frames for the link 1 and 20 to 40 frames for the link 2. Therefore, the link information of FIG. 25 is applied to the moving image information shown in FIG. The IDs of the anchors appearing in FIG. 26 are all the same for 1 anchor 1 1 but the content of the link changes after 20 frames. In this case, when the link search is actually performed on the anchor 1, the video object 1 is searched in the 0 to 20th frame and the video object 2 is searched in the 20 to 40th frame, so that the link is related to the elapsed time of the moving image. A link whose contents change will be realized.

FIG. 27 shows a flow chart in this embodiment. First, when an anchor is selected (209), the link information table 201 is read (210),
The content is analyzed (211). According to the analyzed result, if the link is within the domain, it is valid and the link content is executed (212).

Embodiment 9 FIG. A moving picture hypermedia processing apparatus according to claim 6 of the present invention will be described with reference to FIG. In the figure, reference numeral 251 is a link confirmation means capable of displaying and confirming the data of the link destination when creating the link. Reference numeral 252 is a digest generation unit that edits an arbitrary part of the data and generates display data of an appropriate length when the data of the link destination is displayed and confirmed. 25
Reference numeral 3 is a color removing means for reducing the number of colors used in the displayed data without using the original data. 254
Is a frame skipping means that does not use the original data as it is, but drops the frames at an appropriate rate. Further, reference numeral 255 denotes a head frame display means for displaying the head frame in the moving image as a still image at the time of display.

Next, the operation will be described with reference to FIGS. 29 and 30. FIG. 29 is a diagram illustrating the flow of processing. When the user creates an anchor at an arbitrary position in the displayed moving image display 271 (261) and presses the link creation button 275, the link destination candidate file is read from the moving image storage unit 11 and the first frame display means is displayed. By the action of 255, the first frame of each candidate is displayed as a link destination data confirmation display 272 (262). A user operation is detected in this list display (263), and its type is determined (264). If it is a confirmation instruction, the display 273b in the case of reproducing the original moving image file as it is is subjected to frame reduction and color removal processing (266), and a confirmation display 273a is performed (267). This is repeated until the link confirmation operation is performed. If it is a link confirmation operation, the link destination is confirmed and the link information 7 and the anchor information 8 are updated (265).

Example 10. In the system configuration diagram of FIG. 1, the link data storage unit 5 stores anchor information 8 that is the end point of the link search and link information 7 that indicates the link destination for the anchor. In this embodiment, a method for highlighting anchor information in synchronization with a moving image as described in claim 7 will be described.

FIG. 31 shows an anchor information table in the link data storage unit 5 in this embodiment.
In FIG. 31, 301 is an anchor information table, 302 is an anchor ID, 303 is a start frame, 3
04 is the end frame, 305 is the X coordinate of the anchor, 30
6 is the Y coordinate of the anchor, 307 is the width of the anchor, 308
Is the height of the anchor. Here, the shape of the anchor is rectangular for the sake of explanation. The anchor information 8 shown in FIG. 31 is displayed on the display device 2 in synchronization with the display of the moving image. Here, FIG. 32 shows the anchor information 8 at the time of reproducing 0 frame, which is displayed according to the information stored in FIG. Here, 309a is the anchor 1, 309
b is represented as anchor 2.

In FIG. 32, since the anchors are displayed separately on the moving image, the user can easily recognize that the anchor is the anchor. FIG. 33 shows a flow chart in this embodiment. First, the anchor information table 301 is read (310), and its contents are analyzed (311). Then, according to the analysis result, the moving image / anchor is displayed (312).

Example 11. In the tenth embodiment, the anchor information 8 stored in the anchor information table 301 is rectangular, but the anchor information 8 of any other shape can be handled. Also, it is possible to handle the case where there is a rotation or a change in size.

Example 12 34 shows a link management means 1 of FIG. 1 provided with a click mode setting means 351, a mouse click detection means 352, and a cursor position detection means 353, and a display control means 9 of FIG. 1 provided with a display speed control means 354. is there. The click mode setting means 351 changes to a mode in which the playback speed changes when the mouse is clicked in the moving image, the mouse click detection means 352 determines whether or not the mouse is clicked by the user, and the cursor position detection means 353 is the cursor. The display speed control means 354 detects whether or not the video has entered the video, and changes the playback speed of the video.

FIG. 35 shows this flow chart. When the click mode setting unit 351 does not set the click mode, when the cursor position detecting unit 353 detects that the cursor is inside the moving image, the display speed control unit 354 makes the reproduction speed slow. When the cursor is detected in the moving image by the cursor position detecting means 353 and the mouse is clicked by the mouse click detecting means 352 when the click mode setting means 351 is set to the click mode Then, the display speed control means 354 makes the reproduction speed slow.

Examples of this screen are shown in FIGS. 36 and 37. FIG.
6, 354 is a display screen, 355 is a moving image playback screen, 356 is a mouse cursor, and 357 is an anchor attached to a moving object. In FIG. 37, 358 is a mouse cursor, and 359 is an anchor attached to a moving object. When the movement of the object with the anchor is fast, the user cannot perform the search by designating the anchor he or she wants to designate with the mouse cursor 356, for example, the anchor 357. Therefore, when the mouse cursor enters the moving image reproduction screen 355 and enters the state of 358, the reproduction speed becomes slow like the movement of the anchor 359, which facilitates the search execution. Similarly, when the mouse cursor 358 is clicked, the playback speed becomes slower, which facilitates the search execution.

Example 13 38, the link management means 1 of FIG. 1 is provided with a mouse click detection means 401 and an anchor vicinity determination means 402, and the display control means 9 of FIG. 1 is provided with a display speed control means 403. The mouse click detection unit 401 determines whether the mouse is clicked by the user, the anchor vicinity determination unit 402 determines whether the mouse cursor is near the anchor, and the display speed control unit 403 reproduces the moving image. It changes speed.

FIG. 39 shows this flow chart. When the anchor proximity determination unit 402 detects that the cursor is near the anchor and the mouse click detection unit 401 detects that the mouse has been clicked, the display speed control unit 403 slows down the reproduction speed. Examples of this screen are shown in FIGS. 40 and 41. In FIG. 40, 404 is a mouse cursor, and 405 is an anchor attached to a moving object. In FIG. 41, reference numeral 406 represents the movement of the mouse cursor by the user. As shown in FIG. 36, when the movement of the object to which the anchor is attached is fast, the user cannot perform the search by designating the anchor he or she wants to designate with the mouse cursor 356, for example, the anchor 357. Then, by moving the mouse cursor 356, as shown in FIG.
By clicking the mouse while the mouse cursor 404 is near the anchor as in 0, the playback speed becomes slower, which facilitates search execution. Further, by moving the mouse in accordance with a predetermined motion, such as 406, the reproduction speed becomes slower, which makes it easier to execute the search.

Example 14 42, the link management means 1 of FIG. 1 is provided with a mouse click detection means 452 and a cursor position detection means 453, and the display control means 9 of FIG. 1 is provided with a display enlargement means 454. The mouse click detecting means 452 is for judging whether or not the mouse is clicked by the user, and the cursor position detecting means 45.
3 is for detecting whether or not the cursor is inside the moving image, and the display speed control means 454 is for enlarging the reproduction screen of the moving image.

FIG. 43 shows this flow chart. When the cursor position detecting unit 453 detects that the cursor is inside the moving image and the mouse click detecting unit 452 detects that the mouse is clicked, the display enlarging unit 4
The reproduction screen is enlarged by 54. An example of this screen is shown in FIG.
4 shows. In FIG. 44, 455 is a mouse cursor,
456 is a moving anchor, 457 is a moving anchor that the user wants to specify, 458 is an enlarged screen, and 459 is an enlarged user-specified anchor. When the size of the object with the anchor is small, the user wants to specify the anchor 457 with the mouse cursor 455.
It is difficult to select and execute the search. Therefore, by clicking the mouse in the moving screen and temporarily displaying the vicinity of the mouse like the enlarged screen 458, the anchor 459 can be easily specified and the search can be easily performed. Similarly, by clicking the mouse 460, the vicinity of the mouse is displayed on the enlarged display screen 458.
The anchor 459 can be easily designated by performing the slow reproduction with the display as shown in FIG.

Example 15 A moving picture hypermedia processing apparatus according to claim 11 of the present invention will be described with reference to FIG. In the figure, reference numeral 501 is an anchor notifying means for notifying the user of the presence of an anchor that the user desires to select in the moving image data. Reference numeral 502 is a voice output means for outputting a voice to the user, and 503 is an interactive display means for interactively advancing selection with the user.

Next, the operation will be described with reference to FIGS. First, when the moving image data 520a in which the anchor is defined is reproduced, the presence or absence of the anchor is determined every time the frame advances (512). When an anchor exists in the frame, it is further determined whether or not it is an important anchor (513). If it is an important anchor, a dialog box 512 for confirmation is displayed (514), and it is determined whether or not confirmation has been performed (515). On the other hand, when the moving image data 520b that does not include the important anchor is reproduced, it is possible to draw a sound or perform slow reproduction when the frame in which the anchor is present is reached (516) to get the user's attention. The user refers to this information to determine whether to perform a link search (517), and executes the search if necessary. (518).

Example 16. FIG. 48 shows an anchor creation flow in claim 12. Scene information management means 101
The moving image is divided into scenes by, and this information is stored in the scene information storage unit 102 as a data structure as shown in FIG. As shown in FIG. 50, the user specifies the size and position with a device such as a mouse in the moving image, and the link management unit 1 links the information as a table having a data structure as shown in FIG. 51 as an anchor. The data is stored in the data storage unit 5. When operating the link for this anchor, the user double-clicks the anchor area to select it.

FIG. 52 shows a flow at the time of link operation. The link management means 1 is displayed by a selection means such as a menu as shown in FIG. 53, and the user selects a process to be executed. In the case of creating a link, the link management means 1 displays a dialog as shown in FIG. 54, and when the user inputs the target object name, the link management means 1 stores the link information 7 for it in the link data storage unit 5. To do. The scene and the objects effective at that time are stored in the scene information storage unit 102 in a table as shown in FIG. Anchors are managed for each scene, and the content of the anchor changes as the scene changes over time. For example, as shown in FIG. 56, when the beginning t0 to t1 of the moving image is scene 1 (FIG. 50) and t1 to t2 is scene 2 (FIG. 57), when the moving image is reproduced, the scene management unit 101 stores the scene information storage. From the information of the unit 102, which scene is currently grasped. When the user selects an anchor with a pointing device such as a mouse during the reproduction of the scene 1, the link management unit 1 refers to the link information 7 from the link data storage unit 5, and as a result, the link management unit 1 with respect to the object in the moving image. The link is displayed on the display device 2 in a dialog box or the like. As a result, the user does not have to specify the anchor by changing the place at any time, and the object to be managed by the link management means 1 can be changed in each scene, which is convenient. Moreover, since complicated calculation processing such as moving image analysis is not performed, the amount of calculation is drastically reduced.

[0070]

In the moving picture hypermedia processing apparatus according to the present invention, the display means displays the link data on the moving picture data stored in the moving picture data storage section in accordance with the user's instruction input through the user operating means. Based on the anchor information and the link information stored in the section, the anchor is dynamically displayed with the display attribute stored in the display attribute storage section. Since the moving image data is managed separately from the anchor information and the link information, a hypermedia mechanism of the moving image can be virtually constructed.

Further, the link data storage section stores anchor information serving as an end point of link search and link information indicating a link destination, and the anchor information divides the entire moving image into several areas, By performing the link operation using the area as the anchor, the load of management of the anchor on the system side can be reduced.

When the anchor information is registered, the link data storage unit uses the following anchor creation method for each type of moving image data in the moving image data storage unit to extract an object serving as an anchor. In doing so, the amount of computation on the system side can be reduced. (A) Video that moves the entire screen (b) Video that moves part of the screen

When the anchor information is registered, the link data storage section extracts the object serving as the anchor by the user previously designating the shape and the movement of the appearing object desired as the anchor with the mouse. In this case, the amount of calculation on the system side can be reduced.

Further, the link data storage unit dynamically changes the link destination of the link information in relation to the elapsed time so that the link destination of the same anchor can be changed depending on the situation. Is possible.

When the link information is registered, the link management means visually confirms the content of the moving image at the link destination, and then registers the link information.
When you want to link to moving image data or an object on the moving image, you can visually check the linked data.

Further, the link management means, when displaying the anchor, highlights the created anchor so that the user distinguishes between the anchor and the background moving image. be able to.

Further, the link management means, when retrieving the link information, when the movement of the anchor is fast, when the mouse is framed in the moving image or when the mouse is clicked, the reproduction speed becomes slow. This makes it possible to easily select an anchor that moves quickly.

Further, the link management means, when retrieving the link information, when the anchor moves fast, when the playback speed is such that the mouse is clicked near the anchor or when the mouse makes a certain movement, the playback is performed. The slow speed allows easy selection of fast-moving anchors.

When the link is small and the anchor is difficult to click when retrieving the link information, when the mouse is clicked in the moving image, the designated area of the mouse is enlarged and displayed. It is easy to select anchors that are small and difficult to select.

When a certain anchor appears when searching the link information, the link management means stops displaying the moving image once and displays a selection dialog prompting the user to input the moving image. The input dialog can be displayed by stopping the video at any position in the time.

Further, the link management means fixes the position and size of the anchor when searching the link information, so that the user can perform the link operation without making a mistake in the position of the anchor. You can Also, since the position is fixed, the load on the link operation on the system side is reduced.

[Brief description of drawings]

FIG. 1 is a system configuration diagram of a moving picture hypermedia processing apparatus according to the present invention.

FIG. 2 is a flowchart showing a procedure of anchor drawing and link search according to the first embodiment of the present invention.

FIG. 3 is an example of a screen according to the first embodiment of the present invention.

FIG. 4 is a diagram showing a data structure according to the first embodiment of the present invention.

FIG. 5 is a diagram showing an example of changing the screen size according to the first embodiment of the present invention.

FIG. 6 is a diagram showing a data structure after a screen size change according to the first embodiment of the present invention.

FIG. 7 is a diagram showing an anchor information table in Embodiment 2 of the present invention.

FIG. 8 is a diagram showing an anchor area table in Embodiment 2 of the present invention.

FIG. 9 is an example of a screen for moving image reproduction according to the second embodiment of the present invention.

FIG. 10 is a flowchart showing a procedure of anchor drawing and link search according to the second embodiment of the present invention.

FIG. 11 is a diagram showing an anchor information table in Embodiment 3 of the present invention.

FIG. 12 is a screen example of moving image reproduction in Embodiment 3 of the present invention.

FIG. 13 is a diagram showing a S / W block diagram in a sixth embodiment of the present invention.

FIG. 14 is an example of a screen when a focused object hardly moves in the sixth embodiment of the present invention.

FIG. 15 is an example of a screen when an object of interest moves in Example 6 of the present invention.

FIG. 16 is a flow chart showing a procedure of extracting an object according to the sixth embodiment of the present invention.

FIG. 17 is a diagram showing a table storing scene information according to the sixth embodiment of the present invention.

FIG. 18 is a diagram showing a method of specifying an object to be extracted in the sixth embodiment of the present invention.

FIG. 19 is a diagram showing an anchor area information table in Embodiment 6 of the present invention.

FIG. 20 is a diagram showing extraction of the shape of an appearing object in Embodiment 7 of the present invention.

FIG. 21 is a diagram showing extraction of the shape of a normal appearing object in Embodiment 7 of the present invention.

FIG. 22 is a diagram showing extraction of a trajectory of an appearing object in Embodiment 7 of the present invention.

FIG. 23 is a diagram showing extraction of a trajectory of a normal appearing object in Embodiment 7 of the present invention.

FIG. 24 is a flowchart showing a procedure of extracting an appearing object in the seventh embodiment of the present invention.

FIG. 25 is a diagram showing a link information table according to the eighth embodiment of the present invention.

FIG. 26 is a screen example of a link search in the eighth embodiment of the present invention.

FIG. 27 is a flow chart showing a procedure for link search in Embodiment 8 of the present invention.

FIG. 28 is a diagram showing an S / W configuration diagram according to the ninth embodiment of the present invention.

FIG. 29 is a flowchart showing a procedure of processing for link confirmation in Embodiment 9 of the present invention.

FIG. 30 is a screen example of a link confirmation screen according to the ninth embodiment of the present invention.

FIG. 31 is a diagram showing an anchor information table in Embodiment 10 of the present invention.

FIG. 32 is an example of a screen of anchor highlighting according to the tenth embodiment of the present invention.

FIG. 33 is a flowchart showing a procedure of anchor highlighting processing according to the tenth embodiment of the present invention.

FIG. 34 is a diagram showing a S / W configuration diagram in Embodiment 12 of the present invention.

FIG. 35 is a flowchart showing a procedure of moving image reproduction processing in Embodiment 12 of the present invention.

FIG. 36 is an example of a screen showing a procedure of normal moving image reproduction processing in Embodiment 12 of the present invention.

[Fig. 37] Fig. 37 is an example of a screen showing the procedure of a moving image reproduction process in Embodiment 12 of the invention.

FIG. 38 is a diagram showing a S / W configuration diagram in Embodiment 13 of the present invention.

FIG. 39 is a flowchart showing a procedure of moving image reproduction processing in Embodiment 13 of the present invention.

[Fig. 40] Fig. 40 is an example of a screen of moving image reproduction processing in Embodiment 13 of the present invention.

[Fig. 41] Fig. 41 is a screen example of a mouse movement in the moving image reproduction processing in Embodiment 13 of the present invention.

FIG. 42 is a diagram showing a S / W configuration diagram in Embodiment 14 of the present invention.

FIG. 43 is a flowchart showing a procedure of moving image reproduction processing in Embodiment 14 of the present invention.

[Fig. 44] Fig. 44 is an example of a screen of moving image reproduction processing in Embodiment 14 of the present invention.

FIG. 45 is a diagram showing a S / W configuration diagram in Embodiment 15 of the present invention.

FIG. 46 is a flowchart showing the procedure of processing for an important anchor in Embodiment 15 of the present invention.

FIG. 47 is a screen example of a process for an important anchor according to the fifteenth embodiment of the present invention.

FIG. 48 is a flowchart showing the steps of anchor creation processing in Embodiment 16 of the present invention.

FIG. 49 is a diagram showing a table for storing scene information in Embodiment 16 of the present invention.

FIG. 50 is a screen example of designation processing of a fixed anchor in Embodiment 16 of the present invention.

FIG. 51 is a diagram showing a table for storing fixed anchor information in Embodiment 16 of the present invention.

FIG. 52 is a flowchart showing a procedure of processing for link operation in Embodiment 16 of the present invention.

FIG. 53 is an example of a menu screen when performing a link operation in Embodiment 16 of the present invention.

FIG. 54 is a screen example of a dialog when creating a link in the sixteenth embodiment of the present invention.

FIG. 55 is a diagram showing a table for storing link contents in Embodiment 16 of the present invention.

[Fig. 56] Fig. 56 is a diagram showing a scene according to Example 16 of the present invention.

FIG. 57 is a screen example of fixed anchor designation processing according to the sixteenth embodiment of the present invention.

[Explanation of symbols]

1 link management means, 2 display device, 3 user operation means, 4 users, 5 link data storage section, 6 search data storage section, 7 link information, 8 anchor information, 9 display control means, 10 display attribute storage section, 11 video Storage unit, 12 still image storage unit, 21 display attribute reading process, 22 moving image display process, 23 anchor display process, 2
4 user operation detection processing, 25 user operation type determination, 26 link search processing, 27 link destination data type determination, 28 still image display processing, 29 screen size change processing, 30 display attribute change processing, 31 link creation processing,
41a link source anchor, 41b link source anchor,
42 link destination information, 43a display attribute information, 43b display attribute information, 44 anchor information, 45 link information,
51 anchor information table, 52 anchor ID, 5
3 start frame, 54 end frame, 55 anchor area, 56 anchor area table, 57X coordinate, 5
8 Y coordinate, 59 width, 60 height, 61 movie playback screen, 62 object appearing in movie, 63 timer initialization process, 64 anchor region table read process, 65 anchor information table read process, 66 movie Display processing, 67 mouse event extraction processing, 68 anchor selection determination, 69 link operation processing, 101 scene information management means, 102 scene information storage section, 103 fixed anchor creation means, 104 dynamic anchor creation means, 105 focused object, 106
One frame in moving image, 107 Object of interest, 108 One frame in moving image, 109
Scene information table creation processing, 110 scene confirmation processing in target video, 111 mode change processing to anchor creation mode, 112 scene type determination processing, 113 dynamic anchor creation processing, 114 anchor area definition processing, 11
5 anchor region table creation processing, 116 scene information table, 117 target object, 118
One frame in a moving image, 119 an area information table for storing an anchor area, 151 an object to be extracted, 152 a trajectory specifying an object to be extracted, 153 an object to be extracted, 154a an unrelated object, 154b an unrelated object, 155 A trajectory that specifies the object whose trajectory you want to extract, 156 an object that you want to extract, an object that does not have a relationship with 157a, an object that does not have a relationship with 157b, a video display process of 158, a contour specification process of 159 objects, an edge extraction process of 160, a trajectory of 161 objects Designation process, 162 trajectory extraction process, 163 anchor information storage process, 201 link information table, 202 link ID, 203 anchor I
D, 204 start frame, 205 end frame, 2
06 link destination object ID, 207 selected anchor, 208a link destination object, 208
b link destination object, 209 anchor selection processing, 210 link information reading processing, 211 link content analysis processing, 212 link content execution processing,
251 link confirmation means, 252 digest generation means, 253 color removal means, 254 frame removal means,
255 head frame display means, 261 anchor creation processing, 262 head frame display processing, 263 user operation detection processing, 264 user operation type determination, 265 link information change / addition processing, 266 frame dropping, color removal processing, 267 confirmation screen display processing , 271 video display,
272 Link destination data confirmation display, 273a moving image data, 273b moving image data, 274 anchor, 275
Link creation button, 301 anchor information table,
302 anchor ID, 303 start frame, 304
End frame, 305 X coordinate, 306 Y coordinate, 3
07 width, 308 height, 309a displayed anchor, 309b displayed anchor, 310 anchor information table read processing, 311 anchor information table analysis processing, 312 video / anchor display processing, 351 click mode setting means, 352 Mouse click detection means, 353 Cursor position detection means, 3
54 display speed control means, 355 video playback screen, 35
6 Mouse Cursor, 357 Anchor Attached to Moving Object, 358 Mouse Cursor Entered in Video Playback Screen, 359 Anchor Attached to Moving Object, 401
Mouse click detection means, 402 anchor vicinity determination means, 403 display speed control means, 404 mouse cursor, 405 anchor attached to moving object, 406
User movement of mouse cursor, 452 mouse click detection means, 453 cursor position detection means, 45
4 display enlarging means, 455 mouse cursor, 456
Moving anchor, 457 Moving anchor that the user wants to specify, 458 Enlarged screen, 459 Anchor that the enlarged user wants to specify, 501 Anchor notification means, 502 Audio output means, 503 Interactive display means,
511 video display processing, 512 anchor presence / absence determination, 5
13 important anchor judgment, 514 confirmation request display processing,
515 User confirmation judgment, 516 voice output / slow display processing, 517 search judgment, 518 search execution processing, 5
20a video data, 520b video data, 521
Confirmation screen display, 522 voice output / slow display, 551
Scene delimiter table creation process, 552 Anchor position size designation process, 553 Anchor information table creation process, 554 Scene delimiter table, 555 One frame in a moving image, 556 Object of interest 1,557 Object of interest 2,558 Object of interest 2,558 Object 3, 559 anchor area, 559 anchor area, 560 anchor information table, 561
Anchor selection processing, 562 Link operation determination, 563
Target object selection processing, 564 link destination selection processing, 565 function selection dialog, 566 object specification dialog, 567 object management table for scenes, 568 movies divided into scenes, 5
69 Object of Interest 4, 570 Object of Interest 5,571 Object of Interest 6

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Takeshi Maeda 5-1-1 Ofuna, Kamakura-shi Mitsubishi Electric Co., Ltd. Information Systems Research Center (72) Inventor Hajime Yoshikawa 5-1-1, Ofuna, Kamakura-shi Mitsubishi Electric Corporation Company Information Systems Laboratory

Claims (12)

[Claims] 1. A moving image hypermedia processing device having the following elements: (a) a moving image storage unit for storing a moving image; (b) an arbitrary area on the moving image storage unit as an anchor, and anchor information and link information are stored. Link data storage,
(C) User operation means for managing user input, (d)
Link management means for managing link information and anchor information in the link data storage section using the user operation means,
(E) a display attribute storage unit that stores the display attributes of the moving image and the anchor, (f) on the moving image stored in the moving image storage unit,
Display means for dynamically displaying an anchor with the display attribute stored in the display attribute storage unit based on the anchor information and the link information stored in the link data storage unit. 2. The link data storage unit stores anchor information serving as an end point of a link search and link information indicating a link destination, and the anchor information divides the entire moving image into several areas, 2. The moving picture hypermedia processing apparatus according to claim 1, wherein a link operation is performed using the area as an anchor. 3. The link data storage unit, when registering the anchor information, uses the following anchor creation method for each type of moving image data stored in the moving image storage unit. 1. The moving picture hypermedia processing device according to 1. (A) Video that moves the entire screen (b) Video that moves part of the screen 4. The moving image according to claim 1, wherein the link data storage unit allows the user to previously specify the shape of the appearing object to be used as an anchor and the movement thereof with a mouse when registering the anchor information. Hypermedia processing device. 5. The moving picture hypermedia processing apparatus according to claim 1, wherein the link data storage unit dynamically changes the link destination of the link information in relation to the elapsed time. 6. The link management means, when registering the link information, registers the link information after visually confirming the content of the moving image at the link destination. Video hypermedia processor. 7. The link management means, when displaying the anchor, highlights the created anchor so that the user can easily distinguish the anchor from the background moving image. 7. The moving picture hypermedia processing device according to any one of 1 to 6. 8. The link management means, when retrieving the link information, if the movement of the anchor is fast, when the mouse is framed in the video or when the mouse is clicked, the playback speed becomes slow. The moving picture hypermedia processing device according to claim 1, wherein 9. The link management means slows the reproduction speed when the anchor moves quickly when searching the link information, when the mouse is clicked near the anchor, or when the mouse makes a certain motion. The moving picture hypermedia processing apparatus according to any one of claims 1 to 7, wherein: 10. The link management means, when the link information is searched and the anchor is small and difficult to click, when the mouse is clicked in the moving image, the designated area of the mouse is enlarged and displayed. The moving picture hypermedia processing device according to any one of claims 1 to 7. 11. The link management means, when a certain anchor appears when searching the link information, stops displaying the moving image once and displays a selection dialog prompting the user to input. The moving picture hypermedia processing device according to claim 1. 12. The moving picture hyper according to claim 1, wherein the link management means fixes the position and size of the anchor when searching the link information. Media processing device.