JP3171632B2 - Video display processor - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture display processing apparatus for displaying a moving picture in a plurality of windows on a display.

[0002]

2. Description of the Related Art A technique for displaying a moving image on a display such as a workstation is being developed.

[0003] As a conventional technique of such a moving image display processing, an image is constructed as a bitmap for screen display at each time according to a certain time unit, and the images are displayed on a display in chronological order. Is most commonly used. In this case, the moving image is displayed independently for each window on the display.

[0004] As another conventional technique of the moving image display processing, there is the following one. First, the state of the screen at a certain time is converted into data by some method. next,
The state of the screen at an arbitrary time thereafter is similarly converted into data. Then, the state of the screen at each time during that time is obtained by interpolation from the above two sets of screen data. The moving image is displayed by sequentially displaying the screen states at each time thus obtained on the display. Also in this case, the moving image is displayed independently for each window on the display.

[0005]

However, in the above-described prior art (note: plural forms), moving images having a relevant meaning cannot be simultaneously displayed on a display. There was a problem that it was limited. For example, when a description of the operation of software in a computer system is displayed on a display, if a configuration is adopted in which a sentence of the content of the description and a moving image illustrating the operation are displayed simultaneously in synchronization with the description, it is extremely It is expected that the software operation can be easily explained. However, in the method of displaying a moving image independently for each window as in the above-described conventional technology,
Such a request could not be met at all.

In the conventional moving image display processing, a series of moving images is displayed on a display in a time-wise manner as if a film is being rotated. For this reason, the display content of the moving image is fixed, and there is a problem that the use of the moving image in a workstation or the like is limited. For example, as described above, even if a moving image could be displayed when the operation description of the software in the computer system is displayed on the display, if the user desires a more detailed description of the desired description portion, the moving image is dynamically generated. Since the display content cannot be changed, such a request cannot be met at all.

Moreover, in the conventional moving image display processing, the moving images to be displayed are managed together as if they were a film. For example, in order to change a part of the contents of the moving image, long moving image data There is also a problem that it is necessary to find the corresponding part from among the parts and edit it, which requires a great deal of trouble.

An object of the present invention is to enable a moving image having a relevant meaning to be displayed in a plurality of windows on a display.

[0009]

FIG. 1 shows the principle configuration of the present invention. The data processing device 1 executes a moving image display process. The data input device 2 allows a user to input a scenario description for displaying a moving image. The man-machine interface (MMI) 3 is a data processing device 1 and a data input device 2
Connected to.

The data processing apparatus 1 includes a moving image display unit managing means 4 and a scenario description managing means 5 realized by, for example, an auxiliary storage device, a moving image display controlling means 6, a moving image history managing means 7, an operation mode controlling means. 8 and an interface means 9.

The moving image display unit management means 4 manages a plurality of moving image display units each composed of one or a plurality of frames for describing a screen display definition. Here, the frame constituting each moving image display unit is a data set describing each screen display definition for each prescribed display unit time, and includes, for example, a type of a figure displayed on a display and characteristic information of the figure. Is described with a graphic ID.

The scenario description management means 5 includes a state such as a timing of opening or closing each window on a display and a position thereof, a temporal order relation of a moving image display unit displayed in each window, or a display in each window. Manages a scenario description that describes the synchronization relationship between each moving image display unit. Here, the temporal order relation described above is configured to include an arbitrary temporal transition relation from one moving image display unit to another moving image display unit displayed in the same or different window as the window in which it is displayed. can do. In addition, the above-described temporal transition relationship can be configured to include identification information that is valid when a predetermined screen portion in the currently displayed moving image display unit is specified by the user. Further, the above-described scenario description can be configured to include group information such that a group of a plurality of moving image display units is treated as one moving image display unit.

The moving picture display control means 6 interprets the scenario description managed by the scenario description managing means 5, and sequentially displays time-series data in moving picture display units in each window according to the interpretation.

The moving image history management means 7 manages the history of moving images displayed on the basis of moving image display units which make arbitrary temporal transitions. The operation mode control means 8 responds to an instruction from the man-machine interface 3 and
Is controlled not to operate the management function.

The interface means 9 controls the execution state of the moving picture display control means 6 in response to an instruction from the man-machine interface 3. More specifically, the interface unit 9 has a function of stopping the progress of the moving image display process of the moving image display control unit 6, and moving the moving image display unit stopped by controlling the moving image display control unit 6 back and forth one by one. And a function of moving the frames of the moving image display unit stopped by controlling the moving image display control means 6 back and forth one by one.

Next, the data input device has a moving picture input control means 10, a graphic editing means 11, a moving picture display unit editing means 12, and a scenario description editing means 13. The moving image input control means 10 includes: a man-machine interface 3;
The input / output of data between the graphic editing unit 11, the moving image display unit editing unit 12, and the scenario description editing unit 13 is controlled.

The graphic editing means 11 is a general-purpose or dedicated graphic editor, and allows the user to edit the graphic definition of each frame of each moving image display unit in a graphic format by the man-machine interface 3.

The moving image display unit editing means 12 collects the graphic definition of each frame of each moving image display unit edited by the user by the graphic editing means 11 as one moving image display unit, and converts the moving image display unit into a moving image display unit managing means. 4 is stored.

The scenario description editing means 13 provides the user with
The scenario description managed by the scenario description management means 5 is edited by the man-machine interface 3 by editing processing including graphic symbol editing processing.

Specifically, the scenario description editing means 13
Allows the user to edit the position of each window at the time of displaying a moving image in a text format on the editing window opened on the display at the time of editing, and the temporal order relation of the moving image display unit displayed in each window or The synchronization relationship between the moving image display units displayed in each window is edited in a graphic format using graphic symbols. Further, the scenario description editing unit 13 may allow the user to further edit the display timing of a moving image display unit displayed in each window at the time of displaying a moving image in a text format on the editing window.

In addition to the above-described editing of the moving image display unit, the scenario description editing means 13 provides the user with a moving image on a rectangular parallelepiped in which the display screen displayed on the editing window is three-dimensionally displayed in the time axis direction. The user is allowed to edit the timing of opening or closing each window on the display at the time of display and its position. Alternatively, the scenario description editing means 13 allows the user to edit, in a text format, the timing of opening or closing each window on the display at the time of displaying a moving image on the editing window, and edit each window at the time of displaying the moving image. The position of each window for displaying a moving image on the display is edited by disposing the window in a predetermined area on the window.

[0022]

The moving image display unit management means manages a plurality of moving image display units each composed of one or a plurality of frames that describe a screen display definition. In addition, the scenario description management unit 5 is configured to, according to the scenario description, display the state of each window on the display, the temporal order of each moving image display unit to be displayed in each window, or the time order between each moving image display unit of each window. Manage synchronization relationships. Then, in response to the management processing, the moving image display control means 6 interprets the scenario description managed by the scenario description managing means 5, and in accordance with the interpretation, displays the time series of the moving image display unit in a plurality of windows opened on the display. By displaying the data, moving images having a relevant meaning are displayed on the display.

As described above, according to the present invention, a moving image having a relevant meaning can be displayed on the display in accordance with the multi-window. For example, a sentence of the explanation content and the operation are illustrated in accordance with the explanation. This makes it possible to construct an announcement manual that displays video images while synchronizing them. In addition, since the moving image to be displayed is managed in units of a moving image display unit, there is an advantage that the editing efficiency of the moving image is improved.

Here, the temporal order relation of the moving image display unit described by the scenario description is arbitrary from one moving image display unit to another moving image display unit displayed in the same or different window as the window in which it is displayed. The temporal transition relation includes an identification information that is effective when a specific screen portion in the displayed moving image display unit is designated by the user. In this case, the moving image display control means 6 can dynamically change the display content of the moving image according to the arbitrary temporal transition relation.

Further, when the scenario description includes a configuration including group information such that a group of a plurality of moving image display units is treated as one moving image display unit, the moving image display control means 6 In display units,
The display on the display can be moved forward and backward.

The moving image history management means 7 manages the history of moving images displayed on the basis of moving image display units which make arbitrary temporal transitions. The moving image display control means 6 responds to a request from the user and In accordance with the management data of the history management unit 7, a backward display process of a moving image display unit that performs an arbitrary temporal transition is executed. Thus, even when the scenario description includes an arbitrary temporal transition relation to another moving image display unit, the temporal order can be returned.

When the moving image history management means 7 enters the operating state,
A large amount of memory is consumed to manage the video history. Therefore, in a case where the scenario description includes an arbitrary temporal transition relation to another moving image display unit, if the user does not request to return to the temporal order, the operation mode control unit 8 performs, for example, when the apparatus operates, Control is performed so that the management function of the moving image history management unit 7 is not operated. As a result, waste of memory capacity can be eliminated.

The moving picture display control means 6 stops the moving picture display processing in accordance with the control processing of the interface means 9 or moves the stopped moving picture display unit back and forth one by one or stops it. The moving image display unit is moved forward and backward by one frame. This allows
The user can accurately understand the moving image displayed on the display.

On the other hand, the user can edit the moving image display unit managed by the moving image display unit management means 4 and the scenario description managed by the scenario description management means 5 by using the data input device 2.

In particular, the scenario description can be graphically edited by the scenario description editing means 13. Specifically, on the editing window, the user edits the position of each window at the time of displaying a moving image in a text format, and displays the time order relationship of the moving image display unit displayed in each window or is displayed in each window. The synchronization relationship between the moving image display units can be edited in a graphic format using graphic symbols.

Further, if a configuration is adopted in which the display timing of a moving image display unit displayed in each window at the time of displaying a moving image on the editing window is edited in a text format, the user can control the synchronization relationship between the moving image display units. Can be generated, in which the display timing of each moving image display unit displayed on each window in a state where is stored is explicitly specified.

In addition to the above-described editing of the moving image display unit, the user can display a display screen displayed on the editing window on a rectangular parallelepiped in which a three-dimensional graphic is displayed in the time axis direction. The timing of opening or closing and the position thereof can be edited. Alternatively, on the editing window, the user can edit the timing of opening or closing each moving image display window on the display at the time of displaying a moving image in a text format, and can set each moving image display window to a predetermined time on the editing window. By arranging in the area, the position of each moving image display window on the display can be edited.

[0033]

Next, embodiments of the present invention (note: plural forms) will be described. FIG. 2 shows an embodiment of a device configuration of a moving image display device for realizing the present invention. In FIG. 2, reference numeral 10 denotes a moving image display device for realizing the present invention, and reference numeral 20 denotes a moving image display device 1.
0, an input / output device 30 for inputting a video display unit described in FIG. 1 for inputting a video display unit, a scenario description input device 31 for inputting a scenario description described in FIG.
Reference numeral 40 denotes a moving image display unit storage file that manages a moving image display unit input by the moving image input device 30, and 41 denotes a scenario description storage file that manages a scenario description input by the moving image input device 30.

First, the moving image display device 10 has the following configuration. The moving image display unit interpretation device 11 interprets a moving image display unit stored in the moving image display unit storage file 40. The scenario description interpretation device 12 interprets the scenario description stored in the scenario description storage file 41 using the interpretation result of the moving image display unit interpretation device 11. The window system 13 is, for example, an X window system, and opens a window on the display of the input / output device 20. In the moving image internal data storage file 14, moving image internal data as an interpretation result of the scenario description interpretation device 12 is developed. The moving image operation device 15 performs an interface process with the input / output device 20. The screen state creation device 16 creates screen data to be displayed on the display from the moving image internal data developed in the moving image internal data storage file 14. The screen display device 17 activates the window system 13 and displays the screen data created by the screen state creation device 16 in a window on the display.

As will be described in detail later, the moving image input device 30 incorporates a dedicated editor or a general-purpose editor, and allows the user to input / output a moving image display unit composed of one or a plurality of frames for describing screen display definitions. 20 and the moving image display unit is stored in the moving image display unit storage file 40.

FIG. 3 shows an embodiment of the moving image display unit stored in the moving image display unit storage file 40. This embodiment describes the screen display state shown in FIG. Frame that constitutes a moving image display unit
Is a data set describing each screen display definition for each specified display unit time, and the type of figure displayed on the display and the characteristic information of the figure are represented by a figure ID (identifiee).
r) is added and described.

First, the first frame describing the screen display state at time a describes seven figures. That is, the symbols “arc” to which the graphic IDs “arc0” and “arc1” are respectively described describe arc shapes. Symbol "circle" with figure ID "cir1"
Describes the shape of a circle. The figure IDs “box1” and “b
ox1 orig, each symbol “bo”
“x” describes a rectangular figure, but both overlap, and a symbol “s” with a figure ID “s3” is added.
"string" describes a figure of a character string.
The symbol "arrow" to which "al" is attached describes a figure of an arrow. Following the symbol indicating each figure type, characteristic information indicating the position and size of each figure is described. For example, a numeric string “40” described after the symbol “cir1”
0 300 20 ”means that the intermediate coordinates of the circle are“ 400, 30 ”.
"0" describes that the radius of the circle is "20".
Further, the next frame describes the screen display state at the time (a + 1) next to the time a according to the same description as described above. Here, in order to reduce the load of input processing of a moving image display unit by a user, a configuration is adopted in which a screen display definition relating to a graphic once defined can be reused by referring to a graphic ID.

Next, as will be described in detail later, the moving image input device 30 includes a scenario description that describes a temporal order relation of moving image display units displayed in a plurality of windows opened on the display, The user inputs a scenario description describing the synchronization relationship of the moving image display unit displayed on the input / output device 20, and stores the scenario description in the scenario description storage file 41.

There are two types of scenario description models input by the moving image input device 30, a film type model and a network type model. In the embodiment of FIG. 2, an apparatus configuration for handling a film type scenario description is disclosed. ing.
The scenario description of the film type is as shown in FIG.
This is a model in which the moving image display units displayed in the same window are arranged in chronological order. As shown by the broken line in FIG. 5A, the synchronization relationship between the windows is defined without breaking this chronological order. . Here, a rectangle in the figure represents a moving image display unit, and “→” represents a time order relationship.

On the other hand, the network-type scenario description is a model in which the degree of freedom of the relationship between the moving image display units is expanded, and the time sequence shown by the broken line in FIG. Interaction with
Is a model that allows moving to another moving image display unit.

Next, the basic format of the scenario description stored in the scenario description storage file 41 will be described. (1) As a scenario description instructing the creation of a window, glueOpen <window name><width><height><x><y> is described. In this description, a window having the size of <width> and <height> is opened at the coordinates of <x> and <y>, and the window has an ID of <window name>.
To give. This <window name> is used when the window containing it is referred to in another scenario description.

(2) As a scenario description instructing display of a moving image display unit, glue <glue name><windowname><file name of moving image display unit> is described. This description indicates that the moving image display unit stored in <file name of moving image display unit> is <window name>
D and display it in the video display unit <
Glue name>. In this description format, when descriptions including the same <window name> are described consecutively, a moving image display unit specified in each window having the <window name> as an ID is displayed in the order of the description. Instructs you to go. Here, <glue name> must be unique.

(3) glueClose <window name> is described as the scenario description for instructing the deletion of the window. This description instructs the deletion of the window having the opened <window name> as the ID.

(4) glueSync <Glue name 1><Glue name 2> is described as a scenario description instructing simultaneous display of moving image display units between different windows. This description indicates that a moving image display unit having ID of <glue name 1> and a moving image display unit having ID of <glue name 2> are simultaneously displayed in each window. Here, <Glue name 1> and <Glue name 2>
Must be defined.

(5) As a scenario description instructing grouping of a plurality of moving image display units as illustrated in FIG. 6, group <group name> with {group name | group name} is described. This description is {Glue name | Group name}
It is instructed that a glue or a group whose ID is (either the glue name or the group name) is made into one group and the ID of <group name> is given. This defined group name can be used in glueSync described above. In this case, a moving image display unit having the first group name of the group is used.

(6) In the network type scenario description,
As described above, an arbitrary temporal transition to another moving image display unit is permitted, and as a scenario description corresponding to this, gluebranch from <glue name 1> to <glue name 2> or gluebranch from <figure name> in <glue 1> to <glue name 2> are described. The former description instructs a transition from a moving image display unit having <Glue name 1> as an ID to a moving image display unit having <Glue name 2> as an ID, and the latter description has <Glue name 1> as an ID. <Graphic name> in the moving image display unit
When the graphic designated as D is designated by the user (by a mouse click operation), it is instructed to make a transition to the moving image display unit having <Glue name 2> as the ID.

(7) When a comment is described in the scenario description, "#" is inserted at the beginning of the line. The input of the comment facilitates the decoding of the scenario description. The scenario description stored in the scenario description storage file 41 of the embodiment of FIG. 2 is described in the basic format described above.

FIGS. 7 and 8 show an embodiment of a scenario description stored in the scenario description storage file 41 when a manual of a certain data processing device is realized as a display display by the moving image display processing device of FIG. I do.
It is assumed that the scenario description shown in FIG. 7 is followed by the scenario description shown in FIG.

In FIG. 7 and FIG. 8, first, the coordinate position “100, 100” is displayed in order to display “summary explanation”.
In addition, “s” having a size of width “640” and height “600”
After the user is instructed to open the window having the ID “tagel”, the window shows “sho
tman / kaisetsul ”,“ shotman /
kaisetsu2 "," shotman / kaise
tsu3 "," shotman / kaisetsu4
It is instructed to display the four moving image display units “in order”.

Next, in order to display the “realization diagram” following the display of the “summary explanation” described above, “bookoko / ko” is displayed.
useiz0 ”,“ shotman2 / kousei
zu1 "," shotman2 / kouseizu
2 "and" shotman2 / kouseiz4 "are displayed in the same" stagel "
Are displayed in order in a window having an ID of Thereafter, the user is instructed to delete this window.

On the other hand, after instructing to open a window having an ID “stage2” to display “output example” as a supplement to the above “summary description”, three moving image display units are displayed in this window. Are instructed to be displayed in order.

Next, in order to display the "commentary of realization" in accordance with the display of the "realization diagram" described above, it is instructed to sequentially display the six moving image display units. Thereafter, the user is instructed to delete this window.

Further, in order to display the “source file” in accordance with the display of a part of the “explanation of realization”, “s
After instructing to open a window having an ID of “tag3”, an instruction to display one moving image display unit in this window is issued. Thereafter, an instruction to erase this window is issued.

Following the comment "Synchronization", the synchronization relationship of each moving image display unit displayed in each window described above is described. For example, the glue name “kai2”
The video display unit having a “shotman / kaiset”
It is instructed to simultaneously display on the display the moving image display unit “shotman / example1” having the glue name “ex1” and the description of the synchronization relationship following the comment “synchronization” (note: 2 shows the synchronization relationship between the moving image display units defined by the plural form.In this figure, for the sake of simplicity, the actual length of the moving image display unit is appropriately modified and shown. ing.

The moving image display device 10 of FIG. 2 interprets the scenario description stored in the scenario description storage file 41 and the moving image display unit stored in the moving image display unit storage file 40, and utilizes a multi-window. Processing is performed so that a moving image (in the above example, the manual of the data processing device) is displayed effectively.

FIG. 10 shows an embodiment of the processing flow of the scenario description interpretation device 12 for interpreting a scenario description. First, in step (ST) 10, a scenario description is read from the scenario description storage file 41.

Next, at step 11, it is determined whether or not the scenario description is no longer stored in the scenario description storage file 41. If the scenario description is still present in the scenario description storage file 41 and the determination in step 11 is NO, in step 12, the format of the read scenario description is identified, and each window data, each moving image display unit,
By creating such link relationships, global variables (to be described later), and the like, moving image internal data that is the interpretation result of the scenario description is created. Here, regarding the moving image display unit, since the actual screen data is linked by activating the moving image display unit interpretation device 11 in step 14 described later, at the stage when step 12 is executed, A part (hereinafter, referred to as partial moving image internal data) is created.

FIG. 12 shows an embodiment of the partial moving image internal data created based on the operation of the scenario description interpretation device 12 described above. For example, if the scenario description read from the scenario description storage file 41 is the above-described (1) scenario description “glueOpen”, a “window” structure as shown in FIG. The window system 13 in FIG. 2 is started using the width, height, x-coordinate, and y-coordinate of the window based on the description as parameters, and control data for opening the window, for example, Wi in the case of the X Window system.
Control data called dget is created. And
A pointer to the control data is set as the data of the above-mentioned "window" structure, and other data in the structure is set appropriately.

When the scenario description read from the scenario description storage file 41 is the above-described scenario description “glue” of (2), a “movie display unit” structure as shown in FIG. 12 is created. , A file name, a glue name, etc. are set as data, and a pointer to this structure is set as the last data of the moving image display unit list of the “window” structure relating to the window specified by the window name.

If the scenario description read from the scenario description storage file 41 is the above-described scenario description “glueSync” of (4), the synchronization of the “movie display unit” structure as shown in FIG. One new element is added as the data of the list, and a pointer to a “moving image display unit” structure having a synchronous relationship is set as the new element. Similar settings are made in the “moving image display unit” structure having a synchronous relationship.

The scenario description read from the scenario description storage file 41 is the same as the scenario description “gluebranch from <glue name 1> to <glue name 2” described in (6) above.
>>, the new element is 1 as the data of the trigger destination list of the “movie display unit” structure of glue name 1
And a pointer to a “moving image display unit” structure of glue name 2 is stored as an element.

If the scenario description read from the scenario description storage file 41 is the scenario description “gluebranch from <figure name> in <glue 1> to <glue name 2>” described in (6) above, One new element is added as data of the trigger destination list of the “movie display unit” structure of the glue name 1, and the figure name in the specified text format and the “movie display unit” structure of the glue name 2 are added as the elements. A pointer to the body is stored.

As described above, when the partial moving image internal data is created in accordance with the read scenario description,
The processing is returned to step 10 in FIG. 10, the next scenario description is read from the scenario description storage file 41, and in step 12, partial moving image internal data corresponding to the scenario description is created.

As a result of repeating the above processing, there is no scenario description to be read from the scenario description storage file 41, and if the determination in step 11 becomes YES,
Step 13 is executed.

In step 13, the file name of each moving image display unit is extracted by following each moving image display unit of the created partial moving image internal data. When the file name is extracted, the determination in step 14 is NO, and in step 15, the moving image display unit interpreting apparatus 11 is activated based on the file name of each moving image display unit thus extracted.

FIG. 11 shows an embodiment of the processing flow of the moving image display unit interpreting device 11 for interpreting the moving image display unit. First, in step 20, by referring to the file name of the moving image display unit described above, data of the moving image display unit corresponding to the file name is read from the moving image display unit storage file 40.

Next, in step 21, it is determined whether or not all the data in the referenced moving image display unit has disappeared. If there is still data in the moving image display unit and the determination in step 21 is NO, in step 22, screen data to be displayed is created according to the read data of the moving image display unit, and the created screen data is stored in a scenario. The partial moving image internal data corresponding to the file name created by the description interpretation device 12 is linked. That is, the screen data is created, for example, in a form including the shape, size, and position of each figure as one structure and included in the partial moving image internal data in the form of a list. A pointer to the list is as shown in FIG. This is set as the screen data of the “moving image display unit” structure.

As a result, complete moving image internal data is created, and the created moving image internal data is stored in the moving image internal data storage file 14. afterwards,
Control is returned to the scenario description interpretation device 12, and the scenario description interpretation device 12 returns control to the process of step 13 in FIG. Then, by following each moving image display unit of the created partial moving image internal data, the file name of the next moving image display unit is extracted, and based on that, the moving image display unit interpreter 11 is activated to create screen data. Then, the moving image internal data is created.

As a result of repeating the above processing, there is no longer a file name of the moving image display unit to be traced, and if the determination in step 14 is YES, the moving image operating device 15 of FIG. On the basis of the moving image internal data thus created, a moving image (in the above example, a manual for the data processing device) is displayed on the display of the input / output device 20. That is, first, the moving image operation device 15 displays a control panel window described later on the display of the input device 20. When the user instructs the display of a moving image according to the control panel, the screen state creating device 16 creates the screen data by following the moving image internal data stored in the moving image internal data storage file 14.
Displays the screen data on the display using the window system 13. In this way, a moving image can be displayed on the display in a multi-window format. Regarding the operation of the screen state creation device 16,
This will be described in detail later with reference to FIG.

FIGS. 13 to 22 show a part of a manual of a data processing apparatus which is an example of a moving image displayed on a display in accordance with the above-described scenario description of FIGS. 7 and 8 based on the above moving image display processing. I do.

FIG. 13 is a diagram showing the arrangement of windows on the display screen. In the figure, reference numeral 50 denotes the control panel shown in FIG.
a window having an ID of “e1”; a window 52 having an ID of “stage2”;
Is a window having the ID “stage3” described above.

FIG. 14 is a diagram showing the screen configuration of the control panel 50. The control panel 50 includes a “play” key for instructing reproduction of the animation, a “back” key for instructing rewind, and “s” for instructing stop.
a "top" key, a "head" key for instructing movement to the head, a "tail" key for instructing movement to the end, a "fr ++" key for instructing frame advance, a "fr--" key for instructing frame return, A “sh ++” key for instructing a forward movement between moving image display units, a “sh−−” key for instructing a backward movement between moving image display units, and the like are provided.

Now, in FIG. 13, the window display at time t1 corresponds to the screen example 1 in FIG. On this screen, a window 51 having an ID of “stage1” is opened.
“Summary commentary” illustrated in FIG. 15 corresponding to the moving image display unit having the ID “tman / kaisetsu1” is displayed.

Next, the window display at time t2 in FIG. 13 corresponds to the screen example 2 in FIG. On this screen, a window 5 having an ID “stage1” is displayed.
In FIG. 16, the “summary explanation” illustrated in FIG. 16 corresponding to the moving image display unit “shotman / kaisetsu3” is displayed. At the same time, “stage2”
A window 52 having an ID of “shotman / example” is opened.
An "output example" illustrated in Fig. 17 corresponding to the moving image display unit "2" is displayed. When the window 52 is opened, a bell is ringed to draw the user's attention. .

Subsequently, the window display at time t3 in FIG. 13 corresponds to screen example 3 in FIG. On this screen, a window 51 having an ID of “stage1” has a “realization diagram” illustrated in FIG. 18 corresponding to a moving image display unit of “bookoko / kouseizu0”.
Is displayed. At the same time, a window 52 having an ID of “stage2” is displayed as “shotman2 /
jitsugen2 ”corresponding to the video display unit,
The “commentary of realization” illustrated in FIG. 19 is displayed. At the same time, a window 53 having an ID “stage3” is opened, and “sho3” is displayed in this window.
A “source file” illustrated in FIG. 20 corresponding to a moving image display unit of “tman2 / examplefile” is displayed.

Further, the window display at time t4 in FIG. 13 corresponds to screen example 4 in FIG. On this screen, a window 51 with an ID “stage1” is displayed.
A “realization diagram” illustrated in FIG. 21 corresponding to a moving image display unit “booko / kouseiz0” is displayed, and a window 5 having an ID “stage2” is displayed.
In FIG. 2, “commentation of realization” illustrated in FIG. 22 corresponding to the moving image display unit “shotman / jitsugen3” is displayed. At the same time, the window 53 having the ID “stage3” is deleted.

As in the above example, the manual of the data processing apparatus can be effectively displayed as a moving image according to the multi-window. Another Embodiment of Moving Image Display Device Next, FIG. 23 shows another embodiment of the moving image display device for realizing the present invention. In FIG. 23, portions denoted by the same reference numerals as those described in FIG. 2 have the same functions.

This embodiment discloses an apparatus configuration for handling the network-type scenario description described with reference to FIG. 5B, and has a function of dynamically changing a moving image.

In the film type scenario description described with reference to FIG. 5 (a), since the display units of the moving image are arranged in chronological order, the screen state at an arbitrary time can be easily reproduced, and the time order of the moving image can be reversed. On the other hand, in the network-type scenario description, the display content changes according to the situation, and therefore, it is not possible to realize the reversal of the time order of the moving image in the configuration of the embodiment in FIG. Therefore, in the embodiment of FIG. 23, the history data storage file 18 is provided, and the transition of the moving image display unit is recorded in the history data storage file 18. According to this record, the reverse operation of the time order of the moving image in the network type scenario description is performed. Is realized.

Here, if the transition of the moving image display unit is recorded in the history data storage file 18 without any limitation, a large memory capacity is consumed. Therefore, in the embodiment of FIG. 23, there is provided a model selecting device 19 for selecting whether or not the user has to request that the time order be returned in accordance with the interactive processing with the user. When not selecting is requested, a configuration is adopted in which the storage processing in the history data storage file 18 is controlled so as not to be executed, thereby wasting memory.

FIGS. 24 and 25 show one embodiment of the scenario description stored in the scenario description storage file 41 of the embodiment of FIG. 23. FIG. 26 shows the moving image internal data storage file 14 of the embodiment of FIG. One embodiment of the moving image internal data to be stored is illustrated. It is assumed that the scenario description shown in FIG. 25 follows the scenario description shown in FIG.

24 and 25, (i) to (iv) show the scenario description "glue
branch from ". As shown in (a) to (c) of FIGS. 24 and 25, the scenario description" gr "of (5) instructing grouping of moving image display units is described.
oup "is described. When such grouping is performed, for example, a moving image representing a document having a chapter section structure can be displayed, and the chapter sections can be moved. The detailed operation of the screen state creation device 16 will now be described in detail with reference to the processing flow shown in Fig. 27 executed by the screen state creation device 16. Here, the screen state creation is performed. If the device 16 is implemented on the embodiment of FIG. 2, only the film type scenario description is handled, so that steps 31 to 31 are performed.
When only the process 36 is executed unconditionally and the screen state creation device 16 is realized on the embodiment in FIG. 23, both the film type scenario description and the network type scenario description are handled. The processes from 30 to 42 are executed. Hereinafter, for the sake of simplicity, the description will be made assuming that the screen state creation device 16 is realized on the embodiment of FIG.

First, in step 30, the above-described global variables are read from the moving image internal data storage file 14 and their set values are determined, thereby determining whether the scenario description is of a film type or a network type. Is done.

If it is determined in this determination process that the scenario description is of a film type, an input from the user on the control panel is awaited in step S31.
If there is an input, it is determined in step 32 whether or not the input is an input for instructing the end of the input.

In this determination processing, the description will be made assuming that the input is an input other than the end of the input and that the input is realized above. By reading the above-mentioned global variables and determining their set values, it is determined whether the scenario description is of a film type or a network type. This determination is made. At this time, if the time exceeds the start / end time, an appropriate moving image display unit before and after is searched for, and a similar process is executed for that. When the input is, for example, an instruction of “moving to the beginning / end of a moving image”,
The time is set as the start / end time of the scenario description, and the screen data of the moving image display unit at that time is created for the window opened at that time.

Subsequently, at step 34, the screen display device 1
7 is activated. Thus, the screen display device 17 displays each figure indicated by the screen data in the corresponding window using the drawing function of the window system 13.

In the following step 35, it is determined whether the input from the control panel is "playback / return" or something else. If the input is “playback / return”, an input of “playback / return” is automatically generated in step 36, and control is returned to step 33. Thus, the reproduction / return processing of the moving image display unit is continuously performed by one input of “reproduction / return”.

If the input is other than "reproduction / return", the process returns from step 35 to step 31 to wait for input from the control panel. on the other hand,
If it is determined in the determination processing in step 30 that the scenario description is of the network type, an input from the user on the control panel is awaited in step 37.

If there is an input, it is determined in a step 38 whether or not the input is an input for instructing the end of the input. In this determination processing, if it is determined that the input is an input other than the end of the input, in step 39, screen data is created. Specifically, if the input is, for example, an instruction of “playback / return”, the moving image display unit list currently displayed in the moving image internal data storage file 14 is traced, and the next / previous screen data being displayed is displayed. Screen data is created. At this time, if there is no screen data to be displayed, the first screen data of the next / previous moving image display unit of the current moving image display unit is created. If the input is, for example, an instruction of “clicking on a screen”, a moving list (see FIG. 26) is retrieved from the clicked moving image display unit and the graphic ID and displayed in the window.

Subsequently, at step 40, the screen display device 1
7 is activated. Thus, the screen display device 17 displays each figure indicated by the screen data in the corresponding window using the drawing function of the window system 13.

In the following step 41, it is determined whether the input from the control panel is "reproduction / return" or something else. If the input is “playback / return”, an input of “playback / return” is automatically generated in step 42, and control is returned to step 39. Thus, the reproduction / return processing of the moving image display unit is continuously performed by one input of “reproduction / return”.

In this way, the screen state creation device 16
Executes a process of creating screen data according to the type of scenario description. If the input is something other than "playback / return", the process returns from step 41 to step 37 to wait for input from the control panel. One Embodiment of Moving Image Input Device 30 Lastly, FIG. 28 shows an embodiment of the moving image input device 30 shown in FIG. 2 or FIG. 23, and its operation is shown in FIG. 29 to FIG. Will be explained.

First, in step 50 of FIG. 29, the moving image input device 301 selects one of the moving image display units and the scenario description to be edited in accordance with an instruction from the input / output device 20 by the user.

When the user instructs editing of a moving image display unit, the moving image input device 301 activates the moving image display unit file control device 302. Now, each moving image display unit is managed on a file basis. Therefore, the moving image display unit file control device 302 searches for a file corresponding to the file name specified by the user in the moving image display unit storage file 40 in step 51 of FIG. If the file does not exist, the file is selected as an object to be edited. After that, the moving image display unit file control device 302
Start 3

Now, editing of a moving image display unit is performed in a frame (time) unit. Therefore, the moving image display unit within frame designation device 303 specifies the frame specified by the user as an object to be edited on the file set by the moving image display unit file control device 302 in step 52, and in step 53, the graphic editor Activate 304. At this time, if the file already exists in the moving image display unit storage file 40 and the specified frame exists in the file, the data of the moving image display unit of the frame is read into the graphic editor 304 and displayed. You.

The graphic editor 304 executes steps 54 and 5
By repeating step 5, the user edits the screen data of the corresponding frame. As the graphic editor 304, an existing general-purpose editor can be used, and a new dedicated graphic editor can be developed based on a known technique. At this time, the transfer of graphic data between the data input / output device 20 of FIG. 2 or FIG.

When the user instructs the end of editing in the graphic editor 304, the determination in step 55 becomes YES, and the edited result is stored as graphic editor creation data 305 in a disk storage device or the like. After that, the moving image display unit file control device 302
Start

The graphic data analysis device 306 executes step 5
In step 6, referring to the format definition 307 of the moving image display unit, frame data of the moving image display unit expressed in the internal text format as illustrated in FIG. 3 is generated.

The moving image display unit file control device 302
In step 57, the frame data of the moving image display unit generated as described above is added to the specified file,
Alternatively, the corresponding frame data on the file is replaced by the frame data.

If the editing of another frame is instructed by the user, the moving image display unit file control device 302 returns the control from step 58 to step 52, and repeats the control processing for editing a new frame.

If the user instructs to edit another moving image display unit, the moving image display unit file control device 3
02 returns control from step 59 to step 51,
The control process for editing a new moving image display unit is repeated.

When the user instructs to edit the scenario description, the moving image input device 301 activates the scenario description editing device 308. In step 60, the scenario description editing device 308 follows the editing screen format defined in advance as the scenario description format definition 309,
A scenario description editing screen is displayed, and by repeating steps 61 and 62, the user is made to edit the scenario description on the editing screen as shown in FIG. 32, for example. At this time,
The moving image input control device 301 controls the transfer of data between the data input / output device 20 and the scenario description editing device 308 in FIG.

The user specifies the name, coordinates, size, width, height, etc. of each window, for example, by filling the corresponding items in the table on the screen. 32, the order of the rectangle displayed together with the character string representing the file name is edited by operating the mouse or the like.

If the user wants to trigger a transition from a predetermined figure in a predetermined moving image display unit to another moving image display unit by clicking the mouse when displaying a moving image, for example, as shown in FIG. A desired graphic name (ID) is input in a rectangle corresponding to a moving image display unit allowing a trigger, and an arrow is moved from a rectangle corresponding to a moving image display unit allowing a trigger to a rectangle corresponding to a desired moving image display unit. Set.

When the editing of the scenario description is instructed by the user, the determination in step 62 becomes YES, and in step 63 the editing result is stored as scenario description screen data 310 in a disk storage device or the like. Thereafter, in step 64, the scenario description editing device 308 activates the scenario description input data analysis device 311.

Scenario description input data analyzer 311
In step 65, scenario description screen data 310,
The editing screen format defined in advance as the scenario description format definition 309 and the scenario description format in the internal text format defined in advance as the scenario description format definition 309 are referred to in FIGS. It generates scenario description data expressed in internal text format such as As described with reference to FIGS. 7 and 8, the scenario description to be generated includes the position of each window, the order relation of the moving image display units in the window, the synchronization relation between the moving image display units on different windows, Alternatively, information such as a graphic for activating a transition to an arbitrary moving image display unit and a moving image display unit of a transition destination is described in a format that can be recognized by the scenario description interpretation device 12 in FIG. 2 or FIG. 23 described above.

Scenario description input data analysis device 311
In step 66, the scenario description generated as described above is stored as the scenario description storage file 41 in FIG. 2 or FIG.

As described above, the scenario description storage file 4
1 is interpreted by the scenario description interpretation device 12 of FIG. 2 or FIG. in this case,
As for the progress time of each frame of each moving image display unit displayed on each window according to the scenario description, for example, an appropriate display time is assigned to each frame so that the synchronization relationship between the above moving image display units is preserved. Is automatically determined by this. Another Embodiment of Moving Image Input Device 30 The configuration and operation of the moving image input device 30 of FIG. 2 or FIG.
The present invention is not limited to those shown in FIGS. 28 to 32 described above.

In the first other embodiment, for example, FIG.
The logical time is displayed on the axis of “time” of the order relationship of the moving image display units, and the user sets the order relationship of each moving image display unit by the above-described rectangle and the numerical value representing the above logical time. To enter. In this way, the user generates a scenario description in which the display timing of each moving image display unit displayed on each window in a state where the synchronization relationship between the moving image display units is stored is explicitly specified. be able to. In this case, the display time of each moving image display unit in the moving image internal data obtained as the moving image internal data storage file 14 by the scenario description interpreting device 12 of FIG.
2 is set as start time data and end time data in the “moving image display unit” structure.

In the second other embodiment, as shown in FIG. 33, a rectangular parallelepiped in which the surface representing the display screen is superimposed in the direction of the time axis on the graphic editor is displayed. Specify and edit the position, size, display timing, etc. of the window with. As a result, the user can generate a scenario description including a description for controlling the window. Then, for each window state set in this way, the user edits the state of each moving image display unit on the editing screen shown in FIG. 32, for example, and describes the scenario description corresponding to the editing operation. Generate In this case, in the moving image internal data obtained as the moving image internal data storage file 14 by the scenario description interpreting device 12 of FIG. 2 or FIG.
Are set as open time data and close time data in the "window" structure of the "."

In the third embodiment, as shown in FIG. 34, a two-dimensional window screen area and areas "A" and "B" for inputting a logical time are displayed on the graphic editor. The user designates and edits the position, size, display timing, and the like of the window using these two areas. As a result, as in the case of the second other embodiment, the user can generate a scenario description including a description for controlling the window. Then, for each window state set in this way, the user edits the state of each moving image display unit on the editing screen shown in FIG. 32, for example, and describes the scenario description corresponding to the editing operation. Generate

[0112]

According to the present invention, it is possible to display a plurality of moving image screens in mutually associated windows by using a multi-window, and it is possible to greatly improve the understanding of the contents of the moving image. Become.

Also, by associating moving images in time and space in smaller moving image display units and associating them with windows, it is possible to dramatically improve the freedom of expression.

Further, since the moving image can be re-used as parts by decomposing the moving image into several parts, it is possible to easily edit the moving image for a long time. Efficiency is also dramatically improved.

Further, since the scenario description can be graphically edited, the editing of the moving image is further facilitated.

[Brief description of the drawings]

FIG. 1 is a principle configuration diagram of the present invention.

FIG. 2 is an overall configuration diagram of an embodiment of the present invention.

FIG. 3 is a diagram showing an embodiment of a description format of a moving image display unit.

FIG. 4 is an explanatory diagram of a description format of a moving image display unit.

FIG. 5 is an explanatory diagram of a scenario description model.

FIG. 6 is an explanatory diagram of a structured moving image display unit.

FIG. 7 is a diagram showing an embodiment of a scenario description (part 1);
It is.

FIG. 8 is a diagram showing an embodiment of a scenario description (part 2).
It is.

FIG. 9 is an explanatory diagram of a synchronization relationship of a moving image display unit in one embodiment of the scenario description (FIGS. 7 and 8).

FIG. 10 is a diagram showing a processing flow of the scenario description interpretation device.

FIG. 11 is a diagram showing a processing flow of the moving image display unit interpretation device.

FIG. 12 is a diagram showing an example of moving image internal data to be created.

FIG. 13 is an explanatory diagram of a manual moving image display process.

FIG. 14 is a diagram showing one embodiment of a control panel display.

FIG. 15 is an explanatory diagram (part 1) of screen data displayed on a window.

FIG. 16 is an explanatory view (2) of screen data displayed on a window.

FIG. 17 is an explanatory view (3) of screen data displayed on the window.

FIG. 18 is an explanatory diagram (part 4) of screen data displayed on a window.

FIG. 19 is an explanatory view (5) of screen data displayed on a window.

FIG. 20 is an explanatory view (part 6) of screen data displayed on a window.

FIG. 21 is an explanatory view (No. 7) of screen data displayed on the window.

FIG. 22 is an explanatory view (8) of screen data displayed on the window.

FIG. 23 is an overall configuration diagram of another embodiment of the present invention.

FIG. 24 is a diagram (part 1) illustrating an embodiment of a scenario description.

FIG. 25 is a diagram (part 2) illustrating an embodiment of a scenario description;

FIG. 26 is a diagram showing an example of moving image internal data to be created.

FIG. 27 is a diagram showing a processing flow executed by the screen state creation device.

FIG. 28 is a diagram illustrating an example of a configuration of a moving image input device.

FIG. 29 is a diagram (part 1) illustrating a processing flow of the moving image input device;

FIG. 30 is a diagram (part 2) illustrating a processing flow of the moving image input device.

FIG. 31 is a diagram (part 3) illustrating a processing flow of the moving image input device.

FIG. 32 is a diagram showing an editing screen of a moving image display unit using a two-dimensional table.

FIG. 33 is a diagram showing a window editing screen in a rectangular parallelepiped display.

FIG. 34 is a diagram showing a window editing screen by two-dimensional screen display.

[Explanation of symbols]

 Reference Signs List 1 data processing device 2 data input device 3 man-machine interface 4 video display unit management means 5 scenario description management means 6 video display control means 7 video history management means 8 operation mode control means 9 interface means 10 video input control means 11 graphic editing Means 12 Movie display unit editing means 13 Scenario description editing means

──────────────────────────────────────────────────続 き Continued on the front page (58) Fields surveyed (Int. Cl. ⁷ , DB name) G06T 13/00 G09G 5/00-5/40

Claims (13)

(57) [Claims] 1. A moving image display processing device for displaying a moving image on a plurality of windows opened on a display, comprising a plurality of moving image display units each including one or more frames for describing a screen display definition. A moving image display unit managing means (4) for managing minutes, a state of each window on a display, and a temporal order relation of the moving image display units displayed in each window are created in a list format including a structure; Alternatively, the synchronization relationship between the moving image display units displayed in each of the windows is structured.
A scenario description management means (5) for managing scenario descriptions which are created in a list format including a body and describe each of them; and a scenario description managed by the scenario description management means (5) is interpreted, and each of the windows is interpreted in accordance with the interpretation. And a moving image display control means (6) for sequentially displaying the time series data of the moving image display unit. 2. The temporal order relation includes an arbitrary temporal transition relation from one moving image display unit to another moving image display unit displayed in the same or different window as the window in which the moving image display unit is displayed. The moving image display processing device according to claim 1, wherein: 3. The temporal transition relation includes identification information that becomes valid when a predetermined screen portion in a currently displayed moving image display unit is designated by a user. Item 3. The moving image display processing device according to item 2. 4. A moving image history management unit (7) for managing a moving image display history for each moving image display unit displayed in each window and performing an arbitrary temporal transition according to the temporal transition relationship, The moving image display control means (6) is configured to perform, for each of the moving image display units displayed in each of the windows and performing an arbitrary temporal transition in accordance with the temporal transition relationship according to the history data managed by the moving image history managing means (7), The moving image display processing device according to claim 2, wherein a reverse display process of moving image display is performed. 5. An operation mode control means (8) for controlling not to operate a history management function of said moving picture history management means (7) in response to an instruction from a man-machine interface (3). The moving image display processing device according to claim 4, wherein: 6. The scenario description according to claim 1, wherein the scenario description includes group information such that a group of the plurality of moving image display units is treated as one moving image display unit. Moving image display processing device. 7. The system according to claim 1, further comprising an interface means for controlling an execution state of said moving image display control means in response to an instruction from a man-machine interface. The moving image display processing device according to claim 1. 8. The interface means (9) has a function of stopping the moving image display processing of the moving image display control means (6) and the function of stopping the moving image display control means (6). A function of moving the moving image display units back and forth one by one, and a function of moving the frames of the moving image display units stopped by controlling the moving image display control means (6) one by one, The moving image display processing device according to claim 7, wherein: 9. A moving image display processing device for displaying a moving image on a plurality of windows opened on a display, comprising a plurality of moving image display units each including one or more frames for describing a screen display definition. A moving image display unit managing means (4) for managing minutes, a state of each window on a display, and a temporal order relation of the moving image display units displayed in each window are created in a list format including a structure; Alternatively, the synchronization relationship between the moving image display units displayed in each of the windows is structured.
A scenario description management means (5) for managing scenario descriptions which are created in a list format including a body and describe each of them; and a scenario description managed by the scenario description management means (5) is interpreted, and each of the windows is interpreted in accordance with the interpretation. Moving image display control means (6) for sequentially displaying the time series data of the moving image display unit, and a user edits a scenario description managed by the scenario description managing means (5) by an editing process including a graphic symbol editing process. A moving image display processing device, comprising: 10. The scenario description editing means (13)
Allows a user to edit the position of each window at the time of displaying the moving image in a text format on an editing window opened on the display at the time of editing, and to change the time of the moving image display unit displayed in each window. The moving picture display processing device according to claim 9, wherein the order relation or the synchronization relation between the moving picture display units displayed in each window is edited in a graphic format using graphic symbols. 11. The scenario description editing means (13)
11. The moving image according to claim 10, wherein, on the editing window, the user further edits a display timing of the moving image display unit displayed in each window when the moving image is displayed in a text format. 12. Display processing device. 12. The scenario description editing means (13).
The user is required to open the display screen displayed on the editing window opened on the display at the time of editing on a rectangular parallelepiped with three-dimensional graphics displayed in the time axis direction, and to open each window on the display when displaying a moving image 12. The moving image display processing device according to claim 10, wherein a closing timing and a position of the closing are edited. 13. 13. The scenario description editing means (13).
Allows the user to edit the timing of opening or closing each window on the display at the time of displaying a moving image in a text format on the editing window opened on the display at the time of editing, and to display each window at the time of displaying the moving image. The position on the display of each window when the moving image is displayed is arranged by arranging in a predetermined area on the editing window, whereby the moving image is displayed. Video display processing device.