JPH07192069A - Operation history recording and reproducing device - Google Patents

Abstract

PURPOSE:To reproduce operation history information and the pictures of the external condition of an operator while linking them with each other and to easily confirm and improve the problem of stacks. CONSTITUTION:When an operation for operating the stack is performed or when a card is displayed on a display device 1, the frame number of the pictures of the external condition of the operator recorded by a VTR 5 is received from a VTR control part 6 and is recorded in an operation history information management part 13 as the operation history information, an operation history reproduction part 9 reads the recorded operation history information and the VTR control part 6 retrieves the pictures based on the frame number in the operation history information. Or, the operation history reproduction part 9 receives the frame number from the pictures reproduced by the VTR 5 and retrieves the operation history information based on the frame number. Thus, the pictures and the operation history information are linked with each other and reproduced.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an operation screen of an information device that interacts with a user using a screen display device, such as an automatic teller machine (ATM) in a financial institution such as a bank. The present invention relates to an operation history recording / reproducing device that simulates a stack operation and records and reproduces the operation history.

[0002]

2. Description of the Related Art To create an operation screen, which is a user interface of an information device, in a GUI (graphical user interface) environment, for example, Nikkei Byte 19
N introduced in the March 1991 issue p250-p254
INTER's Inter-face Build
r and Devg from Sun Microsystems
guide (developer's graphica
l userinterface design ed
, such as display buttons and scroll bars (sometimes called media objects) are placed on the card to create multiple cards, and these multiple cards are linked and connected. This makes it possible to easily create a series of operation screens (also called stacks).

The operation screen thus created is actually operated by the operator, and the operation contents at that time are recorded as operation history information. This operation history information is played back to find problems in the operation of the operation screen due to an operator's operation mistake, etc., and is used to improve some of the operation procedures by automating them. There is.

[0004]

Since the operation history information as described above can record the contents of the operation performed by the operator on the operation screen, it is necessary to shift from one screen to another. You can know exactly when you did.
However, for example, if it takes longer than expected to perform one operation after the next, the cause may be whether the operation was unknown or something unrelated to the operation was performed. When you want to make a decision, you cannot know the external situation such as the operator's facial expression and situation that is the material for that decision, so it is inconvenient when you find a problem on the operation screen and make an improvement. There was a point.

Further, in order to know the external situation of the operator, the situation of the operator operating the created operation screen is imaged and recorded on a video tape recorder (hereinafter simply referred to as VTR). , There is a method to reproduce this recorded image, but in this case, although the external situation of the operator can be reproduced,
Since it is difficult to measure exactly what button the operator operated, or the time it took to move from one screen to the next, etc. When making improvements, there was still the problem of inconvenience.

Therefore, the present invention provides an operation history recording / reproducing apparatus which can reproduce the operation history information and the image of the external situation of the operator in association with each other, and can easily confirm the problem of the operation screen and improve it. The purpose is to do.

[0007]

According to an operation history recording / reproducing apparatus of the present invention, a stack created by setting a connection relationship between a plurality of cards is operated according to an operation performed by an operator,
An operation history recording / reproducing apparatus for recording a plurality of operation history information at that time and reproducing the operation history information, wherein a rule management means for managing a connection relationship between a plurality of cards in the stack and the operator Image recording means for recording an image of an external situation when performing an operation of operating the stack;
When an operation for operating the identification information managing means for managing the identification information for identifying the plurality of images recorded by the image recording means and the stack is performed, the rule managing means manages the operation according to the operation. The card display means for displaying the card based on the coupling relationship, and when the card is displayed on the card display means, the identification information for identifying the image recorded by the image recording means at that time is identified. When the operation of operating the stack and the first reading means for reading from the information management means is performed, the identification information for identifying the image recorded by the image recording means at that time is the identification information management means. Second read means for reading from, operation history information recording means for recording operation history information including the identification information read by the first and second read means, and this operation Operation history information display means for displaying the contents of the operation history information recorded by the history information recording means, image search means for searching the image recorded by the image recording means based on the identification information, and this image And an image reproducing means for reproducing the image searched by the searching means.

Further, the operation history recording / reproducing apparatus of the present invention is
An operation history recording / reproducing apparatus that operates a stack created by setting a coupling relationship between a plurality of cards according to an operation performed by an operator, records a plurality of operation history information at that time, and reproduces the operation history information. And a rule management means for managing a connection relationship between a plurality of cards in the stack, an image recording means for recording an image of an external situation when the operator operates the stack, and the image recording means. Identification information management means for managing identification information for identifying a plurality of images recorded by the recording means, and a connection relationship managed by the rule management means in accordance with the operation when the stack is operated. Based on the card display means for displaying the card, and when the card is displayed on the card display means, the image recorded by the image recording means at that time is identified. First reading means for reading the identification information for reading from the identification information management means, and for identifying an image recorded by the image recording means at the time when an operation for operating the stack is performed. Second reading means for reading the identification information from the identification information management means;
Operation history information recording means for recording operation history information including identification information read out by the first or second reading means, image reproduction means for reproducing an image recorded by the image recording means, and An information retrieval means for retrieving the operation history information managed by the operation history information management means based on the identification information, and an operation history information display means for displaying the contents of the operation history information retrieved by this information retrieval means. It has and.

Further, the operation history recording / reproducing apparatus of the present invention is
An operation history recording / reproducing apparatus that operates a stack created by setting a coupling relationship between a plurality of cards according to an operation performed by an operator, records a plurality of operation history information at that time, and reproduces the operation history information. A layout means for laying out media objects on each card, which is a basic unit of a plurality of displays, and a link means for setting a connection relationship between the media objects laid out on the card by the layout means; Layout information management means for managing layout information of media objects laid out on the card by the layout means, information management means for managing information of each media object laid out by the layout means, and layout by the layout means. Bonding between multiple cards A stack creating means for setting a stack to create a stack, a rule managing means for registering and managing a connection relationship between a plurality of cards of the stack created by the stack creating means, and the operator operating the stack. Image recording means for recording an image of an external situation when performing the operation, identification information management means for managing identification information for identifying the image recorded by the image recording means, and operation for operating the stack. When performed, the card display means for displaying the card based on the connection relationship managed by the rule management means according to the operation, and the card recording means when the card is displayed on the card display means, the image recording means at that time A first reading means for reading the identification information for identifying the image recorded in step 1 from the identification information management means, and an operation for operating the stack. When read, the second reading means for reading the identification information for identifying the image recorded by the image recording means at that time from the identification information managing means and the first and second reading means for reading The operation history information recording means for recording the operation history information including the issued identification information, the operation history information display means for displaying the contents of the operation history information recorded by the operation history information recording means, and the identification information. Further, it is provided with an image searching means for searching the image recorded by the image recording means, and an image reproducing means for reproducing the image searched by the image searching means.

Further, according to the operation history recording / reproducing apparatus of the present invention, a stack created by setting a connection relationship between a plurality of cards is operated according to an operation performed by an operator, and a plurality of operation history information at that time is recorded. An operation history recording / reproducing apparatus for recording and reproducing these operation history information, wherein a layout means for laying out media objects on a card, which is a basic unit of a plurality of displays, and a layout means for laying out on the card. Link means for setting the connection relationship of the media objects that have been laid out, layout information management means for managing the layout information of the media objects laid out on the card by the layout means, and each of the media objects laid out by the layout means. An information management means for managing information, and A stack creating means for creating a stack by setting a bond relationship between a plurality of cards laid out by a layout means, and a rule for registering and managing a bond relationship between a plurality of cards of a stack created by this stack creating means Management means, image recording means for recording an image of an external situation when the operator operates the stack, and identification for managing identification information for identifying the image recorded by the image recording means Information management means,
When an operation for operating the stack is performed, a card display means for displaying the card based on the connection relationship managed by the rule management means according to the operation, and a card displayed by the card display means When the operation for operating the stack is performed, first reading means for reading the identification information for identifying the image recorded by the image recording means from the identification information management means at that time,
At that time, second reading means for reading the identification information for identifying the image recorded by the image recording means from the identification information managing means, and identification read by the first or second reading means. Operation history information recording means for recording operation history information including information, image reproduction means for reproducing an image recorded by the image recording means, and management by the operation history information management means based on the identification information The information retrieving means for retrieving the retrieved operation history information and the operation history information displaying means for displaying the contents of the operation history information retrieved by the information retrieving means are provided.

[0011]

When the stack is operated by the operator, the operation history information is recorded by recording, as the operation history information, the identification information for identifying the captured image of the operator's external situation together with the operation content. The images can be played back in conjunction with each other, and problems in the stack can be confirmed and their improvement can be easily performed.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. First, before describing the embodiments, terms used in the following description will be defined. Lay out media information such as text, graphics, images, moving images, and audio in a window of any size on the display screen,
The unit is a card. As a part laid out on a card, a set of media information and functions is called a media object, and a part that expresses the relationship between media objects is called a link. A link is a one-way pointer to a reference partner. , Cards are media objects with no more parents. Depending on the media object laid out on the card, there are some that create a new card.By linking such a media object and a card, multiple cards are connected, and the combination rules at that time are stacked. Call.

FIG. 2 shows an example of a window displaying the result of laying out the media objects 22, 23, 24 on the card 21. FIG. 1 schematically shows a functional configuration of an operation history recording / reproducing apparatus according to an embodiment of the present invention. The display device 1 and the input device 2 are connected to the card layout unit 3, the display device 1 displays a window as shown in FIG. 2, and the input device 2 such as a keyboard or a mouse is used for displaying media objects on the card. Designate the placement position and select media objects.

The card layout section 3 determines the arrangement positions of cards and media objects in the window displayed on the display device 1. The link unit 4 sets reference links between media objects laid out by the card layout unit 3. The VTR 5 records and reproduces an image picked up by an image pickup means such as a video camera (not shown) under the control of the VTR controller 6. The images are recorded on a recording medium (video tape or the like), and each image is managed by the VTR control unit 6 by a frame number as its identification information.

The VTR control section 6 controls the operation of the VTR 5. In addition, the VTR 5 is controlled to notify the stack execution unit 8 of the frame number of each image according to the instruction of the stack execution unit 8 and to reproduce the image from the frame number specified by the operation history reproduction unit 9. The timer 7 manages the time history of the operation when the operator operates the stack.

The stack execution unit 8 is activated by the card layout unit 3 when the operation of operating the stack by the operator is started, and the name of the card displayed on the display device 1 notified from the card layout unit 3 is displayed. Etc., the time notified from the timer 7, the contents of the operation performed on the stack, the frame number notified from the VTR control unit 6, and the like are sent to the operation history information management unit 9 as operation history information.

The operation history reproduction unit 9 reads out the operation history information managed by the operation history information management unit 9 and displays the contents thereof, or instructs the VTR control unit 6 to reproduce an image. To do. The data management unit 10 includes a card layout management unit 11, a primitive data management unit 12, and an operation history information management unit 13, and as a whole,
Collects, processes, and stores data.

The card layout management unit 11 manages the position, size, etc. of media objects laid out on each card by the card layout unit 3. Further, the card layout unit 3 manages a combination rule between a plurality of cards laid out. Primitive data management unit 1
2 is information held by a media object managed by the card layout management unit 7, such as text, images,
Manage audio information.

The operation history information management unit 13 manages the operation history information sent from the stack execution unit 8. Next, an example of a procedure for creating a stack will be described. While looking at the display device 1, the operator first creates a new card using the input device 2 such as a mouse or a keyboard, and then creates a media object to be laid out on this card,
Using the mouse, move to the appropriate position on the card and place it. When this operation is repeated to create one card, the second and subsequent cards are created in the same manner as above.

Then, the plurality of cards are connected by a link. For that purpose, first, another media object such as a button or a scroll bar or a media object for activating the card (hereinafter, a media object having this function is simply called a button) is laid out and linked. Input the original button 2
After designating the link destination card, the link unit 4 is activated to execute the link. By repeating this operation, multiple cards are connected by a link, and a stack is created.

The stack combination rule created in this way is sent from the card layout section 3 to the card layout management section 11 for management. FIG. 3 is a specific example of the created stack, which expresses a combination rule by a link structure of a card and a media object, and is composed of 5 cards, 6 media objects, and 5 links. .

FIG. 4 shows the data structure of the card 1 and the card 2 of FIG. 3 as a concrete example of the internal data structure of the card. The size of the card, a pointer to a media object laid out on the card, It holds a pointer that indicates a reference. FIG. 5 shows the media objects A and B in FIG. 3 as a specific example of the data structure of the information about the structure of the media object, and holds the position on the card, the reference link, and the like.

FIG. 6 shows a data structure of image information held by the media object A of FIG. 3 as a concrete example of the data structure of actual information such as text, voice, and images held by the media object. Then, the display size of the image, the actual display data, and the like are held. Next, a method in which the operator operates the stack as shown in FIG. 3 to execute the stack and records the operation history information will be described with reference to the flowchart shown in FIG. 7.

Executing the stack means that the button or the like laid out on the card is actually operated by the operator,
In accordance with the operation, the cards are sequentially displayed on the display device 1 based on the information managed by the card layout section 11 and the primitive data management section 12. When the operator gives an instruction to execute stacking, the process proceeds to step S1, the card layout unit 3 activates the stack executing unit 8, and the process proceeds to step S2.

In step S2, the timer 7 is initialized,
For example, the time is initialized at 0 o'clock, this time is notified to the stack execution unit 8, and the process proceeds to step S3. In step S3, the card layout unit 3 uses the information managed by the card layout management unit 11 and the primitive data management unit 12, for example, to determine the card 1 which is the top card of the stack as shown in FIG. It is displayed on the display device 1. Then, the stack execution unit 8 receives the card information of this card from the card layout unit 3, and proceeds to step S4.

In step S4, the stack execution unit 8
When the VTR control unit 6 is instructed to start recording an image with the VTR 5, the VTR control unit 6 notifies the stack execution unit 8 of the current frame number, the VTR 5 starts recording the image, and the process proceeds to step S5. . Step S5
Then, the stack execution unit 8 sets the time received in step S2, the card information received in step S3, and the frame number received in step S4 to the operation history management unit 13
And is recorded as operation history information, and the process proceeds to step S6.

In step S6, it is checked whether or not a button arranged on the card or a button for activating the next screen of the currently displayed card (hereinafter referred to as a next screen button) is pressed. If none of the buttons has been pressed, the process proceeds to step S7 and stands by as it is.
On the other hand, if any button is pressed, step S
Go to 8. That is, the stack execution unit 8 waits in that state until the operator performs an operation such as pressing a button arranged on the card displayed on the display device 1.

If any of the buttons arranged on the card is pressed in step S6, the process proceeds to step S8,
The stack execution unit 8 receives the current time from the timer 7 and the current frame number from the VTR control unit 6, and proceeds to step S9. In step S9, the stack execution unit 8 uses the operation history as information (hereinafter, referred to as button information) for identifying the button such as the time, the frame number, the name or number of the pressed button received in step S8. The information is sent to the information management unit 13, which is recorded as one operation history information, and the process proceeds to step S10.

In step S10, the card layout section 3 displays the card linked to the pressed button on the display device 1 based on the information managed by the card layout management section 11 and the primitive data management section 12. indicate. Then, the stack execution unit 8 receives the card information of this card from the card layout unit 3, and proceeds to step S11.

In step S11, the current time is received from the timer 7 and the current frame number is received from the VTR controller 6, and the process proceeds to step S12. In step S12, the stack execution unit 8 sends the card information received in step S10, the time received in step S11, and the frame number operation history management unit 13 to record them as one operation history information, and in step S13. move on.

In step S13, the stack execution unit 8
However, it is checked whether the stack execution is finished.
That is, for example, when no button is arranged on the card displayed on the display device 1 in step S10, or when the operator issues an instruction to end the stack execution, the stack execution unit 8 causes the stack execution unit 8 to perform the stack operation. It is determined that the execution has ended, and in other cases, it is determined that the stack execution continues. If it is determined to end the stack execution, the process proceeds to step S14.

In step S14, the stack execution unit 8
Instructs the VTR control unit 6 to end the recording of the image by the VTR 5, and the operation history information management unit 13
In response, the operation history information management unit 13 saves the plurality of operation history information recorded so far as one file. On the other hand, if it is determined in step S13 that the stack execution is to be continued, the process returns to step S6 and the same processing as described above is performed.

FIG. 8 shows that when the execution of the stack is completed,
6 shows an example of a data structure of operation history information managed by the operation history information management unit 13 as one file. Here, for example, when the stack as shown in FIG. 3 is executed and the processes of steps S1 to S5 are performed as described above, the operation history information 30 is recorded in the operation history information management unit 13. (See Figure 8). Then, when the stack execution unit 8 is waiting in step S7, the subsequent processing will be described below with reference to FIG.

When the button A is pressed while the stack execution unit 8 is on standby, the process proceeds from step S6 to step S8, and the stack execution unit 8 detects the current time from the timer 7 and the VTR control unit 6 from the current time. Receives the current frame number and proceeds to step S9. In step S9,
The stack execution unit 8 sends the time, frame number, and button information of button A to the operation history information management unit 13, and these are recorded as one operation history information 31 (see FIG. 8),
Go to step S10. In step S10, button A
The card 5 linked to is displayed on the display device 1,
The stack execution unit 8 receives the card information of this card 5 from the card layout unit 3, and proceeds to step S11.
In step S11, the current time from the timer 7,
The current frame number is received from the VTR controller 6 and the process proceeds to step S12. In step S12, the stack execution unit 8 sends the received card information, time, and frame number to the operation history management unit 13, which are recorded as one operation history information 32 (see FIG. 8), and the process proceeds to step S13. In step S13, since the buttons to be operated further are not arranged on the card 5, it is determined that the stack execution is completed, the process proceeds to step S14, the termination process is performed, and the process is terminated.

When the button B is pressed while the stack execution unit 8 is on standby, the process proceeds from step S6 to step S8. The stack execution unit 8 causes the timer 7 to detect the current time and the VTR control unit. The current frame number is received from 6, and the process proceeds to step S9. Step S9
Then, the stack execution unit 8 sends the time, the frame number, and the button information of the button B to the operation history information management unit 13, and these are recorded as one operation history information.
Go to 10. In step S10, the card 2 linked to the button B is displayed on the display device 1, the stack execution unit 8 receives the card information of this card 2 from the card layout unit 3, and the process proceeds to step S11. Step S
In 11, the current time is received from the timer 7 and the current frame number is received from the VTR control unit 6, and the process proceeds to step S12. In step S12, the stack execution unit 8 sends the received card information, time, and frame number to the operation history management unit 13, which are recorded as one operation history information, and the process proceeds to step S13. Step S1
In No. 3, since the buttons to be operated further are not arranged on the card 2, it is determined that the stack execution is completed, the process proceeds to step S14, the end process is performed, and the process ends.

When the next screen button (not shown in FIG. 3) of the card 1 is pressed while the stack execution unit 8 is on standby, steps S6 to S8 are performed.
Then, the stack execution unit 8 receives the current time from the timer 7 and the current frame number from the VTR control unit 6, and proceeds to step S9. In step S9, the stack execution unit 8 sets the time, frame number, and button information of the next screen button of the card 1 to the operation history information management 1
3 is recorded as one operation history information, and the process proceeds to step S10. In step S10, the card 3 linked to the next screen button of the card 1 is displayed on the display device 1, the stack execution unit 8 receives the card information of the card 3 from the card layout unit 3, and the process proceeds to step S11. In step S11, the current time is received from the timer 7 and the current frame number is received from the VTR controller 6, and the process proceeds to step S12. In step S12, the stack execution unit 8 sends the received card information, time, and frame number to the operation history management unit 13, which are recorded as one operation history information, and the process proceeds to step S13. In step S13, since the button (the next screen button of card 3) is arranged on the card 3, it is determined that the stack execution is continued, and step S6
Go to step S7, until this button is pressed,
The stack execution unit 8 waits, and the same applies to the following.

Through the above procedure, the history of a series of operations by the operator is stored as operation history information in the operation history information management unit 1.
It is recorded in 3. Next, a procedure for reproducing the operation history using this operation history information will be described with reference to FIG.

First, the first reproduction procedure will be described.
In this case, when the operator gives an instruction to execute the operation history reproduction processing, the operation history reproduction unit 9 reads the operation history information recorded in the operation history information management unit 13, and the display device 1
Display the contents of this information on. For example, when the operation history information as shown in FIG. 8 is displayed, “Button A was pressed while card 1 was displayed and card 5 was displayed” or “5 before pressing button A” "It took a second" is displayed. Further, as shown in FIG. 9A, it is possible to display the card 1, time, and frame number on the display device 1. That is, the operation history reproducing unit 9
The card information, time, and frame number in the operation history information read by the card layout section 3 are sent to the card layout section 3, and based on the information managed by the card layout management section 11 and the primitive data management section 12. The card of the received card information is displayed on the display device 1, and the time and frame number are also displayed.

Then, for example, when it is desired to determine why it took 5 seconds to press the button A, when the operator instructs reproduction of an image from a desired time point by using the input device 2 or the like, the operation is performed. The history reproducing unit 9 sends the frame number at the instructed time to the VTR control unit 6, and the VTR control unit 6
Then, the image of that frame number is searched and that image is displayed on the monitor 7 (see FIG. 8B). The operator can see the operation situation and the external situation by looking at this image. For example, it took 5 seconds for the operator shown in the image to press the button A because the operation was unknown. I mean,
Alternatively, it can be determined whether it is because something other than an operation such as searching for an object is being performed.

Next, the second reproduction procedure of the operation history using the operation history information will be described. In this case, when the operator gives an instruction to execute the operation history reproduction process, the operation history reproduction unit 9 instructs the VTR control unit 6 to reproduce the image from the recording start time. Upon receiving this instruction, the VTR control unit 6 causes the VTR 5 to reproduce the image from the recording start time, and displays the image on the monitor 7.

At this time, for example, the operator operates as shown in FIG.
Looking at the image displayed on the monitor 7 as shown in
When the operator wants to know the card displayed on the display device 1 at this time, or when he / she wants to know which button the operator shown in this image has pressed, the operator gives an instruction to that effect. Is done using the input device 2 etc.,
The operation history reproducing unit 9 is operated by the VTR control unit 6 as shown in FIG.
The frame number of the image shown in is received. The operation history reproducing unit 9 searches and reads out operation history information including this frame number from the operation history information managed by the operation history information management unit 13. Then, the card information in the read operation history information, the time, and the frame number are sent to the card layout section 3, and the card layout section 3 also includes the information managed by the card layout management section 11 and the primitive data management section 12. Then, the card of the received card information is displayed on the display device 1, and the time and frame number are also displayed. At this time, when the card is displayed on the display device 1, for example, it becomes as shown in FIG. 9B.

As described above, according to the above embodiment, when executing the created stack, the operator performs an operation such as pressing a button arranged on the card or the card is displayed. By reading the frame number at that time from the VTR control unit 6 and recording it as the operation history information in the operation history information management unit 13, the operation history information and the image of the external situation of the operator can be reproduced in association with each other.
You can easily check the problems of the created stack and improve it.

[0043]

As described above, according to the present invention, the operation history information and the image of the external situation of the operator can be played back in association with each other, and the problem of the operation screen can be confirmed and its improvement can be easily performed. An operation history recording / reproducing device can be provided.

[Brief description of drawings]

FIG. 1 is a configuration diagram schematically showing a configuration of an operation history recording / reproducing apparatus according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of an operation screen displaying a result of laying out media objects on a card.

FIG. 3 is a diagram showing an example of a link structure between a card and a media object.

FIG. 4 is a diagram showing an example of an internal data structure of a card.

FIG. 5 is a diagram showing an example of a data structure of information about the structure of a media object.

FIG. 6 is a diagram showing an example of a data structure of media information.

FIG. 7 is a flowchart illustrating a method of recording operation history information.

FIG. 8 is a diagram showing an example of a data structure of operation history information.

FIG. 9 is a diagram for explaining a reproduction procedure of an operation history.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Display device, 2 ... Input device, 3 ... Card layout part, 4 ... Link part, 5 ... VTR, 6 ... VTR control part, 7
... monitor, 8 ... timer, 9 ... operation history reproducing section, 10 ... data managing section, 11 ... card layout managing section, 12 ... primitive data managing section, 13 ... operation history information managing section

Claims (4)

[Claims] 1. A stack created by setting a connection relationship between a plurality of cards is operated according to an operation performed by an operator, a plurality of operation history information at that time is recorded, and the operation history information is reproduced. An operation history recording / reproducing apparatus, comprising: rule management means for managing a connection relationship between a plurality of cards in the stack; and image recording for recording an image of an external situation when the operator operates the stack. Means, identification information management means for managing identification information for identifying a plurality of images recorded by the image recording means, and when an operation for operating the stack is performed, management is performed by the rule management means according to the operation. The card display means for displaying the card based on the existing connection relationship, and when the card is displayed on the card display means, the image is recorded by the image recording means at that time. First reading means for reading the identification information for identifying the recorded image from the identification information managing means, and an image recorded by the image recording means at the time when an operation for operating the stack is performed. Second reading means for reading the identification information for identifying the identification information from the identification information managing means, and an operation history recording operation history information including the identification information read by the first and second reading means. Information recording means, operation history information display means for displaying the contents of the operation history information recorded by the operation history information recording means, and retrieval of the image recorded by the image recording means based on the identification information. An operation history recording / reproducing apparatus comprising: an image retrieving unit; and an image replaying unit that replays an image retrieved by the image retrieving unit. 2. A stack created by setting a coupling relationship between a plurality of cards is operated according to an operation performed by an operator, a plurality of operation history information at that time is recorded, and the operation history information is reproduced. An operation history recording / reproducing apparatus, comprising: rule management means for managing a connection relationship between a plurality of cards in the stack; and image recording for recording an image of an external situation when the operator operates the stack. Means, identification information management means for managing identification information for identifying a plurality of images recorded by the image recording means, and when an operation for operating the stack is performed, management is performed by the rule management means according to the operation. The card display means for displaying the card based on the existing connection relationship, and when the card is displayed on the card display means, the image is recorded by the image recording means at that time. First reading means for reading the identification information for identifying the recorded image from the identification information managing means, and an image recorded by the image recording means at the time when an operation for operating the stack is performed. Second reading means for reading the identification information for identifying the identification information from the identification information managing means, and an operation history recording operation history information including the identification information read by the first or second reading means. Information recording means, image reproducing means for reproducing the image recorded by the image recording means, and information retrieval for retrieving operation history information managed by the operation history information managing means based on the identification information. An operation history recording / reproducing apparatus comprising: a means and an operation history information display means for displaying the contents of the operation history information retrieved by the information retrieval means. 3. A stack created by setting a coupling relationship between a plurality of cards is operated according to an operation performed by an operator, a plurality of operation history information at that time is recorded, and these operation history information are reproduced. In the operation history recording / reproducing apparatus, a layout means for laying out media objects on each of a plurality of cards, which is a basic unit of display, and a layout relationship for setting media objects laid out on the card by the layout means Link means, layout information management means for managing layout information of media objects laid out on the card by the layout means, information management means for managing information of each media object laid out by the layout means, Plural laid out by layout means A stack creating means for creating a stack by setting a bond relationship between cards, a rule managing means for registering and managing a bond relationship between a plurality of cards of a stack created by the stack creating means, and the operator An image recording unit that records an image of an external situation when performing an operation to operate the stack, an identification information management unit that manages identification information for identifying the image recorded by the image recording unit, and the stack, When an operation to operate is performed, a card display means for displaying the card based on the connection relationship managed by the rule management means according to the operation, and a card displayed on the card display means, A first reading unit for reading the identification information for identifying the image recorded by the image recording unit from the identification information management unit; When the operation for activating is performed, second identification means for reading out the identification information for identifying the image recorded by the image recording means at that time from the identification information management means; Operation history information recording means for recording operation history information including identification information read by the second reading means, and operation history information display means for displaying the contents of the operation history information recorded by the operation history information recording means. An image retrieving means for retrieving the image recorded by the image recording means based on the identification information, and an image replaying means for replaying the image retrieved by the image retrieving means. Operation history recording / reproducing device. 4. A stack created by setting a coupling relationship between a plurality of cards is operated according to an operation performed by an operator, a plurality of operation history information at that time is recorded, and these operation history information are reproduced. In the operation history recording / reproducing apparatus, a layout means for laying out media objects on each of a plurality of cards, which is a basic unit of display, and a layout relationship for setting media objects laid out on the card by the layout means Link means, layout information management means for managing layout information of media objects laid out on the card by the layout means, information management means for managing information of each media object laid out by the layout means, Plural laid out by layout means A stack creating means for creating a stack by setting a bond relationship between cards, a rule managing means for registering and managing a bond relationship between a plurality of cards of a stack created by the stack creating means, and the operator An image recording unit that records an image of an external situation when performing an operation to operate the stack, an identification information management unit that manages identification information for identifying the image recorded by the image recording unit, and the stack, When an operation to operate is performed, a card display means for displaying the card based on the connection relationship managed by the rule management means according to the operation, and a card displayed on the card display means, A first reading unit for reading the identification information for identifying the image recorded by the image recording unit from the identification information management unit; When the operation for operating the is performed, second identification means for reading out the identification information for identifying the image recorded by the image recording means at that time from the identification information management means; Operation history information recording means for recording operation history information including identification information read by the second reading means, image reproducing means for reproducing the image recorded by the image recording means, and the identification information based on the identification information. And an operation history information display means for displaying the contents of the operation history information searched by this information search means, and an information search means for searching the operation history information managed by the operation history information management means. An operation history recording / reproducing apparatus characterized by the above.