JPH1021167A - Multimedia electronic mail system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To transmit the input operation of a mail generating person by storing input operation information in generating a mail in an electronic mail system consisting of a transmission terminal for inputting/editing plural kinds of media information and transmitting them as an electronic mail and a reception termi nal for receiving the electronic mail with a network and presenting reception information. SOLUTION: An input operation control means 111 stores input operation information inputted from input means 101-106 to an input operation storage means 112 in order of an input device operation. At this time, a link is made by the order of the input operation and a delay time till a succeeding input is reproduced is simultaneously set. Therefore, the input operation is reproduced time sequentially at a reception side by transmitting the information.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic mail system used on a network, and more particularly to an electronic mail system for structuring and transmitting a plurality of pieces of media information by simple input and editing operations.

[0002]

2. Description of the Related Art In a conventional multimedia electronic mail system, first, media data such as text, image, voice, etc., is input, and then, synchronization information between the input multimedia data is edited to obtain multimedia information. Create a mail. Such a multimedia electronic mail system is disclosed in JP-A-6-79992. A conventional multimedia electronic mail system will be described below with reference to FIGS. still,
The same reference numerals indicate the same objects throughout the drawings.

(Conventional Example 1) FIG. 12 is a system configuration diagram of a transmission terminal of a conventional multimedia electronic mail system.
FIG. 13 shows a system configuration diagram of the receiving terminal. In FIG. 12, a text input unit 1201 is an input unit for inputting text information.
02 is an input means for inputting voice, a still image input means 12
03 is an input means for inputting a still image. These pieces of input information are temporarily stored in the text input storage unit 1204, the voice input storage unit 1205, and the still image input storage unit 1206, and are stored in a state where they can be transmitted to the synchronization information storage unit 1208. The synchronization information changing unit 1207 is a unit for changing the synchronization information, and the synchronization information presenting unit 1209 is a unit used for confirming the synchronization information before transmission. The transmission means 1210 is for transmitting the information stored in the synchronization information storage means 1208 to the transmission line.

[0004] In FIG. 13, the receiving terminal is a receiving means 130.
1, synchronization information storage means 1302 and synchronization information presentation means 1
303. The synchronization information storage unit 1302 and the synchronization information presentation unit 1303 are the same as those shown in FIG. Although the transmitting terminal and the receiving terminal may be independent, they are usually configured as a single unit.

FIG. 14 is a block diagram showing the configuration of a terminal system having a transmitting / receiving function. The CPU 205 controls various operations of the transmitting terminal in accordance with a system program stored in the external storage device 207 or the like.
2, a keyboard 204, a main storage device 206, an external storage device 207, a pointing device 209, a speaker 203, a voice input device 208, and an image input device 210 are connected, and the CPU 205 controls these input / output operations. The video RAM 202 stores image data to be displayed on the display 201. Keyboard 2
When a key on the keyboard 204 is operated, a code 04 corresponding to the operation key is given to the CPU 205. Network interface 21
Reference numeral 1 denotes an interface with a network (not shown).

[0006] Text input storage means 120 shown in FIG.
4. The voice input storage unit 1205, the still image input storage unit 1206, the synchronization information change unit 1207, and the synchronization information storage unit 1208 are composed of a CPU 205 and a main storage device 206.

The operation of the conventional example configured as described above is as follows.
This will be described using an example of a multimedia mail shown in FIG. FIG. 3A is a time series at the time of object reproduction, and FIG.
FIG. 4 is a diagram showing a state of being reproduced on a display screen. “# Number” in FIG. 3 indicates an object identifier. In addition, objects input from the respective input means are collectively referred to as media objects.

First, an operation procedure for creating the multimedia mail shown in FIG. 3 will be described below. [Input Procedure] (Procedure 1) The input person uses the text input unit 1201 to input the text object # 301. (Procedure 2) The input person uses the text input unit 1201 to input the text object # 302. (Procedure 3) The input user inputs the still image object # 303 using the still image input unit 1203. (Procedure 4) The input user uses the still image input unit 1203 to input the still image object # 304. (Procedure 5) The input person uses the voice input means 1202,
Voice object # 305 is input. (Procedure 6) The input user inputs the still image object # 306 using the still image input unit 1203. (Procedure 7) The input person uses the voice input means 1202,
Voice object # 307 is input. (Procedure 8) The input user uses the still image input unit 1203 to input the still image object # 308. (Procedure 9) The input person uses the voice input means 1202,
Voice object # 309 is input. (Step 10) The input person uses the voice input unit 1202 to input the voice object # 310. The media information thus input is stored in the respective input storage means 1204 to 1206 at the time of input, but there is no relation between these pieces of information.

FIG. 15 shows a text input storage means 120.
4, an example of media information stored in the voice input storage means 1205 and the still image input storage means 1206.
Next, in order to set the positional / spatial synchronization between the media information by the synchronization information changing means 1207, the synchronization between the media information is set on the screen using a user interface as shown in FIG. The screen shown in FIG. 16 is a display screen when the presentation position, presentation start time, and presentation end time of each piece of media information are input as follows. [Synchronization Setting Procedure] (Procedure 1) The input person sets the object # 301 to t0 on the time axis. (Procedure 2) The input person sets the object # 302 to t0 on the time axis. (Step 3) The input person sets the object # 303 to t0 on the time axis. (Step 4) The input person sets the object # 304 to t1 on the time axis. (Step 5) The input person sets the object # 305 to t2 on the time axis. (Step 6) The input person sets the object # 306 to t3 on the time axis. (Step 7) The input person sets the object # 307 to t4 on the time axis. (Step 8) The input person sets the object # 308 to t5 on the time axis. (Procedure 9) The input person sets the object # 309 to t6 on the time axis. (Step 10) The input person sets the object # 310 to t7 on the time axis.

FIG. 17 shows an example of the synchronization information stored in the synchronization information storage means 1208. The data of each object input in this manner is stored in the synchronization information storage unit 1 in a state where coordinates, specific start time, end time, and presentation time are input.
208. Then, the data is transmitted to the receiving terminal through a network (not shown) via the transmitting unit 1210. As described above, the mail is created by the input procedure several times and the synchronization setting procedure several times.

On the other hand, in the receiving terminal, the synchronous information presenting means 1303 reads out the synchronous information from the synchronous information storing means 1302, and extracts the corresponding information of each object, and the information of the presentation start time and the presentation end time. Next, an event list as shown in FIG. 18 is created from two pieces of information, the presentation start time and the presentation end time. This event list is shown in Figure 1.
7 is a list in which the presentation start time and the presentation end time for each piece of media information are rearranged in ascending order of time.

Next, the process up to presentation using this event list will be described with reference to the flowchart of FIG. First, in step S1, the head media information of the event list is read to know the target media information and the presentation start / end times. Next, in step S2, it is determined whether the presentation starts or ends after waiting for the corresponding time. If it is the presentation start, the process proceeds to step S3 and waits until the presentation start time. Then, a start message is sent to the corresponding object (step S4), and position information and media information in the object are obtained (step S5).
Next, the presentation start is instructed to the device driver corresponding to the media, and the process returns to step S1.

On the other hand, if the presentation is completed in step S2, the process waits until the presentation end time in step S7. Then, a stop message is sent to the corresponding object (step S8), and the end of presentation is instructed to the device driver corresponding to the media (step S9), and the process returns to step S1. In this case, the synchronous reception storage means 120
As shown in FIG. 8, objects can be sequentially reproduced by a reproduction sequence as shown in FIG.

Next, an insertion / deletion operation procedure will be described. [Insert] In FIG. 3, still image object # 304
Between voice object # 305 and voice object #
The operation procedure when inserting 304 'is shown below. (Procedure 1) The input person uses the voice input means 1202 to
The voice object # 304 'is input. (Procedure 2) The input person sets the object # 304 'to t2 on the time axis. (Step 3) The input person resets the reproduction start time and end time of the object # 305 on the time axis. (Step 4) The input user resets the reproduction start time and end time of the object # 306 on the time axis. (Step 5) The input person resets the reproduction start time and the end time of the object # 307 on the time axis. (Step 6) The input user resets the reproduction start time and end time of the object # 308 on the time axis. (Step 7) The input person resets the reproduction start time and the end time of the object # 309 on the time axis. (Step 8) The input person resets the reproduction start time and end time of the object # 310 on the time axis. As described above, a desired object is inserted by operating the input procedure several times of the insertion object and the synchronization setting procedure several times of the object reproduced after the insertion object several times.

[Delete] Next, referring to FIG. 3, an operation procedure for deleting the voice object # 305 by the synchronization information changing means will be described below. (Procedure 1) The input person deletes the object # 305. (Procedure 2) The input person resets the reproduction start time and end time of the object # 306 on the time axis. (Procedure 3) The input user resets the reproduction start time and the end time of the object # 307 on the time axis. (Step 4) The input user resets the reproduction start time and the end time of the object # 308 on the time axis. (Step 5) The input person resets the reproduction start time and the end time of the object # 309 on the time axis. (Step 6) The input person resets the reproduction start time and the end time of the object # 310 on the time axis. As described above, unnecessary objects are deleted by operating the deletion procedure several times and the synchronization setting procedure reproduced several times after the deletion object. Through the above processing, a plurality of media information is presented in a form that is spatially and temporally synchronized.

(Conventional Example 2) FIG. 20 shows an example of a reply operation in a conventional electronic mail system. 2. Description of the Related Art Conventionally, when a document is transmitted / received using an electronic mail system, a part of the other party's document may be quoted and returned to the transmitted other party's document. In this case, as shown in FIG. 20, an indicator >>> is inserted at the beginning of the line in the text of the partner document to be quoted, and the responder indents the partner document by indenting the partner document. Expressing the quote.

[0017]

Generally, in the process of creating an electronic mail, a pre-sentence, a main sentence, and a last sentence are described in this order. or,
In the case of the explanation of the operation or the operation, it is described in order from the beginning. That is, the correlation between the input operation sequence of the mail creator and the actual mail reproduction sequence is high.

However, in the conventional multimedia electronic mail system, input operation information is not reflected in the association between objects when a media object is input. Therefore, when an electronic mail document including N media objects is created, N times of e-mail documents are created. Media object input operation and N synchronization information setting operations (2N operations in total)
There was a problem that required. In addition, there is a problem that a pointing device operation cannot be expressed when describing an operation or an operation.

In the conventional user interface, the presentation position of the object is specified at an arbitrary coordinate in the document creation window. Therefore, for a document as shown in FIG. In the expression, there is a problem that a quote of a window of a text object arranged in a horizontal direction cannot be expressed.

The inventions of claims 1 to 8 of the present application have been made in order to improve such a conventional problem, and a correlation between an input operation sequence of a mail creator such as an electronic mail and an actual mail reproduction sequence. Focusing on the fact that the number of media objects is high, the input operation information is stored at the time of the input operation, so that the temporal and spatial associations between the media are simultaneously performed. An object of the present invention is to create an electronic mail by a media object input operation and n (where n is sufficiently smaller than N) synchronization information changing operations.

In addition, it is an object of the present invention to reproduce a pointing device operation of an input person on a receiver side by preparing a pointing device operation object as a new media object.

According to the fifth and sixth aspects of the present invention, in addition to this, the media object created by the responder has a visual effect, so that the object can be quoted in a time and three-dimensional manner, and an electronic mail system. The purpose of this is to cite objects that exist in the creation window.

[0023]

According to a first aspect of the present invention, there is provided a transmitting terminal for inputting / editing a plurality of media information and transmitting the same as electronic mail, and receiving and receiving electronic mail from the transmitting terminal via a network. In a multimedia electronic mail system having a receiving terminal for presenting information, the transmitting terminal includes a text input unit for inputting text, a graphic input unit for inputting a graphic, a voice input unit for inputting voice, and an image. Image input means for inputting
The text input unit, a drawing operation input holding unit that temporarily holds drawing operation information obtained from the graphic input unit, a voice input holding unit that temporarily holds voice input information obtained from the voice input unit, Image input holding means for temporarily holding image input information obtained from the image input means, the text input means, the graphic input means,
An input operation control unit for performing an ordering operation in accordance with an input operation sequence from information input to the image input unit, the drawing operation input holding unit, the voice input holding unit, and information held in the image input holding unit; Input operation storage means for storing input operation information ordered by the operation control means; input operation change means for changing input operation information stored in the input operation storage means; Input means for presenting input operation information; and transmitting means for transmitting the information of the input operation storage means as electronic mail, wherein the receiving terminal receives the input means and receives the received electronic mail. Input operation reproducing means for reproducing as the operation information, and input operation presenting means for presenting the input operation information reproduced by the input operation reproducing means. It is characterized in.

In the invention of claim 2 of the present application, the transmitting terminal has pointing device operation input means for inputting a pointing device operation, and the input operation control means includes an input from the pointing device operation input means. The input operation is performed in accordance with the input operation order including information, and the input operation reproducing means of the receiving terminal reproduces the movement of the pointing device at the time of input from the received electronic mail.

According to the invention of claim 3 of the present application, the image input means is an input means for inputting at least one of a still image and a moving image.

According to a fourth aspect of the present invention, there is provided a communication terminal having a transmission function of inputting / editing a plurality of media information and transmitting the same as electronic mail, and a receiving function of receiving electronic mail via a network and presenting received information. A multimedia electronic mail system including: a text input unit for inputting text, a graphic input unit for inputting a graphic, a voice input unit for inputting voice, and an image input unit for inputting an image. A drawing operation input holding unit that temporarily holds drawing operation information obtained from the text input unit and the graphic input unit; and a voice input holding unit that temporarily holds voice input information obtained from the voice input unit. Image input holding means for temporarily holding image input information obtained from the image input means; text input means; Means, the image input unit information input to the drawing operation input holding means, said audio input holding means, from information held in the image input holding means,
Input operation control means for performing ordering in accordance with the input operation order; input operation storage means for storing input operation information ordered by the input operation control means; and input operation information stored in the input operation storage means. Input operation changing means, receiving means, input operation reproducing means for reproducing a received electronic mail as an input operation, input operation information changed by the input operation changing means, and input operation reproduced by the input operation reproducing means An input operation presenting means for presenting information, and a transmitting means for transmitting the information of the input operation storing means as an electronic mail are provided.

[0027] In the invention of claim 5 of the present application, the input operation control means adds reply history information together with the ordering according to the input operation order, and the input operation reproduction means adds the reply history information to the reply history information. It is characterized in that the media information is displayed three-dimensionally on the basis of the media information.

[0028] In the invention of claim 6 of the present application, the input operation control means adds reply history information together with the ordering according to the input operation order, and the input operation reproduction means adds the reply history information to the reply history information. It is characterized in that the media information is displayed as a gradation based on the media information.

[0029] In the invention of claim 7 of the present application, the communication terminal includes pointing device operation input means for inputting a pointing device operation, and the input operation control means includes an input from the pointing device operation input means. The input operation is performed in accordance with the input operation order including information, and the input operation reproducing means reproduces the movement of the pointing device at the time of input from the received mail.

According to the invention of claim 8 of the present application, the image input means is an input means for inputting at least one of a still image and a moving image.

According to the first aspect of the present invention, the transmitting terminal has input means for inputting a multimedia object, and input holding means for temporarily holding input information, and performs input operation in accordance with the input order. The ordering is performed by the control means. Then, the information is changed by the input operation changing means and transmitted by the transmitting means. When the receiving terminal receives this, the input operation is reproduced and presented in the order of input based on the input operation information. In this way, all the objects can be easily synchronized, and the objects can be displayed sequentially in chronological order. In the invention of claim 4, the communication terminal has the transmission function and the reception function. In claims 5 and 6, the input operation control means adds reply history information to the input operation information at the time of reply. In addition, the input operation reproducing means displays the area information three-dimensionally or in a gradation based on the reply history information, so that the quotation can be made clearly. According to the second and seventh aspects of the present invention, the operation of the pointing device is fetched as input information, and the movement of the pointing device is reproduced by the input operation reproducing means on the receiving terminal side.

[0032]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to FIGS. Note that the same reference numerals indicate the same objects throughout the drawings. (Embodiment 1) First, a transmitting terminal of the multimedia electronic mail system of the present invention will be described. FIG. 1 shows a system configuration of a transmission terminal of the multimedia electronic mail system of the present invention. In FIG. 1, a text input unit 101 is an input unit for inputting a text. The graphic input means 102 is an input means for inputting a graphic using a graphic editor or the like. The pointing device operation input means 103 is an input means for directly inputting the operation of the pointing device. Voice input means 10
Reference numeral 4 denotes a device for inputting audio, and a still image input unit 1
05, a moving image input means 106 is an input means for taking in information obtained from a device such as a still electronic camera, a scanner, or a video camera. Input data from these input means is temporarily held in the drawing operation input holding means 107, the voice input holding means 108, the still image input holding means 109 and the moving image input holding means 110, or is directly given to the input operation control means 111. Can be The input operation control unit 111 orders the voice, text, and image data stored in these storage units according to the input order, and stores the data in the input operation storage unit 112 in a transmittable state. The contents held in the input operation storage means 112 can be changed by the input operation change means 113, and the input operation presentation means 11
4. The transmitting means 115 transmits information held in the input operation storing means 112 to the receiving terminal via a network (not shown).

FIG. 2 shows a hardware configuration of the transmitting terminal. The CPU 205 controls various operations of the transmission terminal according to a system program stored in the external storage device 207 or the like.
A display 201, a video RAM 202, a keyboard 204, a main storage device 206A, an external storage device 207, a pointing device 209, a speaker 203, a voice input device 208, an image input device 210, and a network interface 211 are connected.
5 controls these input / output operations. Video RAM 20
Reference numeral 2 stores image data to be displayed on the display 201. When a key on the keyboard 204 is operated, the keyboard 204 gives a key code corresponding to the operation key to the CPU 205. Drawing operation input holding means 107,
Sound / voice input holding unit 108, still image input holding unit 10
9, moving image input holding means 110, input operation control means 11
1. Input operation storage means 112, input operation change means 113
Is composed of a CPU 205 and a main storage device 206A.

The operation of the present embodiment configured as described above will be described with reference to an example of a multimedia electronic mail shown in FIG. However, “# number” in FIG. 3 represents an object identifier.

First, an operation procedure for creating the multimedia electronic mail of FIG. 3 will be described below. [Input Procedure] (Procedure 1) The input person uses the text input means 101 to input the text object # 301. The input operation control unit 111 links the text object # 301 to the root object indicating only the beginning of the data, and sets a delay value to the text object # 301. Here, the delay value refers to the time from when the object is reproduced to when the reproduction of the next object is started. Regardless of how long it takes to enter an object or the time to enter the next object,
Initial value is set. It is conceivable that the initial value of the delay value is obtained by adding a fixed offset value to a predetermined readable time corresponding to the amount of text data.

(Procedure 2) The input person is the text input means 1
01, a text object # 302 is input. The input operation control unit 111 links the text object # 302 to the text object # 301,
A delay value is set in the same manner for text object # 302.

(Procedure 3) The input person is the still image input means 1
05, a still image object # 303 is input. The input operation control unit 111 links the still image object # 303 to the text object # 302,
A delay value is set for the still image object # 303. This delay value is also set to an initial value capable of reading a still image.

(Procedure 4) The input person is the still image input means 1.
05, a still image object # 304 is input. The input operation control means 111 links the still image object # 304 to the still image object # 303,
A delay value is set for the still image object # 304.

(Procedure 5) The input person is the voice input means 104
Is used to input the voice object # 305. The input operation control unit 111 selects the still image object # 304
Is linked to the audio object # 305, and a delay value is set to the audio object # 305. When (procedure 5) occurs during the operation of (procedure 4), the voice object # 305 input in (procedure 5) is stored in the voice input holding unit 108.
Is held. In this case, first, the delay value of (procedure 4) is set. Next, after the operation of (Procedure 4) ends, the processing of (Procedure 5) is executed. Detailed processing of the input operation control means 111 will be described later with reference to FIG.

(Procedure 6) The input person is the still image input means 1
02, a still image object # 306 is input. The input operation control unit 111 outputs the voice object # 3
The still image object # 306 is linked to 05, and a delay value is set to the still image object # 306.

(Procedure 7) The input person is the voice input means 104
, A voice object # 307 is input. The input operation control unit 111 outputs the still image object # 306
Is linked to the audio object # 307, and a delay value is set to the audio object # 307. When (procedure 7) occurs during the operation of (procedure 6), the voice object # 307 input in (procedure 7) is stored in the voice input holding unit 108.
Is held. In this case, first, the delay value of (procedure 6) is set. Next, after the operation of (procedure 6) is completed, the process of (procedure 7) is executed. Detailed processing of the input operation control means 111 will be described later with reference to FIG.

(Procedure 8) The input person is the still image input means 1
02, a still image object # 308 is input. The input operation control unit 111 outputs the voice object # 3
07, the still image object # 308 is linked, and a delay value is set to the still image object # 308.

(Procedure 9) The input person is the voice input means 104
, A voice object # 309 is input. The input operation control unit 111 selects the still image object # 308
Is linked to the audio object # 309, and a delay value is set to the audio object # 309. When (procedure 9) occurs during the operation of (procedure 8), the voice object # 309 input in (procedure 9) is stored in the voice input holding unit 108.
Is held. In this case, first, the delay value of (procedure 8) is set. Next, after the operation of (procedure 8) is completed, the process of (procedure 9) is executed. Detailed processing of the input operation control means 111 will be described later with reference to FIG.

(Procedure 10) The input person uses the voice input means 10
4, the voice object # 310 is input. The input operation control unit 111 links the voice object # 310 to the voice object # 309, and sets a delay value for the voice object # 310. This object #
Since 310 is the final object, the delay value is meaningless. Thus, the input operation is completed.

FIG. 4 shows an example of the input operation information stored in the input operation storage means 112. FIG. 5 is a flowchart illustrating the above-described processing of the input operation control unit 111. First, in step S1, a pointer is set to the start address of the input operation storage area of the input operation storage unit 112. Next, in step S2, the start / end of the input operation from the input device is checked. If started, the process proceeds to step S3, and the input operation control unit 111 checks whether input information of another object is being stored. If the object is being stored, a delay value is set for the currently stored object (step S4), and the object is temporarily held in any of the input holding units (step S5). If it is not being stored, the storage of the input object in the input operation storage unit 112 is started (step S6).

On the other hand, when the input is stopped in step S2, it is checked whether or not the object is held in the input holding means (step S7). If the object is stored, the storage of the input object is terminated and the pointer is advanced (steps S9 and S10). If the object is not stored, a delay value is set for the object currently stored, and step 2 is performed.
Return to

Next, a description will be given of how to set the synchronization using the input operation changing means 113 in order to match the reproduction sequence shown in FIG. 3 with the input information shown in FIG. [Synchronization Setting Procedure] (Procedure 1) The input person resets the delay value of text object # 301 to 0. (Step 2) The input person resets the delay value of the text object # 302 to 0. (Step 3) The input person resets the delay value of the still image object # 303 to 0. Text object # 30
1, # 302 and still image objects # 303, # 30
Assuming that 4 is simultaneously presented to the receiver, this can be realized by setting the delay value to 0. As described above, the mail can be created by the input procedure of the object several times and the synchronization setting procedure of the object of the synchronization setting several times.

Next, an insertion / deletion operation procedure will be described. First, in FIG. 3, a still image object # 304
Between voice object # 305 and voice object #
The operation procedure when inserting 304 'is shown below. [Insertion Procedure] (Procedure 1) The input person uses the text input means 101 to input the voice object # 304 '. The input operation control unit 111 links the audio object # 304 'to the still image object # 304, and sets a delay value for the audio object # 304'. As described above, an object can be inserted by an input procedure several times.

Next, referring to FIG.
The operation procedure when deleting 305 is shown below. [Delete Procedure] (Procedure 1) The input person deletes the voice object # 305. As described above, it is possible to delete an object by deleting the object several times.

Next, a receiving terminal of the multimedia electronic mail system of the present invention will be described. FIG. 6 shows a system configuration of a receiving terminal of the multimedia electronic mail system of the present invention. The hardware configuration is the same as the hardware configuration of the transmitting terminal. FIG.
In, the receiving means 601 receives a signal from the network, and the received signal is stored in the input operation storage means 602. The input operation reproducing means 603 reconstructs the input operation based on this information, and the input operation presenting means 604 presents it. The input operation storage means 602 and the input operation reproduction means 603
05 and the main storage device 206A.

The operation of the receiving terminal of the present embodiment configured as described above will be described below. Consider a case in which a multimedia electronic mail as shown in FIG. 3 is received and an input operation is reproduced. In FIG. 3, “# number” represents an object identifier. First, the input operation reproducing unit 603 reads the input operation information for one object from the input operation storage unit 602, and extracts the object type, coordinates, object attribute value, synchronization information, and the like. Then, the extracted information is transferred to the input information presenting means 604, and an output request is made to an output device corresponding to the object type.

FIG. 4 shows an example of the input operation information stored in the input operation storage means 112. FIG. 7 is a flowchart illustrating a process of reproducing an input operation performed by the input operation reproducing unit 602. First, in step S1 of FIG. 7, the pointer is set to the head address of the input operation storage area of the input operation storage unit 602. Then, in step S2, it is checked whether or not there is input information. If there is, the input information of the pointer is read in step S3. Next, the input information is interpreted, and the type, coordinates, attribute values, and synchronization information of the object are extracted (step S4). Then, in steps S5 and S6, the synchronous reproduction object is extracted, and a simultaneous execution request is output to the device. Then, after the delay time, the pointer is advanced and the process returns to step S2. If there is no more input information in step S2, the process ends (step S7).

As described above, according to the present embodiment, the drawing operation input holding unit, the voice input holding unit, the still image input holding unit, the moving image input holding unit, the input operation control unit, the input operation By providing the storage means and the input operation reproducing means, the input operation information is stored at the time of the input operation, and the temporal and spatial association between the media is simultaneously performed. By transmitting a natural input operation of a human being reflected on the multimedia mail and reproducing the input operation at the receiving terminal, it is possible to convey the creator's intention to create a mail.

(Embodiment 2) As shown in FIG. 8, consider a case where a multimedia electronic mail including a pointing device operation is created at a transmitting terminal, received at a receiving terminal, and the pointing device operation is reproduced. The hardware configurations of the transmitting terminal and the receiving terminal are the same as in the first embodiment. Consider a case in which a multimedia mail as shown in FIG. 8 is received and an input operation is reproduced. In FIG.
“# Number” represents an object identifier. Also, in FIG. 8, an object # 804 indicating a handwritten circle,
Object # 806, object # 808 indicating a handwritten mark, object # 80 indicating a handwritten arrow
9 and an object # 810 indicating a handwritten X are pointing device operation objects.

First, the operation of the pointing device is input from the pointing device operation input means 103. In this case, the input is approximated by a broken line, and the coordinates are stored in the storage means. FIG. 11 shows the input information stored in the input operation storage unit 112 by the input operation control unit 111. Here, the reply history information is used when quoting a mail received at the time of replying the object, and +1 is added every time the mail is quoted. On the other hand, the input operation reproducing means 603 reads input operation information for one object from the input operation storage means 602,
Coordinates, object attribute values, synchronization information, etc. are extracted. Then, the extracted information is transferred to the input information presenting means 604, and an output request is issued to a device output corresponding to the object type. The input information presenting means 604 moves and displays the pointing device while referring to the pointing device operation coordinate list. Through the above processing, the pointing device operation information at the time of input can be exchanged by electronic mail, and the input operation can be reproduced on the receiver side.

In the first and second embodiments, the transmitting terminal and the receiving terminal have been described as being independent from each other. However, a communication terminal having the functions of the transmitting terminal and the receiving terminal is provided. It is needless to say that a multimedia e-mail system can be configured by connecting a large number of computers to a network.

Third Embodiment Next, a third embodiment of the present invention will be described. In the present embodiment, a reply mail when receiving a reply based on the mail after receiving the electronic mail has a feature. In this embodiment, the multimedia electronic mail system is configured by connecting a plurality of communication terminals to a network. The communication terminal is the first
The transmission function and the reception function of the embodiment are included.
The input operation control means 111A adds reply history information when citing the received document together with the ordering according to the input operation order of the first embodiment. That is, a reply mail is created in the same procedure as in the first embodiment described above, but when a specific object is quoted, an electronic mail is created as shown in FIG. The procedure for creating an electronic mail is the same as in the above-described embodiment.

Next, the operation on the receiving side will be described. The input operation reproducing means 603A reads the input operation information for one object from the input operation storage means 602, and extracts reply history information. Then, the extracted information is input to the input information presenting means 6.
04, and a concave display corresponding to the number of replies is performed. FIG.
Shows the system presentation state after performing the receiving operation in the present invention. The input operation reproducing means 603 is shown in FIG.
As shown in (1), the input operation object of the sender of the electronic mail is displayed so as to be depressed by one level.
In this way, the object can be displayed three-dimensionally, and a three-dimensional indentation expression that the object is in a state where the reception history is old and the object is concave is possible. By using such an expression method, the exchange history of two-dimensionally scattered objects can be expressed, and the object can be easily quoted at the time of reply.

In general, text quoting in reply operation of the electronic mail system is performed by inserting an indicator and indenting. However, according to the present invention, by giving a visual effect to the display of the reply operation, multimedia can be displayed. It is possible to refer to objects temporally and spatially when dealing with objects.

(Fourth Embodiment) Next, a fourth embodiment of the present invention will be described. In the present embodiment, a reply mail when receiving a reply based on the mail after receiving the electronic mail has a feature. In the present embodiment, similarly to the third embodiment, a communication terminal having a transmission / reception function is connected to a network. The communication terminal includes the transmission function and the reception function of the first embodiment described above. The input operation control means 111A adds reply history information when citing the received document together with the ordering according to the input operation order of the first embodiment. That is, a reply mail is created in the same procedure as in the first embodiment described above, but when a specific object is quoted, an electronic mail is created as shown in FIG. The procedure for creating an electronic mail is the same as in the above-described embodiment.

Next, the operation on the receiving side will be described. The input operation reproducing means 603A reads the input operation information for one object from the input operation storage means 602, and extracts reply history information. FIG. 10 shows a system presentation state after performing a receiving operation in the present embodiment. As shown in FIG. 10, the input operation reproducing means 603B in the present embodiment changes the gradation of the input operation object of the sender of the electronic mail so that the color density and the luminance are reduced by one level, for example. indicate. Here, the portion where the luminance decreases is shown by hatching. In other words, every time an electronic mail is quoted and exchanged, it is possible to express that the cited portion is reduced in color depth by one level each time the quote is quoted, so that the object is blurred from the oldest transmission / reception history. .

In general, a sentence is quoted in the reply operation of the electronic mail system by inserting an indicator and indenting. However, according to the present invention, by giving a visual effect to the display of the reply operation, multimedia can be displayed. It is possible to refer to objects temporally and spatially when dealing with objects.

[0063]

As described above, according to the present invention, a multimedia electronic mail reflecting the intention of an electronic mail creator can be created by utilizing natural human input operation information, and a sender can be created. Operations can be exchanged. Also, a reply operation of an input operation can be realized even in a reply operation indispensable to the electronic mail system, and a noticeable effect that a mail exchange history can be presented by giving a visual effect to the display of the reply operation. Is obtained.

[Brief description of the drawings]

FIG. 1 is a system configuration diagram of a transmission terminal according to an embodiment of the present invention.

FIG. 2 is a hardware configuration diagram of a transmission terminal according to one embodiment of the present invention.

FIG. 3 is a diagram showing an example of creating a multimedia electronic mail.

FIG. 4 is a diagram showing an example of input operation information stored in input operation storage means.

FIG. 5 is a diagram showing processing of an input operation control unit according to one embodiment of the present invention.

FIG. 6 is a system configuration diagram of a receiving terminal according to an embodiment of the present invention.

FIG. 7 is a diagram showing processing of an input operation reproducing unit according to the embodiment of the present invention.

FIG. 8 is a diagram illustrating an example of a multimedia electronic mail including a pointing device operation object.

FIG. 9 is a diagram showing a display example of a reply mail according to the embodiment of the present invention.

FIG. 10 is a diagram showing a display example of a reply mail according to the embodiment of the present invention.

FIG. 11 is a diagram illustrating an example of input operation information including a pointing device operation object stored in an input operation storage unit.

FIG. 12 is a system configuration diagram of a transmission terminal according to a conventional example.

FIG. 13 is a system configuration diagram of a receiving terminal according to a conventional example.

FIG. 14 is a block diagram showing a configuration of a terminal system having a transmission / reception function according to a conventional example.

FIG. 15 is a diagram showing an example of a user interface for designating a temporal / spatial relationship according to a conventional example.

FIG. 16 is a diagram of media information stored in input storage means according to a conventional example.

FIG. 17 is a diagram of synchronization information stored in a synchronization information storage unit according to a conventional example.

FIG. 18 is a diagram showing an example of a list created from a temporal relationship according to a conventional example.

FIG. 19 is a flowchart illustrating a procedure for presenting information according to a conventional example.

FIG. 20 is a diagram for explaining a reply operation according to a conventional example.

FIG. 21 is a diagram for explaining a reply operation according to a conventional example.

[Explanation of symbols]

 Reference Signs List 101 Text input means 102 Graphic input means 103 Pointing device operation input means 104 Voice input means 105 Still image input means 106 Moving image input means 107 Drawing operation input holding means 108 Voice input holding means 109 Still image input holding means 110 Moving image input holding Means 111, 111A Input operation control means 112 Input operation storage means 113 Input operation changing means 114 Input operation presenting means 201 Display 202 Video RAM 203 Speaker 204 Keyboard 205 CPU 206 Main storage 207 External storage 208 Microphone 209 Pointing device 210 Image input Apparatus 601 Receiving means 602 Input operation storage means 603, 603A, 603B Input operation reproducing means 604 Input operation presenting means 1201 Text input Stage 1202 voice input unit 1203 still image input unit 1204 text input storage means 1205 voice input storage means 1206 still image input storing means 1207 synchronization information changing unit 1208 synchronization information storage unit 1209 synchronization information presenting means

Claims (8)

[Claims] 1. A multimedia electronic device comprising: a transmitting terminal for inputting / editing a plurality of media information and transmitting the same as electronic mail; and a receiving terminal for receiving electronic mail from the transmitting terminal via a network and presenting received information. In the mail system, the transmission terminal includes: text input means for inputting text; graphic input means for inputting a graphic; voice input means for inputting voice; image input means for inputting an image; and the text input means. A drawing operation input holding unit that temporarily holds drawing operation information obtained from the graphic input unit; a voice input holding unit that temporarily holds voice input information obtained from the voice input unit; Image input holding means for temporarily holding obtained image input information, the text input means, the graphic input means, Information input to the image input means, the drawing operation input holding means,
An input operation control unit configured to perform an ordering according to an input operation order from information held in the voice input holding unit and the image input holding unit; and an input operation storage unit configured to store the input operation information ordered by the input operation control unit. Input operation change means for changing input operation information stored in the input operation storage means; input operation presentation means for presenting input operation information changed by the input operation change means; and input operation storage means Transmitting means for transmitting the information as electronic mail, the receiving terminal comprising: a receiving means; an input operation reproducing means for reproducing the received electronic mail as input operation information; and the input operation reproducing means. Multimedia input mail providing means for presenting reproduced input operation information. Temu. 2. The transmitting terminal has a pointing device operation input means for inputting a pointing device operation, and the input operation control means includes an input operation sequence including input information from the pointing device operation input means. 2. The multimedia electronic mail system according to claim 1, wherein the input operation reproducing means of the receiving terminal reproduces a movement of a pointing device at the time of input from the received electronic mail. 3. The multimedia electronic mail system according to claim 1, wherein said image input means is an input means for inputting at least one of a still image and a moving image. 4. A multimedia electronic mail system including a communication terminal having a transmission function of inputting / editing a plurality of media information and transmitting it as electronic mail, and a receiving function of receiving electronic mail via a network and presenting received information. The communication terminal includes: text input means for inputting text; graphic input means for inputting a graphic; voice input means for inputting voice; image input means for inputting an image; A drawing operation input holding unit that temporarily holds drawing operation information obtained from a graphic input unit; a voice input holding unit that temporarily holds voice input information obtained from the voice input unit; Image input holding means for temporarily holding input image input information, the text input means, the graphic input means, and the image input means. Information input to the input means, the drawing operation input holding means,
An input operation control unit configured to perform an ordering according to an input operation order from information held in the voice input holding unit and the image input holding unit; and an input operation storage unit configured to store the input operation information ordered by the input operation control unit. An input operation changing unit that changes input operation information stored in the input operation storage unit; a receiving unit; an input operation reproducing unit that reproduces a received electronic mail as an input operation; An input operation presenting means for presenting the changed input operation information and the input operation information reproduced by the input operation reproducing means, and a transmitting means for transmitting the information of the input operation storage means as an electronic mail. A multimedia electronic mail system, characterized in that: 5. The input operation control means adds reply history information together with the order according to the input operation order, and the input operation reproduction means three-dimensionally converts the media information based on the reply history information. 5. The multimedia electronic mail system according to claim 4, wherein the multimedia electronic mail system is displayed. 6. The input operation control means adds reply history information together with an order according to an input operation order, and the input operation reproduction means gradates media information based on the reply history information. 5. The multimedia electronic mail system according to claim 4, wherein the multimedia electronic mail is displayed. 7. The communication terminal includes a pointing device operation input unit for inputting a pointing device operation, and the input operation control unit includes an input operation sequence including input information from the pointing device operation input unit. 5. The multimedia electronic mail system according to claim 4, wherein the input operation reproducing means reproduces a movement of a pointing device at the time of input from the received mail. 6. 8. The multimedia electronic mail system according to claim 4, wherein said image input means is an input means for inputting at least one of a still image and a moving image.