EP0909420A3

EP0909420A3 - System for composing multimedia letters

Info

Publication number: EP0909420A3
Application number: EP98912660A
Authority: EP
Inventors: Aninda Dasgupta
Original assignee: Koninklijke Philips Electronics NV; Philips AB; Philips Svenska AB
Current assignee: Koninklijke Philips NV; Philips AB; Philips Svenska AB
Priority date: 1997-04-25
Filing date: 1998-04-20
Publication date: 2002-11-06
Also published as: WO1998049631A2; EP0909420A2; JP2000512788A; WO1998049631A3

Abstract

A web server is equipped with software allowing users to compose multimedia e-mails with personal content. The users download plug-ins for acquiring assets. The users then provide the acquired assets to the server which organizes them into multimedia e-mails. The recipient is notified of the e-mail and can browse it on the server with only a web browser.

Description

System for composing multimedia letters.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The invention relates to a system for composing and sending multimedia e-mails.

2. Related Art

There are a number of systems for creating multimedia e-mail in a computer work station or personal computer environment. These include Alta Vista's "HOWDY"; EUDORA mail; Sun's SOLARIS e-mail package; and Pineapple's "ANIMATED GREETINGS" .

These systems suffer from several drawbacks. First, incompatibilities between various machines and e-mail software packages in a network, such as the Internet, often lead to the format of the multimedia message being undecipherable by the receiving system. Second, if the recipient has a relatively simple piece of equipment, which functions only as a web browser, the recipient may not be able to receive a complex multi-media e- mail. Third, the process of composing multimedia e-mail on the sender's end is very cumbersome, including 1) having to run separate programs to capture/acquire content like video, audio, and images; 2) having to save the content into individual files on local disk; and 3) specifying those filenames to the e-mail composition software.

SUMMARY OF THE INVENTION

The object of the invention is to create a system which can allow the recipient to receive multimedia e-mails with personal content of the sender while avoiding the listed drawbacks. This object is achieved by a system in which e-mails are stored on a central server so that they can be browsed by low function equipment so long as the user knows the necessary security information.

This system has the additional advantage that the e-mail facility provides a draw to the central server. If the central server provides advertising or infomercial type information, users will be exposed to such information in making use of the e-mail facility.

Some systems which allow users to compose postcards on a server are known. Examples are shown at http : //www . yahoo . com/Entertainment/Humor_Jokes_and_Fun/Greeting_Cards_on_www/ . Such systems are not true e-mail systems, because there is no personal multimedia content. The users are constrained to use canned multimedia content from the web site.

BRIEF DESCRIPTION OF THE DRAWING

The invention is described by way of non-limitative example with reference to the following figures.

Fig. 1 shows a network in which the invention can operate.

Fig. 2 shows a user system.

Fig. 3a shows a flow chart of the operation of the invention from the point of view of the server. Figs. 3b is a flowchart of the plug-in DLL and further expands box 307 of fig. 3a from the point of view of the user system or browser.

Fig. 3c is a flowchart of the asset capture DLL.

Fig. 4 shows more aspects of the user system.

Figs. 5a-f show screen appearances of a preferred embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Fig. 1 shows a system which includes a network 101, such as the internet, a server 102, one user's system 103, and another user's system 104. For the purpose of this disclosure, the one user is going to be the sender of the multimedia e-mail message, while the other user is going to be the receiver of the multimedia e-mail message.

Fig. 2 shows more detail of the one user's system 103. The system includes a network interface unit 201. A network interface unit can be any standard unit, such as a telephone modem, a cable modem, or an ethernet card. The system also includes a display device 202, which can be a TV monitor, computer monitor, LCD screen or other display. The composition unit can be any processor such as a PC or set top box. There is a user input device 204 such as a wired keyboard, a wireless keyboard, a touch sensitive component of the display, a telephone keypad, a TV remote control, or a voice recognition unit. Content for a multimedia e-mail message is to be acquired from content sources, such as a VCR 205, a video camera 206, a microphone or other audio equipment 207, a PC hard drive or CD-ROM 208, or a scanner 209.

Fig. 3a shows a detailed flow chart of the operation of the system according to the invention from the point of view of the server. At 301 the user, who wishes to create a multimedia e-mail, visits the web site. At 302, the user logs in or registers and is assigned a session i.d. and, if a new user, a user i.d. At 303, the user is shown an HTML page with command options. Fig. 5a shows the page of the preferred embodiment.

At 315, it is tested whether a send command is received. If not, the system receives a user asset selection at 304. Fig. 5b shows an asset selection page of the preferred embodiment. The selection is achieved by the user by clicking on a button on the user screen. This screen includes 4 buttons for text 501, audio 502, still photos 503, or video 504, respectively.

The system then retrieves an asset "plug- in" for the type of asset selected. "Plug-ins" are standard extensions to web browsers. Here plug-ins are used to capture individual types of assets. The code is particular to the type of device which captures the asset. For example, the plug-in to scan and digitize a photograph will use the industry standard TWAIN scanner library and specifications to interact with any TWAIN compliant scanning device. The plug-in for capturing audio and video will use standard Microsoft libraries, like MFC and /or Video for Window, on a PC. The plug-ins are preferably implemented as DLL's (Dynamic Link Libraries) according to the plug-in specification published by Netscape.

More information about plug-ins can be found at http .-//home, netscape, com/eng/mozilla/3.0/handbook/pg guide, htm. Other plug-ins can be found associated with Netscape Navigator and the Microsoft Internet Explorer. Other platforms, for instance set top boxes or digital TVS, have software development tools which allow those of ordinary skill in the art to develop equivalents of plug- ins for their own platforms. Accordingly, the architecture shown herein is readily portable to all Internet browsers. Figs. 3b & c show two portions of the operation of box 307 from the point of view of the browser. The portion of Fig. 3b is the portion used by the browser to receive assets and send the captured assets to the server. Fig. 3b is known as the plug- in DLL. The software of Fig. 3b also creates interface screens for the user. The portion of Fig. 3c shows the software which interacts with the driver software and acquisition hardware, e.g. the video capture board, to capture the asset and save it on the local disk. The module of Fig. 3c is also called the asset capmre DLL. The functions of Fig. 3b and 3c could be combined by one of ordinary skill in the art to create a monolithic piece of software which performs both functions. Fig. 3b starts at 320 with the reception of parameters from the browser: session i.d.; user i.d.; window size; colors; etc. At 321 and 322 the plug-in DLL, used in Fig. 3b, and the asset capture DLL, used in Fig. 3c, are initialized. At 323, the user screen is created or modified. Examples of user screens are shown in Figs. 5a-f. At 324, the software waits for user commands or messages from the asset capture DLL. At 325 the software determines whether it is time to start capmre. If so, at 326, the asset capture DLL is requested to capmre the asset and control is returned to 323.

If it is not time to start capture, at 327, it is tested whether it is time to stop capmre. If so, the asset capture DLL is requested to stop capmre at 328 and control is returned to 323.

If it is not time to stop capmre, at 329 it is tested whether it is time to play a captured asset. Such play would be in response to a user request. If there has been such a request, at 330, the asset capmre DLL is requested to play the recorded asset. If there has not been such a request, then control proceeds to 331. At 331, it is tested whether it is time to send a captured asset to the server. If so, at 332, the filename of the captured asset is provided to the browser and the browser is asked to upload the file to the server. Thereafter, at 334, the plug-in DLL is deinitialized at 334 and this procedure ends.

If, at 331, it is not time to send a captured asset to the server, some error has occurred. Accordingly, a time out notification from the asset capmre DLL is sent at 333 and a diagnostic message is displayed for the user at 335. Then control is returned to 323.

Fig. 3c shows operation of the asset capmre DLL. At 340, the asset capmre DLL is initialized. Then at 341, the asset capmre DLL waits from a request from the plug-in DLL, an error, or a timeout. At 342, it is tested whether there is a capmre request from the plug-in DLL. If so, a max timeout time is set at 352. The capmre device is requested to start capture at 351, and control returns to 341.

If there is no request to start capmre, then it is tested at 343 whether there is a request to stop capmre. If there is a request to stop capture, then the asset is saved to a file at 344, the plug-in DLL is notified of the file name, and control proceeds to 350. If there is no request to stop capmre, then at 345 it is tested whether there is a request to play. If there is such a request the capmred asset is played at 353 and control returns for 341.

If there is no request to play, then, at 346, it is tested whether there is a timeout. If so, the asset is saved in a file at 347, and a notification message is made for the plug-in DLL at 349. Then control passes to 550.

If there is no timeout after box 346, then an error must have occurred and an error message is made for the plug-in DLL at 348 and control passes to box 350. At 350, the asset capmre DLL is deinitialized. Returning to Fig. 3a, once the user selection is received, the plug-in software is to be retrieved at 305. The system must then determine, at 306, if the user has previously captured this type of asset, or more specifically, whether the user already has the relevant plug-in installed on local disk.

If the user needs a plug-in, then at 312, that plug-in is downloaded from the web server. Once the system has a plug-in, the plug-in needs to be loaded and run at 307. Capmred assets are then received from the user computer at 309 and the assets are added to the multimedia message at 310. The assets are transferred to the web server using standard 'Multipart forms' format.

The following is pseudocode for capmre and transfer of an asset by the browser on the user computer.

Initialize plug-in

Read parameters passed by browser (e.g. user i.d., session i.d., asset type)

Scan local setup for a device or devices to be used for asset capmre Initialize local devices

Display user-interface

Process user input to start, stop, edit capmre Save captured asset on local storage Ask browser to stop connection with server Use browser to submit multi-part HTML form containing stored asset

Pass session and user i.d. 's to server End The following is pseudo code for the server If first time user then register user else check identity assign session i.d. show screen with asset buttons if user clicks on asset button then serve empty html file with embedded plug-in object else if user clicks on address/send button then take user through addressing and sending of message; notify recipient if browser connects for upload of capmred asset then save uploaded asset using session i.d. and user i.d. as identifier of message being composed

Fig. 5c shows a screen as seen by the user when running the photo scanning plug-in in the preferred embodiment. Fig. 5d shows a screen as seen by the user when running the video recording plug-in. Fig. 5e shows a screen as seen by the user when running the audio recording plug-in. Fig. 5f shows a part of a finished letter as displayed in the preferred embodiment.

When the recipient is notified of the message, the recipient needs to be given some identification of the message. In the case of the above pseudo code, the session i.d. and user i.d. are used to identify the message for the recipient. This serves as a kind of security, since only the recipient will know these i.d. 's. If additional security is desired, password access can be established for the messages or the messages can be scrambled.

Fig. 4 shows more detail of the configuration of the software modules on the user system. The web server 102 is the same as in Fig. 2. The Figure also shows the web browser 401, the plug-in 402, and the asset capmre device 403.

Claims

CLAIMS:

1. A server comprising interface means for interfacing to a network and for receiving therefrom and providing thereto multimedia information; memory means; processing means, cooperating with the interface means and the memory means, for organizing the multimedia information in downloadable or browsable format in the memory means for access by a remote receiver via the interface means, which multimedia information includes multimedia e-mails with personal content, which multimedia e-mails are received via the interface means from at least one remote sender; coordinating message identification information received via the interface from the remote receiver; allowing the remote receiver to access a particular multimedia e-mail. 2. A system comprising a network; the server of claim 1 ; and the remote sender, comprising a multimedia e-mail composition facility.

3. The system of claim 2, wherein the multimedia e-mail composition facility comprises at least one plug-in or plug-in equivalent downloaded from the server.

4. The system of claim 3 wherein the remote receiver is able to browse the particular multimedia e-mail, but not able to receive multimedia e-mail sent over the network.

5. A method of multimedia communication comprising the steps of first receiving, via a network, at a server, a multimedia e-mail composed by a remote sender; organizing the multimedia e-mail in a database for later retrieval; sending information, via the network, about the stored e-mail to a remote receiver from the remote sender; second receiving, via the network, at the server, a request for the multimedia e- mail from the remote receiver; and allowing, by the server, the remote receiver to access the multimedia e-mail via the network. 6. The method of claim 5 wherein the remote sender is unable to send the e- mail directly to the remote receiver via the network.

7. The method of claim 5 wherein the server provides a home page for browsing via the world wide web and the server requires the remote receiver or the remote sender to browse at least part of the home page prior to the first or second receiving step, respectively.

8. A method of composing a mμltimedia e-mail at a send station, the method comprising the following steps; a. accessing a server via a network; b. displaying screen information received from the server indicating user choices; c. transmitting asset selection information from the send station to the server; d. determining whether the send station has a plug-in or plug-in equivalent for acquiring a desired asset; e. upon a negative result of the determining step, downloading the plug-in or plug-in equivalent from the server for acquiring the desired asset; and f. acquiring the desired asset using the plug-in or plug-in equivalent.

9. A method for using a server to assist in composing a multi-media e-mail, the method comprising the steps of: a. receiving an access request from a send station; b. transmitting screen information indicating user choices to the send station; c. receiving asset selection information from the send station; d. determining whether the send station has a plug-in or plug-in equivalent for acquiring a desired asset; e. upon a negative result of the determining step, downloading the plug-in or plug-in equivalent to the send station for acquiring the desired asset; and f. receiving the desired asset from the send station after the send station acquires the desired asset using the plug-in or plug-in equivalent.

10. The method of claim 9 further comprising the steps of using the desired asset to compose a multimedia e-mail; and retaining the multimedia e-mail in browsable format on the server.