KR20060135165A - System for real-time transmitting and receiving of audio/video and handwriting information - Google Patents

Abstract

A system for transceiving A/V(Audio/Video) and handwriting information in real-time is provided to simply, conveniently, and directly mix messages, and prevent virus infection through an email in a portable device and a PDA(Personal Digital Assistant). A format identifier(11) enables a CPU to identify a source and a format of an information piece if the information piece is received to a computer. A packet former(12) forms an information packet having multiple storage sections in a data storage device of the computer. The storage sections include a destination, image, audio, handwriting trace, and captured information storage section. An encapsulator(13) stores the information piece having the identified format to the storage section matched with the information packet. An information transmitter(14) transmits the information packet to a destination of the destination information storage section. An information receiving/displaying part(15) displays contents of each storage section of the information packet as a prompt window when the information packet is received to the computer.

Description

System for Real-Time Transmitting and Receiving of Audio / Video and Handwriting Information}

1 is a flowchart illustrating an information transmission and reception process according to the present invention;

2 is a diagram schematically illustrating various information sources according to the present invention;

3 is a diagram schematically showing the structure of an information picket according to the present invention;

4 is a diagram schematically showing a message prompt window according to the present invention;

5 is a diagram schematically illustrating a nested message prompt window according to the present invention; and

 6 through 8 are flowcharts illustrating various execution flows in accordance with an embodiment of the present invention.

FIELD OF THE INVENTION The present invention relates generally to information transmission and reception, and more particularly to a system for transmitting and receiving static and dynamic information such as audio, video, and correspondence information in real time.

To date, despite the advances in technology, most publicly available information delivery devices are either telephones that provide only voice exchange or e-mails that exchange only text between information carriers. Telephone-based information transfer is simple but expensive and only voice can be sent and received. On the other hand, e-mail has gained widespread popularity with the spread of the Internet. Inexpensive and capable of delivering multiple messages has become an important means of communication for modern people. However, e-mail is mainly text based, and other information than text can be transmitted only by an attached file type. To send and receive information other than text, you must have some knowledge of the computer. Moreover, for those who perform e-mail through typing on the keyboard, it is burdensome for those who use English as a second language or who do not want typing.

It is a primary object of the present invention to provide a system that is easy to operate to transmit and receive static and dynamic information such as audio, video, textual image information in real time.

As a means according to the invention, first identify the source and format of the information to be processed. Information formats include images, audio, video, text and the like. These pieces of information are identified and then encapsulated one by one in the corresponding storage section in the information packet. After all these pieces of information have been processed, the information packets are stored and / or sent to the receiver via Ethernet. The receiver presents all the pieces of information that accompany an information packet, without immediately returning to multiple applications after receiving the entire information packet. Thus, the present invention can overcome current e-mail limitations that require character typing and must be familiar with the application in order to view non-letter information accompanying the e-mail. The present invention allows a user to read and reply to a message by simply executing a click.

As shown in Fig. 1, the present invention includes the following processing means.

A. Format identification means (11) for causing the central processing unit of the computer to identify the source and format of the information fragment when the piece of information is received at the computer. All possible formats of static and dynamic information can be video, images, audio, documents, and the like. This piece of information is received through several mechanisms as shown in FIG. 2, including the following apparatus.

a. A camera 111 mounted to a personal computer (laptop computer) which catches video or image information;

b. A tablet (or mouse) 112 connected to a personal computer (laptop computer) to obtain information by hand operation when used by a user;

c. A microphone 113 connected to a personal computer (laptop computer) in which voice information is recorded; And

d. Video display 115 where static or dynamic video currently being displayed is captured.

B. Packet preparation means for preparing an information packet 20 having a plurality of storage sections in a data storage device of a personal computer (notebook computer). This section comprises the transfer destination information storage section 21, the image information storage section 22, the audio information storage section 23, the character information storage section 24 and the captured image storage section 25 as shown in FIG. Include. The transfer destination information storage section 21 may include a recipient address (eg, an email address), a file path and name in the data storage, or the recipient's account ID. The image information storage section 22 is used for storage of video or image information.

C. Encapsulation means for storing a piece of information with a particular format identified by the format identification means in a corresponding storage section of the information packet.

D. Information transmission means for transmitting an information packet via an internal bus or the Internet to a recipient in a delivery destination storage section 21, such as the recipient's email address or account ID or the path and name of a file.

E. Information receiving / notifying means for displaying a message prompt window 26 as shown in FIG. 4 as the top window on the video display 27 when an information packet is received at a personal computer (notebook computer). The user can read the packet contents by clicking on the message prompt window 26, and the computer's operating system continuously draws the information stored in each storage section of the packet and formats it together on the video display 27 by the multiplexing player. To display this information.

An embodiment of the message prompt window 26 described above is shown in FIG. The message prompt window 26 'of another embodiment is shown as multiple nested message prompt windows 26' on top of the video display 27 as shown in FIG. The message prompt windows 26, 26 'include the following:

The date and time 261 at which the information packet was received;

Sender information 262 indicating the name of the sender, the name of the sender (or entity), the source of the information packet, the subject, and the like;

An open state 263 indicating whether the content of the information packet is already open and being displayed;

A message level 264 indicating normal, urgency, and importance for the importance of the information packet;

A window control button 265 for closing the message prompt window 26; And

Window control button 266 for deleting the information packet.

Therefore, when the information packet 20 is received, the user can see the corresponding message prompt window 26 through the video display 27, and can know who is the sender and the importance of the contents. If the user wants to read the contents of the information packet 20, the contents of the information packet 20 are displayed on the video display 27 by clicking on the open state 263 of the message prompt window 26. If desired, the user can also reply to the sender or forward the content to others using the reply means. By the reply means, the user can perform the above-mentioned AE process by using the tablet, camera, and microphone installed in the computer while reading the contents of the information packet 20, and can store the contents of the reply in the information packet 20. In addition, you can send it to the original sender or anyone else you want.

In accordance with the present invention, a user can store video and images captured using a camera, tablet, and microphone installed in a computer, traces derived from the tablet, and the like, completely and directly in an information packet. Costs are very important for individuals and companies that carry packet transmissions over the Internet and already have broadband connections (such as ADSL or wired networks).

Based on the format by the storage section of the information packet, the present invention can perform several applications as follows:

(1) When the information packet 20 consists only of the transmission destination information storage section 21, the image information storage section 22, and the audio information storage section 23, the present invention can be installed as a video / audio response system. have.

(2) the information packet 20 has a transmission destination information storage section 21, an image information storage section 22, an audio information storage section 23, a character information storage section 24 and a captured image storage section 25. In the case of the present invention, the present invention can be installed as an electronic report system or an electronic document system including video, audio and text.

(3) When the information packet 20 consists only of the handwriting trace information storage section 24, the present invention can be installed as an electronic whiteboard or an electronic notepad system.

By using the means according to the invention, a real-time information transmission and reception system can be made, and the processing thereof is shown in FIG. The system includes three main functions, the Function Zoo, M-messages, and M-writing, each of which is described below.

Function Zoo

As shown in Fig. 6, when the operating system of the computer is started, it automatically enters the step 30 of starting the real-time information transceiving system and displaying the user interface on the video display as shown in Fig. 4. For the functional group to work, you must annotate pieces of information with video, audio and text. Thus, the user first opens a data window displaying the information to be annotated or reported. In the next step 31 it waits until the user selects the desired function. When the user selects a functional group, it enters execution flow step 32 and simultaneously records video captured by camera 111, audio captured by microphone 113, and text captured by tablet 112. It begins. When recording is complete, the recorded content automatically appears in the preview window in step 35. In this case, the user may print the contents of the preview window in step 351. This is actually a screen capture function that allows the user to not only print the contents of any window that appears on the video display but also save the contents to a file at steps 40 and 391. In other words, this feature not only allows the user to save the captured content of the display to a file, but also to save it with video / audio annotations. In this way, you can view all kinds of reports or documents later with live video / audio annotations. The user can then select a recipient from the address book to select whether to send the contents of the preview window in an information packet in step 36. Once the recipient is determined, the system proceeds to step 38, asking for the subject of the information packet (importance, etc.) and continuing to send the packet. Subsequently, in step 39, the user determines whether to save the contents of the preview window in a file and performs step 391 if the user wants to save the file. Otherwise, the user performs step 392.

As shown in FIG. 7, when the receiver operates the real-time information transmission / reception system in step 950 and the information packet arrives, the system displays a message prompt window in step 51 shown in FIG. 4. Depending on how the information packet is formed (functional groups, M-messages, etc.), the message prompt window will use different colors and / or styles. The following information from the received information packet can be displayed in the message prompt window:

(a1) the date the information packet was formed and sent;

(b1) the time at which the information packet was formed and sent;

(c1) how the information packet was formed;

(d1) the name of the sending company;

(e1) the name of the sender;

(f1) whether the information packet has been viewed;

(g1) whether or not the information packet has been stored;

(h1) whether the information packet has been deleted;

(i1) the importance of the information packet; And

(j1) The subject of an information packet.

The receiver may store the information packet in step 53, delete it in step 54, or view the information packet in step 52. When opened in step 52 to read an information packet, the recipient can choose to:

(a) reply (step 55) or forward (step 56) the information packet;

(b) after steps 55 and 56, deletion 54 of the information packet;

(c) after steps 55 and 56, storage 53 of information packets;

(d) printing 57 an information packet;

(e) reproduction 58 of the information packet;

(f) deletion 54 of the information packet; And

(g) File storage 53 of information packets.

When the recipient replies or forwards the information packet, the system automatically activates the same authentication environment as forming the information packet as shown in FIG. For example, if the recipient replies after step 55 of FIG. 7, the execution flow continues to step 32 or 33 of FIG. 6. If the receiver delivers after step 56 of FIG. 7, the original information packet is maintained and the execution flow will continue to step 34 of FIG. 6. The original information packet will be delivered with the newly created comment.

M-message

Returning to FIG. 6 again, once the operating system of the computer is started, it automatically enters the real-time information transmission / reception step 30 to present the video display to the user. In the next step 31, the user waits in accordance with the present invention to select the desired function. If the user selects an M-message, the execution flow enters step 33 where the simultaneous recording of video captured by camera 111 and audio captured by microphone is started. When recording is complete, the recorded content automatically appears in the preview window in step 35. At this time, the user selects whether to save the contents of the preview window in step 40. In other words, the M-message function acts like a recorder to store the user's video and audio in a file for later viewing or reference. The user can select a recipient from the address book to select whether to send the contents of the preview window in an information packet in step 36. Once the recipient is determined, the system proceeds to step 38, asking for the subject of the information packet (importance, etc.) and continuing to send the packet. Subsequently, in step 39, the user determines whether to save the contents of the preview window in a file and performs step 391 if the user wants to save the file. Otherwise, the user performs step 392.

Similarly, if an information packet is opened by the receiver for reading after step 52 of Figure 7, the receiver selects:

(a) reply (step 55) or forward (step 56) the information packet;

(b) after steps 55 and 56, deletion 54 of the information packet;

(c) after steps 55 and 56, storage 53 of information packets;

(d) printing 57 an information packet;

(e) reproduction 58 of the information packet;

(f) deletion 54 of the information packet; And

(g) File storage 53 of information packets.

When the recipient replies or forwards the information packet, the system automatically activates the same authentication environment as that of forming the information packet as shown in FIG. For example, if the recipient replies after step 55 of FIG. 7, the execution flow continues to step 32 or 33 of FIG. 6. If the receiver delivers after step 56 of FIG. 7, the original information packet is maintained and the execution flow will continue to step 34 of FIG. 6. The original information packet will be delivered with the newly created comment.

M-writing

Referring to Figures 6 and 8, when the operating system of the computer is started automatically enters the real-time information transmission and reception step 30 so that the video display appears to the user. In the next step 31, the user waits in accordance with the present invention to select the desired function. If the user selects M-Create, the execution flow enters step 34 to begin recording of the document captured by the tablet. Details of step 34 are shown in FIG. 8. The user decides in step 66 whether to start a new recording. Otherwise, the user searches for and opens the recording saved in step 61 to modify it. When the user presses the "New" button in step 62 to start a new recording, the system starts recording the user documenting information on the tablet 112. When recording is complete, the user selects whether to print the information in step 64 or continue to step 65 to simultaneously record the video, audio and document (step 32 in FIG. 6) in combination with the newly created document information. When recording is complete, the recorded content automatically appears in the preview window in step 35. At this time, the user selects whether to save the contents of the preview window in step 391 or export it as an information packet in step 36. Once the recipient is determined, the system continues with step 38 of sending the packet. Subsequently, in step 39, the user determines whether to save the contents of the preview window in a file and performs step 391 if the user wants to save the file. Otherwise, the user performs step 392.

As described above, the system according to the present invention replaces the conventional e-mail transmission / reception method, which can simplify and simplify message composition, and can directly perform a message, as well as prevent virus infection through e-mail. The system can also be applied to portable devices or PDAs.

Claims (30)

Systems for transmitting and receiving information in real time between computer devices over the Internet, including: Format identification means for causing a central processing unit of the computer to identify the source and format of the information piece when the information piece is received at the computer; An information packet having a plurality of storage sections is formed in a data storage device of a computer, where the plurality of storage sections are a transmission destination information storage section, an image information storage section, an audio information storage section, a handwriting trace information storage section. Packet forming means for including a captured storage section; Encapsulation means for storing a piece of information having a format identified by the format identification means in a corresponding storage section of an information packet; Information transmitting means for transmitting the information packet to a predetermined destination of the transmission destination information storage via the Internet; And Information receiving / displaying means for displaying the contents of each storage section of the information packet as a prompt window on the video display when the information packet is received by the computer. 2. The system of claim 1, wherein the format of the pieces of information is one of video, image, audio, and correspondence traces. 2. The system of claim 1, wherein the transmission destination information storage section includes an email address of a recipient. 2. The system of claim 1, wherein the transmission destination information storage section includes a file path and name of the data storage device. 2. The system of claim 1, wherein the transmission destination information storage section includes a recipient's account ID. The system of claim 1, wherein the contents of the message prompt window include: The date and time the information packet was received; Sender information indicating the name of the sender, the name of the sender (or entity), the information packet and the subject; A browsing state indicating whether the contents of the information packet are already open and being displayed; Message importance indicating the importance of the information packet; And Multiple window control buttons to close or delete the message prompt window. 7. The system of claim 6, wherein the message importance is one of normal and urgent. The system of claim 1, wherein the pieces of information are image information captured by a camera installed in a computer. The system of claim 1, wherein the pieces of information are correspondence trace information captured by a tablet or a mouse installed in a computer. The system of claim 1, wherein the piece of information is audio information captured by a microphone installed in a computer. The system of claim 1, wherein the pieces of information are still picture information captured by a video display connected to a computer. 2. The system according to claim 1, further comprising information display means for simultaneously displaying the contents of said information packet on a video display of a computer. 2. The system of claim 1, further comprising reply means for allowing a user to capture information from a camera, tablet, microphone, and video display and store it in an information packet. Systems for transmitting and receiving information in real time between computer devices over the Internet, including: Video / audio information capturing means for capturing pieces of video information and pieces of audio information from a video display of a computer; An information packet with a plurality of storage sections is formed in a data storage device of a computer, and the plurality of storage sections contain a transfer destination information storage section, an image information storage section, an audio information storage section, a letter trace information storage section, and a captured storage section. Packet forming means for inclusion; Information transmitting means for transmitting the information packet to a predetermined destination of the transmission destination information storage via the Internet; And Information receiving / displaying means for displaying the contents of each storage section of the information packet as a prompt window on the video display when the information packet is received by the computer. 15. The system of claim 14, wherein the transmission destination information storage section includes a recipient's email address. 15. The system of claim 14, wherein the transmission destination information storage section includes a file path and name of the data storage device. 15. The system of claim 14, wherein the transmission destination information storage section includes a recipient's account ID. 15. The system of claim 14, wherein the contents of the message prompt window include: The date and time the information packet was received; Sender information indicating the name of the sender, the name of the sender (or entity), the information packet and the subject; A browsing state indicating whether the contents of the information packet are already open and being displayed; Message importance indicating the importance of the information packet; And Multiple window control buttons to close or delete the message prompt window. 19. The system of claim 18, wherein the message importance is one of normal and urgent. 15. The system of claim 14, further comprising information display means for simultaneously displaying the contents of the information packet on a video display of a computer. 15. The system of claim 14, further comprising reply means for allowing a user to capture information from a camera, tablet, microphone, and video display and store it in an information packet. Systems for transmitting and receiving information in real time between computer devices over the Internet, including: Video / audio information capturing means for capturing a piece of correspondence trace information from a video display of a computer and recording captured image information of a currently active window; An information packet having a plurality of storage sections is formed in a data storage device of a computer, and the plurality of storage sections contain a transfer destination information storage section, an image information storage section, an audio information storage section, a letter trace information storage section, and a captured storage section. Packet forming means for inclusion; Information transmitting means for transmitting the information packet to a predetermined destination of the transmission destination information storage via the Internet; And Information receiving / displaying means for displaying the contents of each storage section of the information packet as a prompt window on the video display when the information packet is received by the computer. 23. The system of claim 22, wherein the transmission destination information storage section includes an email address of a recipient. 23. The system of claim 22, wherein the transmission destination information storage section includes a file path and name of the data storage device. 23. The system of claim 22, wherein the transmission destination information storage section includes a recipient's account ID. 23. The system of claim 22, wherein the contents of the message prompt window include: The date and time the information packet was received; Sender information indicating the name of the sender, the name of the sender (or entity), the information packet and the subject; A browsing state indicating whether the contents of the information packet are already open and being displayed; Message importance indicating the importance of the information packet; And Multiple window control buttons to close or delete the message prompt window. 27. The system of claim 26, wherein the message importance is one of normal and urgent. 23. The system according to claim 22, further comprising information display means for simultaneously displaying the contents of said information packet on a video display of a computer. 23. The system of claim 22, further comprising reply means for allowing a user to capture information from a camera, tablet, microphone, and video display and store it in an information packet. 23. The system of claim 22, further comprising means for storing the contents of the information packet in a file.

Images