JPH10171590A - Annotation information additional display system to existing application program and presentation system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To additionally display annotation information on the screen information of an application program without altering the existing application program by plotting the inputted annotation information on the screen information of the existing application program. SOLUTION: The window of the existing application program arranged under an annotation AP(application program for plotting an annotation) is not displayed and the window of the existing application program is made invisible even when the window of the annotation AP is moved (S4). In the case that an event received in an even reception queue (S5) is the annotation information (Yes of S6), based on the annotation information (the annotation information includes the kind and coordinate of the annotation), the annotation information is displayed on the window of the annotation AP (S7).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for annotating an existing application program, and a presentation system suitable for use in an attended meeting using the application program.

[0002]

2. Description of the Related Art Conventionally, even with existing application programs, annotations such as handwriting and pointer
notation (for example, meaning "annotation") is known to exist. When developing such an application program, it is common to develop it from the beginning so as to have an annotation function.

In a desktop conference system, a video conference system, and the like in which a plurality of remote terminals are connected via a network, a system for sharing annotations in a shared window such as a color board or a whiteboard application program is disclosed in, for example, Japanese Unexamined Patent Publication No. 58
658.

[0004] As an improvement in the use of annotation in a shared window, for example, Japanese Patent Laid-Open No. 7-20 / 1995
No. 0472 proposes a configuration of a computer communication conference support system that clearly associates each pointer with each user at the time of a conference and indicates which user the displayed pointer belongs to. -1
Japanese Patent Laid-Open No. 05105 proposes a conference system in which personal information of annotations and shared information can be handled separately.

[0005] However, in the systems proposed in the above publications, any of the shared windows has been developed from the beginning to have an annotation function.

[0006]

By the way, if there is a request to perform an annotation in an application program that does not have the annotation function, it is conventionally impossible to add the annotation function unless the existing application program is modified. could not.

Accordingly, the present invention has been made in view of the above circumstances, and has as its object to eliminate the need to modify existing application programs that do not have an annotation function, that is, to provide annotation information without modification. Is to provide an annotation information additional display method and a presentation system, which enables an additional display on the screen information of the application program.

[0008]

In order to achieve the above object, an annotation information addition display method according to the present invention obtains screen information of an existing application program and draws an annotation in accordance with the obtained screen information. Annotation AP ”window is displayed, the window of the existing application program is hidden, and the input on the screen of the existing application program is processed by the annotation AP. If the input is the annotation information, The annotation information is drawn on screen information of the existing application program.

Further, the annotation information addition display method of the present invention comprises the steps of (a) searching for an existing application program to be annotated, and (b)
Obtaining the screen information of the found application program; (c) displaying a window of an application program (referred to as “annotation AP”) for drawing an annotation based on the obtained screen information; Hiding the program window;
(E) receiving an event and determining the received event; (f) a processing step corresponding to the received event; and (g) an annotation on the annotation AP when the received event is annotation information. An annotation processing step for drawing information;
(H) acquiring the window information of the annotation AP when the received event is the end of the annotation; and (i) displaying the window of the existing application program which has been hidden based on the window information. (I)
And returning the existing application program to the original state.

Further, in the presentation system of the present invention, a server having data transmitting / receiving means and data broadcasting means, an existing application program, and an event received in addition to the above steps (a) to (i) are received from a network. Determine whether or not, in the case of reception from the network, including a process of emulating the received event into a local event, the annotation processing step, the step of drawing annotation information and transmitting to the network,
A plurality of the terminals are connected to the server via a network, and an annotation AP of a certain terminal is included.
Then, the annotation information processed on the screen information of the existing application program is transmitted to the server via the network, and in the server, the annotation report received by the data transmitting / receiving means is transmitted to the server by the data broadcasting means. Annotation information transmitted to all the other terminals connected to and received by each terminal is processed by the annotation AP, and the annotation information is displayed on the screen information of the existing application program similarly to the terminal that transmitted the annotation information. Is displayed, and it is possible to perform a work using the annotation information by using an existing application program among the users of the plurality of terminals.

[Summary of the Invention] In the present invention, when an annotation is to be performed on an existing application program, the position, size, and display screen information of a target application program on a display screen are obtained, and the annotation is obtained. The window of the application program (annotation AP) having a function is displayed according to the acquired position, size, and screen information of the existing application program, and the existing application program is hidden so that the annotation AP is not displayed.
Serves as a display image of an existing application program, processes an input on the display image, and draws an annotation on the display image if the input is annotation information.

Further, in the present invention, by adding a function of transmitting and receiving annotation information to the annotation AP, an annotation can be added to an existing application program without using a dedicated application program such as a white board in a desktop conference system. It is possible to work while performing.

According to the present invention having the above-described configuration, when it is desired to perform an annotation on an existing application program having no annotation function,
Existing application programs can be realized without modification. That is, it is possible to easily perform a presentation using the annotation using the application program used by the user. In addition, by sharing the annotation information with other users via a network, communication using the annotation on an existing application program becomes possible.

[0014]

Embodiments of the present invention will be described in detail below with reference to the drawings. First, as an embodiment of the present invention, an application program (“Annotation A
P ").

[0015]

Embodiment 1 FIG. 1 is a diagram showing a flow of processing of an annotation information addition display method according to an embodiment of the present invention (see claims 1 and 11). Referring to FIG. 1, when the annotation AP is activated, in step 1,
Search for the application program that is to be annotated. At this time, if the target application program has not been started, it is started.

In this step 1, for example, a process as shown in FIG. 4 may be performed. Referring to FIG. 4, in step 13, as shown in FIG. 5, a list of the currently activated existing application programs is presented to the user, and in step 14, the user performs annotation from this list. By selecting the desired application program, it is also possible to determine the target application program for performing the annotation.

Referring again to FIG. 1, in step 2,
The position on the display screen of the application program that is the target of the annotation (for example, x,
y-coordinate), and the size of the window.
A window display image is converted to a bitmap (bitma).
p) Obtain as data.

In the next step 3, the annotation AP
Is changed to be the same as the position and size of the window of the target application program acquired in step 2 above. Further, the bitmap data obtained in step 2 is pasted and displayed on the address of the annotation AP.

In this state, the annotation AP is displayed in the same size and the same display image on the window of the application program for the annotation.

In the next step 4, the annotation AP
Is hidden so that the window of the existing application program cannot be seen even if the window of the annotation AP is moved.

Thereafter, in step 5, the annotation AP enters an event reception waiting state, and repeats the processing until the received event is the end of the annotation.

If the event received in the event reception waiting state in step 5 is the annotation information (Yes in step 6), an annotation process is performed in step 7. The annotation information includes the type and coordinates of the annotation. In step 7, based on the annotation information, the annotation information is displayed on the window of the annotation AP.

In this case, if the annotation type is handwriting, a line segment is drawn from the coordinates of the previously received annotation information to the newly received coordinates.

If the annotation type is pointer, the previously drawn pointer is deleted and the pointer is drawn at the received coordinates.

Further, in the window of the annotation AP, an image which is not normally drawn, such as a scroll bar, a button, or a menu, may be displayed. In such a region, drawing is not performed, and the process shown in FIG. 6 is performed in step 7.

That is, referring to FIG.
Then, it is determined whether or not the coordinate information of the annotation information is within the drawable area in the window of the annotation AP. If the coordinate information is within the drawable area, the annotation information is drawn in step 16 and the coordinate information is outside the drawable area. If it does, it ignores the received annotation information and does nothing.

Referring to FIG. 1 again, if the event received in step 5 is the end of the annotation, the display position of the window of the annotation AP is acquired in step 9 and the existing Is displayed below the window of the annotation AP on the basis of the coordinates acquired in step 9 and the annotation AP is terminated, thereby ending the annotation on the existing application program.

As a result, the annotation information addition display method according to the present invention (see claim 1 or claim 11) is realized.

[0029]

Embodiment 2 Next, FIGS. 2 and 3 are diagrams showing the flow of processing of the annotation information addition display system according to an embodiment of the present invention (refer to claim 2 or 12). Note that FIG.
3 and FIG. 3 are merely separated for the sake of drawing convenience. Referring to FIGS. 2 and 3, in this embodiment, in the event processing loop after step 5 in FIG. 1, a step 11 for determining whether the received event is a control command is newly added, If it is determined that the command is a control command, in step 12, the control command accompanying the annotation information can be processed.

In this embodiment, the control command includes saving, restoring, and deleting annotation information.

In this embodiment, in step 12 (see FIG. 3) for processing a control command, preferably, the processing shown in FIG. 7 is performed.

Referring to FIG. 7, first, at step 17, it is checked whether or not the received control command is storage of annotation data. If the data is to be saved (Yes in step 17), in step 18, the annotation data currently displayed on the annotation AP is saved in a file. The data to be saved is the arrangement of the coordinates of the annotation information received so far.
The name of the file to be saved is made to correspond to the display image of the existing application that performs the annotation. For example, when annotation is performed on a WWW (World Wide Web) browser and the annotation data is saved, the saved file name is the URL displayed by the WWW browser at that time.
(Uniform Resource Locator) is used as the file name. After saving the data, the process returns to the event reception waiting state.

On the other hand, in the determination process of step 17,
If the control command is not data storage, in step 19, it is confirmed whether or not data recovery is performed.

If the control command is data recovery, it is further checked in step 20 whether there is any saved file that can be recovered. If there is a saved file, the annotation information is drawn in step 21 based on the saved data, and the process returns to the event reception waiting state.

In step 20, if there is no stored file, the process returns to the event reception waiting state without doing anything.

If it is determined in step 19 that the control command is not data recovery, it is confirmed in step 22 whether data is to be deleted. If the control command is data deletion, it is checked in step 23 whether there is any stored file that can be deleted, and if there is a stored file that can be deleted, the file is deleted and the process returns to the event reception waiting state.

In the determination process of step 23, if there is no saved file, the process returns to the event reception waiting state without any action.

If the control command is not data deletion (No in step 22), the process returns to the event reception waiting state without doing anything.

Thus, the annotation information addition display method according to the present invention (claim 2 or claim 12) is realized.

[0040]

Embodiment 3 Next, as an embodiment of the present invention (refer to claim 8 or 18), an embodiment of an application program for realizing a display method with annotation information and a presentation system using the application program will be described. explain.

FIGS. 8 and 9 are flowcharts showing the processing flow for realizing the annotation information additional display method. In step 5 of FIGS. 1 and 2, the processing steps shown in FIG. In step 7 of 3, the processing steps shown in FIG. 9 are performed.

In the flowcharts shown in FIGS. 1 and 2, an event is received in step 5. In this embodiment, referring to FIG. 8, in step 25, the received event is transmitted via the network. If the event is a local received event, the process proceeds to the next step without doing anything. If the event is a received event from the network, the received event is replaced with the locally generated event in step 26. Emulate and go to the next step.

Further, in step 7 of FIGS. 1 and 3, annotation processing is performed. In this embodiment, with reference to FIG. 9, annotation information is drawn in step 27, and in step 28, Send the annotation information to another application program via the network.

Accordingly, the present invention (claim 8 or claim 1)
The annotation information additional display method according to 8) is realized.

[0045]

Embodiment 4 FIG. 10 shows the present invention (claim 9 or 1).
It is a block diagram showing an example of composition of an embodiment of a presentation system concerning 9).

In FIG. 10, 200, 300,...
Reference numeral 00 denotes a terminal such as a personal computer having an input device such as a keyboard and a mouse. These terminals are connected to networks 110, 120, and 1N0, respectively.
Is connected to the server 100 via the. On each terminal, the annotation AP20 that implements the annotation information additional display method according to the third embodiment of the present invention is provided.
, N01 and existing application programs 202, 302,..., N02 are activated.

Referring to FIG. 10, server 100 has data transmitting / receiving means 101 and data broadcasting means 102,
The data transmitting / receiving means 101 transmits / receives data to / from the annotation AP on each terminal, and the data broadcasting means 102
Has a function of transmitting data received from a certain terminal to another terminal (a plurality of terminals) connected to the server 100.

For example, when the existing application program and the annotation AP are respectively activated on the terminal, the window of the annotation AP having the window image of the existing application program is displayed on the screen of each terminal as shown in FIG.

FIG. 12 shows terminals 200, 300, 400
Is connected to the server 100.
In this state, for example, if the user of the terminal 200 performs handwriting input in the annotation AP 201, the handwriting is drawn on the window of the annotation AP 201 and transmitted to the server 100 via the network 110.

The handwritten data received by server 100 is transmitted to terminals 300 and 40 determined by data broadcasting means 102.
The data is transmitted by the data transmitting / receiving means 101 to each of the 0 annotation APs.

The annotation APs 301 and 4 of the terminals 300 and 400 that have received the handwritten data from the server 100
01 is the annotation AP based on the received data
Draw handwriting on the window of.

Thus, each terminal 200, 300, 40
0 users can share handwritten information displayed on the annotation AP with each other.

The pointer is also displayed on each terminal in the same manner as handwriting, and can be shared. In other words, on the screen of the existing application, the user can convey the information to other users by, for example, enclosing important parts by handwriting, underlining, or pointing with a pointer, A presentation using such annotation information can be performed.

[0054]

Fifth Embodiment FIG. 11 shows the present invention (claim 10 or 2).
FIG. 2 is a diagram showing a configuration of a server 100 for realizing a presentation system according to the embodiment 0).
Referring to FIG. 11, a server 100 includes a data transmitting / receiving unit 101, a data broadcasting unit 102, a data analyzing unit 10
3 and data storage means 104.

The data analysis means 103 examines the contents of the received data, and copies and stores the received data in the data storage means 104, for example, annotation data and user data. If the data is control data and the request is to restore the stored data, the data stored in the data storage unit 104 is extracted, and the data is transmitted to the connected terminals by the data broadcasting unit 102.

FIG. 13 shows an embodiment of the present invention.
FIG. 4 is a diagram illustrating an example in which handwritten data is stored in a server.

Referring to FIG. 13, for example, when the user of the terminal 200 performs handwriting, the handwritten data is transmitted to the server 100 via the network. Server 100
Then, the received data is analyzed by the data analysis means 103. In this case, since the data is handwritten, the data is stored in the data storage means 104, and the data is broadcast to the connected terminal 300.

Further, thereafter, as shown in FIG. 14, when the terminal 400 is connected to the server 100,
The above annotation AP transmits control data of “participation midway” to the server 100,
Transmits the annotation data stored in the data storage unit 104 to the terminal 400, and stores the annotation AP4
01 draws the received annotation data, so that the annotation information before the terminal 400 participates can be obtained.

[0059]

As described above, according to the present invention,
When it is desired to perform an annotation on an existing application program that does not have the annotation function, there is an effect that the existing application program can be realized without modification. For this reason, according to the present invention, it is possible to easily perform a presentation using the annotation by using the application program used by the user.

The reason is that, in the present invention, the screen information of the existing application program is acquired, the window of the annotation AP for drawing the annotation is displayed according to the acquired screen information, and the window of the existing application program is hidden. , The annotation AP serves as a display image of the existing application program, processes the input on the display image, and draws the annotation on the display image if the input is the annotation information. It depends.

Further, according to the present invention, by sharing the annotation information with other users via the network, there is an effect that communication using the annotation on an existing application program becomes possible.

[Brief description of the drawings]

FIG. 1 is a diagram for describing an embodiment of the present invention, and is a diagram illustrating an example of a processing flow of an annotation information addition display method for an existing application program.

FIG. 2 is a diagram for describing an embodiment of the present invention, and is a diagram illustrating an example of a processing flow of a display method in which an annotation information is added to an existing application program;

FIG. 3 is a diagram for explaining the embodiment of the present invention, and is a diagram illustrating an example of a processing flow of a display method in which an annotation information is added to an existing application program;

FIG. 4 is a diagram for describing the embodiment of the present invention, and is a diagram illustrating an example of a processing flow of a method of adding and displaying annotation information to an existing application program.

FIG. 5 is a diagram illustrating a specific example of the embodiment of the present invention, and is a diagram illustrating an example of a list display for selecting an existing application program to be annotated.

FIG. 6 is a diagram for explaining the embodiment of the present invention, and is a diagram illustrating an example of a flow of an annotation process of an annotation information adding display method to an existing application program;

FIG. 7 is a diagram for describing the embodiment of the present invention, and is a diagram illustrating an example of a flow of a control command process of a display method with an annotation information added to an existing application program.

FIG. 8 is a diagram for describing the embodiment of the present invention, and is a diagram illustrating an example of a flow of an event receiving process of an annotation information adding display method to an existing application program.

FIG. 9 is a diagram for describing the embodiment of the present invention, and is a diagram illustrating an example of a flow of an annotation process of an annotation information addition display method for an existing application program.

FIG. 10 is a diagram illustrating a configuration of an embodiment of the present invention, and is a diagram illustrating an example of a configuration of a presentation system using an existing application program.

FIG. 11 is a diagram illustrating an example of a configuration of a server according to the embodiment of the present invention.

FIG. 12 is a diagram for describing a configuration of an embodiment of the present invention, and is a diagram illustrating an application example of a presentation system using an existing application program.

FIG. 13 is a diagram illustrating a configuration of an embodiment of the present invention, and is a diagram illustrating an application example of a presentation system using an existing application program.

FIG. 14 is a diagram illustrating a configuration of an embodiment of the present invention, and is a diagram illustrating an application example of a presentation system using an existing application program.

[Explanation of symbols]

Reference Signs List 100 server 101 data transmission / reception function 102 data broadcasting function 103 data analysis function 104 data storage function 120, 130, 140, 1N0 network 200, 300, 400, N00 terminal 201, 301, 401, N01 Annotation information additional display method of the present invention Application program (annotation AP) 202, 302, N02 that implements the existing application program for performing the annotation

Claims (22)

[Claims] An information processing apparatus acquires screen information of an existing application program, displays a window of an application program (referred to as "annotation AP") for drawing an annotation in accordance with the acquired screen information, Display, input on the screen of the existing application program is processed by the annotation AP, and if the input is annotation information, the annotation information is drawn on the screen information of the existing application program. An annotation information addition display method characterized by the following. 2. The annotation information addition display method according to claim 1, further comprising means for receiving a control command attached to the annotation and processing the received control command. 3. The annotation information addition display method according to claim 1, wherein the annotation information to be displayed includes handwritten information. 4. The annotation information addition display method according to claim 1, wherein the annotation information to be displayed includes pointer information. 5. The apparatus according to claim 1, further comprising: means for selecting an application program designated by a user from application programs running the existing application program to be annotated. Annotation information addition display method. 6. The annotation information addition display method according to claim 1, wherein the annotation information is displayed only in a drawable area of the acquired screen information. 7. The apparatus according to claim 1, further comprising means for storing the displayed annotation information in association with the screen information, and for displaying and / or deleting the stored annotation information. Item 2. An annotation information addition display method according to item 2. 8. An application program connected to another application program via a network, emulating an event received from the network into a local event, and processing the event. If the local event is annotation information, The annotation information addition display method according to any one of claims 1 to 7, further comprising: means for drawing an image and transmitting the annotation information to another application program via a network. 9. An application program for realizing the annotation information additional display system according to claim 8 transmits and receives annotation information via a server.
A presentation system, wherein annotation information can be shared among a plurality of terminals. 10. The server analyzes data from the terminal and stores the information if the information can be stored in a storage unit. If the server is a control command for acquiring the data stored by the data from the terminal, the server analyzes the data. 10. The presentation system according to claim 9, further comprising means for broadcasting data stored in said storage means to each terminal. (11) a step of searching for an existing application program to be annotated, (b) a step of obtaining screen information of the searched application program, and (c) drawing an annotation based on the obtained screen information. Application program ("Annotation A
P); (d) hiding the window of the existing application program; (e) receiving an event and determining the received event; (f) A processing step according to the received event; (g) an annotation processing step of drawing annotation information on an annotation AP when the received event is annotation information; In this case, the method includes the steps of: acquiring window information of the annotation AP; and (i) displaying a window of the existing application program that has been hidden based on the window information. By the existing app Returning the application program to its original state. 12. The annotation information addition display method according to claim 11, further comprising a processing step corresponding to the control command when the received event is a control command. 13. The annotation processing step, wherein when the received event is coordinate information, handwritten information is displayed on an application capable of annotation based on the coordinate information. Annotation information addition display method described in. 14. The annotation processing step according to claim 11, wherein the received event is coordinate information, and pointer information is displayed on an application capable of annotation based on the coordinate information. How to add and display annotation information. 15. A step (a) of searching for an existing application program for which the annotation is to be performed.
(A-1) presenting a list of application programs running at that time to the user; and (a-2) selecting a desired application program from the list by the user. The annotation information adding and displaying method according to claim 11 or 12, wherein: 16. The annotation processing step (g).
The method according to claim 11, further comprising: (g-1) a step of determining a drawable area of the acquired screen information; and (g-2) a step of drawing annotation information when drawing is possible. Or the annotation information additional display method described in 12. 17. The control command includes data saving, restoring, and deleting, the control command processing step includes a step of determining the command, and a processing step according to the determined command. If the command is a data save, the annotation information displayed at that time is stored in association with the display screen information, and if the command is a data return, the stored data is based on the stored data, if any. 13. The annotation information adding display method according to claim 12, wherein the annotation information is displayed, and when the command is data deletion, the stored data is deleted if there is any stored data. 18. The event receiving step includes: determining whether a received event has been received from a network; and in the case of receiving from a network, emulating the received event into a local event. 18. The annotation information adding and displaying method according to claim 11, wherein the annotation processing step includes a step of drawing the annotation information and a step of transmitting the annotation information to a network. 19. A server having a data transmission / reception means and a data broadcasting means, a terminal on which an existing application program and the annotation AP for realizing the annotation information addition display method according to claim 18 are respectively activated, Wherein the plurality of terminals are connected to the server via a network, and annotation information processed on screen information of an existing application program is transmitted to the server via the network by an annotation AP of a certain terminal, In the server, the annotation information received by the data transmitting / receiving means is transmitted by the data broadcasting means to all the other terminals connected to the server, and the annotation information received by each terminal is an annotation AP. Processed in The annotation information is displayed on the screen information of the existing application program in the same manner as the terminal that transmitted the annotation information, and the user of the plurality of terminals performs the work using the annotation information using the existing application program , A presentation system characterized by being able to do. 20. The server, comprising: data analysis means for analyzing data received from the data transmission / reception means; and data storage means for storing data, wherein the data analyzed by the data analysis means can be stored. If it is information, it is stored in the data storage means, and if the analyzed data is a control command for obtaining the stored data, the corresponding data is obtained from the data storage means, and the data broadcasting means and the data 20. The presentation system according to claim 19, wherein data is transmitted to each terminal via data transmission / reception means. 21. Screen information of a first application program having no annotation function is obtained, a second application program having an annotation drawing function is activated and displayed on a screen, and the second application program is displayed on a screen. In the application program screen,
Annotation information is input to the application program, and the annotation information input via the second application program is transmitted to the first application and displayed on the screen of the first application. Annotation information additional display method. 22. The annotation information addition display method according to claim 1, wherein said existing application program is an application program having no annotation function.