JP2010074639A - Information processing apparatus and system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing apparatus and an information processing system in which a user at an information processing apparatus side is enabled to grasp information about a user, who wrote an annotation, or about an external terminal. <P>SOLUTION: When an annotation is written into an image photographed by a first camera 5, a CPU 201 of a client 2 displays on a monitor 20 the written annotation and ID information of the client 2 (information about a user who wrote the annotation or about an external terminal) and transmits to a server 1 the written annotation and the ID information of the client 2. A CPU 101 of the server 1 receives the written annotation and the ID information of the client 2 from the client 2 and projects the received annotation and ID information on the client 2 on a projector 7. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an information processing apparatus and an information processing system.

  Conventionally, a remote instruction system including a server (for example, a computer) to which a camera and a projector are connected and a remote client (for example, a computer) connected to the server via a network is known (for example, (See Patent Documents 1 and 2).

In this remote instruction system, a camera connected to a server captures an object and transmits a captured image to a client via a network. The client displays the captured image on the display screen of the display device, and when an annotation or the like is written on the display screen, the client transmits the annotation to the server via the network. The server transmits the received annotation to the projector, and the projector projects it to an appropriate position of the projection area including the received annotation (this corresponds to the position where the client wrote the annotation within the range of the captured image).
US Patent Publication No. 2004/0070674 JP 2005-33756 A

  An object of the present invention is to provide an information processing apparatus and an information processing system in which a user on the information processing apparatus side can grasp information related to a user who has written an annotation or an external terminal.

  In order to achieve the above object, the information processing apparatus according to claim 1 includes: a receiving unit that receives an annotation from an external terminal and a user who has written the annotation or information related to the external terminal; an annotation received by the receiving unit; Projecting means for projecting information related to the user who wrote the annotation or the external terminal onto a projection area including an object, photographing means for photographing the projection area, and an image photographed by the photographing means to the external terminal Transmitting means for transmitting.

  The information processing apparatus according to claim 2 is the information processing apparatus according to claim 1, wherein the projection unit projects the information about the user who wrote the annotation or the information regarding the external terminal within a preset time. And

  The information processing system according to claim 3 is connected to the information processing device that includes the projection device and the object, and that can be connected to an imaging device that images the projection area of the projection device, and the information processing device and the first display device. The external terminal includes a writing unit that writes an annotation to an image captured by the imaging device, a comment that is written by the writing unit, and a user that writes the annotation Alternatively, a first control unit that displays information on the external terminal on the first display device, and an annotation written by the writing unit and a user who wrote the annotation or information on the external terminal are transmitted to the information processing device. A first transmission / reception means, wherein the information processing apparatus is configured to receive the written annotation and the user who wrote the annotation from the external terminal or the external A second transmission / reception unit configured to receive information about a terminal; and a second control unit configured to cause the projection apparatus to project information about the received annotation and a user who wrote the received annotation or the external terminal. And

  The information processing system according to claim 4 is the information processing system according to claim 3, wherein the first display device displays a first display area for displaying an image photographed by the photographing device and information other than the image. The first display means displays the annotation written by the writing means on the first display area, and displays information about the user who wrote the annotation or the external terminal in the first display area. Two projection areas are displayed, and the projection apparatus projects the received annotation and information related to the user who has written the received annotation or the external terminal onto the projection area.

  The information processing system according to claim 5 is the information processing system according to claim 3 or 4, wherein the information processing device is an annotation directly written on the object based on images continuously photographed by the photographing device. Or a detecting means for detecting the movement of the object, wherein the second transmitting / receiving means uses the photographing apparatus when the detecting means detects an annotation written directly on the object or a movement of the object. The continuously captured image and information related to the information processing device are transmitted to the external terminal, and the first transmitting / receiving unit receives the image continuously captured by the capturing device and information related to the information processing device from the information processing device. The first control means displays on the first display device the images continuously taken by the photographing device received by the receiving means and information on the information processing device. And characterized in that.

  An information processing system according to a sixth aspect is the information processing system according to any one of the third to fifth aspects, wherein a plurality of external terminals are connected to the information processing apparatus. Each includes the writing means, the first control means, and the first transmission / reception means, and the first transmission / reception means writes the annotation written by the writing means of another external terminal and the annotation. When receiving information related to the user or the other external terminal, the first control means displays the received annotation and the information related to the user who wrote the received annotation or the other external terminal in the first display. It is displayed on a device.

  The information processing system according to claim 7 is the information processing system according to claim 6, wherein the first display device includes a virtual image of the user on the information processing device side, information on the information processing device, and other external terminals. The user's virtual image and information on the other external terminal are displayed.

  The information processing system according to claim 8 is the information processing system according to claim 6 or 7, wherein a second display device is connected to the information processing device, and the information processing device includes the information processing system. A virtual image of a user of each external terminal connected to the apparatus and information regarding each external terminal are displayed.

  According to the first aspect of the invention, the user on the information processing apparatus side can grasp information related to the user who wrote the annotation or the external terminal.

  According to the second aspect of the present invention, it is possible to display only the annotation after a predetermined time has elapsed after projecting the annotation and information on the user or the external terminal to which the annotation has been written.

According to the third aspect of the present invention, the user on the information processing apparatus side and the user on the external terminal side can grasp the information related to the user who wrote the annotation or the external terminal.

  According to invention of Claim 4, the positional relationship of the display position on the 1st display device of the information regarding the user or external terminal which wrote the annotation and the projection position of the information regarding the user or external terminal which wrote the annotation is shifted. Can do.

  According to the invention of claim 5, the user on the external terminal side can grasp that the annotation is directly written on the object or the object is moved by the user on the information processing apparatus side.

  According to the invention of claim 6, the user on the external terminal side can grasp the annotation written by the user of the other external terminal and the information related to the user who wrote the annotation or the other external terminal.

  According to the seventh aspect of the present invention, compared to the case where the virtual image of the user on the information processing device side and the information related to the information processing device and the virtual image of the user of other external terminal and the information related to the other external terminal are not displayed, Communication with feeling is possible.

  According to the eighth aspect of the present invention, realistic communication is possible compared to a case where a virtual image of a user of each external terminal connected to the information processing apparatus and information regarding each external terminal are not displayed.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention.

  The information processing system in FIG. 1 includes a server 1 (information processing apparatus) configured by a computer and clients 2 and 3, which are connected to each other via a network 4. The server 1 is connected to a first camera 5 (imaging device, imaging means), a second camera 9, a projector 7 (projection device, projection means), and a monitor 10 (second display device).

  Based on the control command from the server 1, the projector 7 irradiates the object 8 with a simple light beam via the half mirror 6 or projects an annotation (annotation) input from the clients 2 and 3. Note that the annotation includes a line, a character, a symbol, a figure, a color, a font (typeface), and the like.

  The first camera 5 is composed of a video camera, captures a reflected image of the object 8 through the half mirror 6, and outputs the captured image to the server 1. The optical axis and field angle of the first camera 5 coincide with the optical axis and field angle of the projector 7. The first camera 5 captures the object 8 as a still image periodically (for example, at intervals of 0.1 seconds). The second camera 9 photographs a place where the object 8 is placed and a user around the object 8.

  The server 1 displays the captured image captured by the first camera 5 on the monitor 10 and outputs it to the clients 2 and 3 via the network 4. Monitors 20 and 30 are respectively connected to the clients 2 and 3, and the monitors 20 and 30 (first display device) display a captured image display area 15 and a user interface (UI) 14. Each display area 15 displays a captured image captured by the first camera 5. The clients 2 and 3 may be constituted by an integrated computer including the monitors 20 and 30.

  The UI 14 includes a group of buttons such as pen, text, and erase, and icons of line types and color types. In the figure, a captured image obtained by photographing the object 8 with the first camera 5 is displayed in the display area 15. For example, when the pen button on the UI 14 of the monitor 20 is pressed and an annotation is described on the object 8 in the display area 15, the annotation information (specifically, coordinate data) is transferred from the client 2 to the server 1. Is output. The server 1 decodes the annotation information (specifically, coordinate data), converts it into a projection image for the projector 7, and outputs it to the projector 7. The projector 7 projects the projection image on the object 8.

  Further, the client 2 and the client 3 output a control command to the server 1, and the operations of the projector 7 and the first camera 5 (for example, the shooting angle of the first camera 5 and the brightness of the shot image of the first camera 5). And the brightness of the projection image of the projector 7 can be controlled. The cameras 11 and 12 are connected to the clients 2 and 3, respectively, and take pictures of the users of the clients 2 and 3.

  In the figure, although the clients 2 and 3 are described as clients, the information processing system may include three or more clients (clients).

  FIG. 2 is a block diagram showing the hardware configuration of the server 1 and the clients 2 and 3. Since the hardware configuration of the server 1 is the same as that of the clients 2 and 3, the hardware configuration of the server 1 will be described here. In FIG. 2, reference numerals 201 to 209 indicate constituent elements of the client 2, and reference numerals 301 to 309 indicate constituent elements of the client 3.

  The server 1 includes a CPU 101 for controlling the entire apparatus, a ROM 102 having a control program, a RAM 103 functioning as a working area, a hard disk drive (HDD) 104 having various information and programs, a key input terminal 105 to which a keyboard and a mouse are connected, A network interface 106 for connecting to another computer, a video interface 107 for connecting a monitor, and a USB (universal serial bus) interface 108 for connecting to a USB device (not shown) are provided. The CPU 101 is connected to a ROM 102, a RAM 103, a hard disk drive (HDD) 104, a key input terminal 105, a network interface 106, a video interface 107, and a USB interface 108 via a system bus 109.

  The HDD 104 of the server 1 authenticates a location ID indicating the location of the server 1 (for example, a room number, a meeting room name, etc.), a terminal ID for identifying the server 1, and a user ID of a user accessing from the client 2 or 3. It has. The HDD 204 includes a location ID indicating the location of the client 2 (for example, a room number, a conference room name, etc.) and a terminal ID for identifying the client 2. The HDD 304 includes a location ID indicating the location of the client 3 (for example, a room number, a conference room name, etc.) and a terminal ID for identifying the client 3.

  The key input terminal 205 constitutes writing means, and the CPU 201 constitutes first control means and first transmission / reception means. The CPU 101 constitutes second transmission / reception means, second control means, and detection means.

  FIG. 3 is a flowchart showing processing executed by the information processing system when annotation is written by the client 2.

  First, the CPU 201 of the client 2 acquires the user ID input from the key input terminal 205 (step S1), and determines whether or not the annotation is written in the display area 15 of the monitor 20 (step S2). Note that when the annotation coordinate data is input from the key input terminal 205, the CPU 201 determines that the annotation has been written in the display area 15 of the monitor 20.

  If NO in step S2, this process ends. On the other hand, if YES in step S2, the CPU 201 transmits the coordinate data of the annotation and the ID information of the client 2 to the server 1 (step S3). The ID information of the client 2 includes the location ID, the terminal ID of the client 2, and the user ID in step S1.

  Next, the CPU 101 of the server 1 determines whether or not the annotation coordinate data and the client 2 ID information are received from the client 2 (step S4). If NO in step S4, the determination is repeated. On the other hand, if YES in step S4, the CPU 101 transmits the annotation coordinate data and the ID information of the client 2 to the projector 7, and causes the projector 7 to project the annotation and the ID information of the client 2 (step S5).

  FIG. 4 is a diagram illustrating an example of the annotation projected on the object 8 and the ID information of the client 2.

  The CPU 101 presets the display position of the ID information of the client 2 in the HDD 104 of the server 1 in response to an input from the key input terminal 105 or an input from the clients 2 and 3. For example, the display position of the ID information of the client 2 may be designated by four preset coordinates. In this case, the ID information of the client 2 is displayed in an area surrounded by four preset coordinates. Further, for example, the coordinates of the display position of the ID information of the client 2 (for example, (x1 + 500, y1 + 50)) may be specified on the basis of the coordinates (x1, y1) of the upper right edge of the annotation. The display position of ID information 2 is set within a range that can be projected by the projector 7.

  Returning to FIG. 3, the CPU 101 determines whether or not a client other than the client 2 (for example, the client 3) is connected to the server 1 (step S6). If NO in step S6, this process ends. On the other hand, if YES in step S6, the CPU 101 transmits the annotation coordinate data of the client 2 and the ID information of the client 2 to the client 3 (step S7).

  Next, the CPU 301 of the client 3 determines whether or not the coordinate data and ID information of another client's annotation (that is, the coordinate data and ID information of the annotation of the client 2) has been received (step S8). If NO in step S8, this determination is repeated. On the other hand, if YES in step S8, the CPU 301 displays the annotation and ID information of the client 2 on the monitor 30 (step S9), and ends this process.

  5A and 5B are diagrams showing examples of annotations displayed on the monitor 30 and ID information of the client 2. FIG.

  5A and 5B, a broken line 16 indicates a display area of a captured image from the first camera 5. The display area 16 constitutes a part of the display area 15 that can display various data and information.

  The CPU 301 sets the display position of the ID information in the HDD 304 of the client 3 in advance in response to an input from the key input terminal 305 or an input from the client 2 or the server 1. For example, the display position of the ID information may be designated by four preset coordinates. In this case, the ID information is displayed in an area surrounded by four preset coordinates. For example, the coordinates of the upper right end of the display position of the ID information (for example, (x1 + 500, y1 + 50)) may be specified based on the coordinates (x1, y1) of the upper right end of the annotation.

  Further, as shown in FIG. 5A, the display position of the ID information may be the same as the display position of the ID information specified by another client (for example, the client 2). Further, the display position of the ID information may be different from the display position of the ID information specified by another client (for example, the client 2) or the projection position of the ID information of the projector 7.

  Further, as shown in FIG. 5B, the display position of the ID information may be a portion that does not overlap the display area 16 of the captured image in the display area 15. Thereby, for example, even if the annotation and the ID information are displayed overlappingly on the monitor 20, the annotation and the ID information are separated on the monitor 30, and the entire annotation is displayed.

  The display timing of the ID information is when the annotation is written in the display area 15, when the annotation is edited, or when the annotation is deleted. The timing for deleting the display of the ID information is when the next annotation is written in the display area 15 or when a certain time (for example, 10 seconds) has passed since the annotation was written.

  FIG. 6 is a diagram illustrating a display state of the monitors 10 and 20 when annotations are written in the display area 15 of the monitor 20.

  When the clients 2 and 3 are connected to the server 1, the CPU 201 of the client 2 receives information indicating that the client 3 is connected to the server 1 and the ID information of the client 3 from the server 1, The avatar 19 indicating the user of the client 3 and the ID information of the client 3 are displayed in the display area 15 of the monitor 20. An avatar is a virtual image showing people and things on the network. Note that the display position of the ID information of the clients 2 and 3 is set in advance by the CPU 201 of the client 2.

  Further, the CPU 201 causes the display area 15 to display the avatar 18 a indicating the user's hand of the client 2 and the ID information of the client 2 when the annotation is written in the display area 15 of the monitor 20. The avatar 18 a and the avatar 19 are associated with the ID information of the client 2 and stored in the HDD 204 in advance.

  At this time, the CPU 301 of the client 3 receives information indicating that the client 2 is connected to the server 1 and the ID information of the client 2 from the server 1, and the client 3 user's hand is displayed in the display area 15 of the monitor 30. , The avatar 18 indicating the user of the client 2 and the ID information of the client 2 are displayed. These avatars are associated with the ID information of the client 3 and stored in the HDD 304 in advance.

  Further, when the clients 2 and 3 are connected to the server 1 and the annotation is written in the display area 15 of the monitor 20, the CPU 101 of the server 1 performs the avatar 18, the avatar 19, and the clients 2 and 3. ID information is displayed on the monitor 10. At this time, the avatar 18 is displayed as an animation indicating the operation of writing the annotation. The projector 7 projects the avatar 18a in addition to the annotation and the ID information of the client 2. The images of the avatar 18, the avatar 18 a, and the avatar 19 are stored in advance in the HDD 104 in association with the ID information of the clients 2 and 3. When the CPU 101 receives the coordinate data of the annotation and the ID information of the clients 2 and 3 from the client 2, the CPU 101 reads the images of the avatar 18, the avatar 18 a and the avatar 19 from the HDD 104. The ID information is displayed on the monitor 10, and the annotation, the ID information of the client 2 and the avatar 18 a are projected on the projector 7.

  In addition to the avatar 18 and the avatar 19, the CPU 101 of the server 1 displays captured images of the cameras 11 and 12 connected to the clients 2 and 3 (for example, face images of the users of the clients 2 and 3) on the monitor 10. You may let them. In addition to the avatar 19, the CPU 201 of the client 2 may display a captured image of the camera 12 (for example, a face image of the user of the client 3) in the display area 15 of the monitor 20. In addition to the avatar 18, the CPU 301 of the client 3 may display a captured image of the camera 11 (for example, a face image of the user of the client 2) in the display area 15 of the monitor 30.

  Further, as shown in FIG. 7, the CPU 201 of the client 2 may display the ID information of the client 2 in a portion that does not overlap the display area 16 of the captured image in the display area 15 of the monitor 20. In this case, the positional relationship between the display position of the ID information of the client 2 on the monitor 20 and the projection position of the ID information of the client 2 is shifted. That is, the ID information of the client 2 is displayed on the monitor 20 at a position that does not overlap the object 8, but the ID information of the client 2 is projected on the object 8 on the object 8 side.

  Similarly, the CPU 101 of the server 1 may display the ID information of the client 2 and the ID information of the client 3 in a portion that does not overlap the display area of the captured image in the display area of the monitor 10.

  FIG. 8 is a flowchart showing a process for displaying a change generated in the object 8 on a monitor connected to each client.

  First, the CPU 101 of the server 1 detects a difference between a plurality of captured images (that is, frames) continuously captured by the first camera 5, and determines whether or not a change has occurred between the plurality of captured images (step). S11). Here, the change between a plurality of captured images means that an annotation is directly written with the pen on or around the object 8 or the object 8 is moved, and is written by each client. Changes between a plurality of captured images due to the projection of annotations and ID information are not included. For this reason, the CPU 101 of the server 1 ignores changes between a plurality of captured images within a certain time after the annotation or ID information written by each client is projected, or the annotation or Changes between a plurality of captured images corresponding to the area where the ID information is projected are ignored.

  If NO in step S11, this process ends. On the other hand, if YES in step S11, the CPU 101 of the server 1 transmits a message indicating that a change has occurred between the captured image and the captured image to all clients (step S12).

  An example of a message transmitted from the server 1 to all clients is shown in FIG. As shown in FIG. 9, the message transmitted from the server 1 to all clients includes the type of change, the ID information of the server 1, and change position specifying data. The ID information of the server 1 includes the location ID, the terminal ID of the server 1, and the user ID of the user authenticated by the server 1. The type of change includes “writing” indicating that an annotation is directly written with a pen on or around the object 8 or “moving” indicating that the object 8 has been moved.

  The change position specifying data is specified by the coordinates of the four vertices of the area where the change occurs or the barycentric coordinates and area of the area where the change occurs. The reason why the message includes the change position specifying data is to display the ID information of the server 1 near the area where the change occurs.

  Note that the above message is transmitted at a constant interval (for example, 1 second) while a change between a plurality of captured images occurs, or a timing at which detection of a change between a plurality of captured images is started and a plurality of captured images. It is transmitted at the timing when the detection of the change in between ends.

  Next, it is determined whether the CPU of each client has received a captured image and a message (step S13). If NO in step S13, the determination is repeated. On the other hand, if YES in step S13, the CPU of each client displays the captured image, the avatar of another client and its ID information, the user's avatar on the object 8 side, and the ID information of the server 1 on the corresponding monitor. Is displayed (step S14), and this process is terminated.

  The server 1 ID information displayed in step S14 may be at least one of the location ID, the server 1 terminal ID, and the user ID of the user authenticated by the server 1.

  FIG. 10 is a diagram illustrating a display state of the monitors 10 and 20 when annotations are directly written on the object 8.

  The CPU 201 of the client 2 receives information indicating that the client 3 is connected to the server 1 from the server 1, ID information of the server 1, and ID information of the client 3, and displays the client 3 's ID in the display area 15 of the monitor 20. The avatar 19 indicating the user, the user avatar 17 on the object 8 side, the ID information of the server 1 and the ID information of the client 3 are displayed. At this time, the avatar 17 is displayed with an animation indicating the operation of writing the annotation. The avatar 17 and the avatar 19 are associated with the ID information of the client 2 and stored in the HDD 204 in advance. Note that the display positions of the ID information of the server 1 and the client 3 are set in advance by the CPU 201 of the client 2.

  Further, the CPU 201 of the client 2 may display the ID information of the server 1 around the annotation written to the user on the server 8 side based on the change position specifying data in FIG.

  The CPU 301 of the client 3 receives the information indicating that the client 2 is connected to the server 1 from the server 1, the ID information of the server 1, and the ID information of the client 2, and displays the avatar 17 in the display area 15 of the monitor 30. The avatar 18, the server 1 ID information, and the client 2 ID information are displayed. These avatars are associated with the ID information of the client 3 and stored in the HDD 304 in advance.

  As shown on the left side of FIG. 10, the monitor 10 displays ID information of the avatar 18, the avatar 19, and the clients 2 and 3. Note that the ID information of the clients 2 and 3 need not always be displayed.

  In addition to the avatar 18 and the avatar 19, the CPU 101 of the server 1 monitors the captured images of the cameras 11 and 12 connected to the clients 2 and 3 (for example, face images of the users of the clients 2 and 3) 10. May be displayed. In addition to the avatars 17 and 19, the CPU 201 of the client 2 adds a captured image of the second camera 9 (for example, a face image of the user of the server 1) and a captured image of the camera 12 (for example, the client) to the display area 15 of the monitor 20. 3 user face images) may be displayed. In addition to the avatars 17 and 18, the CPU 301 of the client 3 adds a captured image of the second camera 9 (for example, a face image of the user of the server 1) and a captured image of the camera 11 (for example, the client) to the display area 15 of the monitor 30. 2 user face images) may be displayed.

  Further, the CPU 201 of the client 2 may display the ID information of the server 1 and the ID information of the client 3 in a portion that does not overlap the display area of the captured image in the display area 15 of the monitor 20. Similarly, the CPU 101 of the server 1 may display the ID information of the client 2 and the ID information of the client 3 in a portion that does not overlap the display area of the captured image in the display area of the monitor 10.

  As described above in detail, according to the present embodiment, when an annotation is written in an image captured by the first camera 5, the CPU 201 displays the written annotation and the ID information of the client 2 on the monitor 20. The written annotation and the ID information of the client 2 are transmitted to the server 1. The CPU 101 of the server 1 receives the annotation written from the client 2 and the ID information of the client 2 and causes the projector 7 to project the received annotation and the ID information of the client 2.

  Therefore, the annotation written in the client 2 and the ID information of the client 2 are transmitted to the user on the server 1 side and the user on the client 2 side.

  Note that the present invention is not limited to the above-described embodiment, and can be implemented with various modifications without departing from the scope of the invention.

It is a block diagram which shows the structure of the information processing system which concerns on embodiment of this invention. 2 is a block diagram illustrating a hardware configuration of a server 1 and clients 2 and 3. FIG. 10 is a flowchart illustrating processing executed by the information processing system when an annotation is written by a client 2; It is a figure which shows the example of the annotation projected on the target object 8, and ID information of the client 2. FIG. (A), (B) is a figure which shows the example of the annotation displayed on the monitor 30, and ID information of the client 2. FIG. It is a figure which shows the display state of the monitors 10 and 20 when the annotation is written in the display area 15 of the monitor 20. It is a figure which shows the modification of the display state of the monitors 10 and 20 when the annotation is written in the display area 15 of the monitor 20. FIG. It is a flowchart which shows the process which displays the change which arose in the target object 8 on the monitor connected to each client. It is a figure which shows an example of the message transmitted to all the clients from the server. It is a figure which shows the display state of the monitors 10 and 20 in case the annotation is directly written in the target object 8. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Server 2, 3 Client 4 Network 5 1st camera 6 Half mirror 7 Projector 8 Object 9 2nd camera 11, 12 Camera 10, 20, 30 Monitor 101, 201, 301 CPU
102, 202, 302 ROM
103, 203, 303 RAM
104, 204, 304 HDD
105, 205, 305 Key input terminal 106, 206, 306 Network interface 107, 207, 307 Video interface 108, 208, 308 USB interface

Claims (8)

Receiving means for receiving an annotation from an external terminal and a user who wrote the annotation or information on the external terminal;
Projection means for projecting the annotation received by the reception means and the user who wrote the annotation or information on the external terminal onto a projection area including the object;
Photographing means for photographing the projection area;
Transmitting means for transmitting an image photographed by the photographing means to the external terminal;
An information processing apparatus comprising:   The information processing apparatus according to claim 1, wherein the projection unit projects information related to the user who wrote the annotation or the external terminal within a preset time. An information processing system that includes a projection device and an information processing device that can be connected to an imaging device that captures a projection area of the projection device, and an information processing system that includes an external terminal connected to the information processing device and the first display device. There,
The external terminal is
Writing means for writing an annotation on an image photographed by the photographing device;
First control means for causing the first display device to display the annotation written by the writing means and the user or the external terminal that wrote the annotation;
An annotation written by the writing means and a first transmission / reception means for transmitting information related to the user who wrote the annotation or the external terminal to the information processing apparatus,
The information processing apparatus includes:
Second transmission / reception means for receiving from the external terminal the written annotation and information about the user who wrote the annotation or the external terminal;
An information processing system comprising: a second control unit that causes the projection device to project information about the received annotation and a user who wrote the received annotation or the external terminal. The first display device includes a first display region that displays an image captured by the imaging device, and a second display region that displays information other than the image,
The first control means causes the annotation written by the writing means to be displayed in the first display area, and displays information related to the user who wrote the annotation or the external terminal in the second display area,
The information processing system according to claim 3, wherein the projection apparatus projects information about the received annotation and a user who has written the received annotation or the external terminal onto the projection area. The information processing apparatus includes a detection unit that detects an annotation directly written on the object or a movement of the object based on images continuously captured by the imaging apparatus,
The second transmission / reception unit displays, when the detection unit detects an annotation directly written on the object or a movement of the object, images taken continuously by the imaging device and information on the information processing device. Send it to an external device,
The first transmission / reception means receives images taken continuously by the photographing device and information on the information processing device from the information processing device
The first control unit causes the first display device to display an image continuously received by the imaging device received by the receiving unit and information on the information processing device. The information processing system described. A plurality of external terminals are connected to the information processing apparatus,
Each of the plurality of external terminals includes the writing unit, the first control unit, and the first transmitting / receiving unit.
When the first transmission / reception means receives the annotation written by the writing means of another external terminal and the information about the user who wrote the annotation or the other external terminal, the first control means receives the annotation. 6. The information processing system according to claim 3, wherein information related to the user who wrote the received annotation and the received annotation or the other external terminal is displayed on the first display device. .   The first display device displays a virtual image of the user on the information processing device side, information on the information processing device, a virtual image of a user of another external terminal, and information on the other external terminal. The information processing system according to claim 6.   A second display device is connected to the information processing device, and a virtual image of a user of each external terminal connected to the information processing device and information regarding each external terminal are connected to the second display device. The information processing system according to claim 6 or 7, wherein the information processing system is displayed.

Images