JP4438870B2 - Instruction system, instruction program and instruction apparatus - Google Patents

Description

  The present invention relates to an instruction system, an instruction program, and an instruction device.

  In a conference system or the like, it is necessary to give various instructions such as instructions for work procedures from the terminal side to the real object side. As such a conference system capable of giving an instruction from the terminal side to the real object side, for example, as shown in Patent Document 1, a captured image is captured by the video camera while imaging an object existing on the real object side. In addition, an instruction system for projecting an annotation image instructed based on a captured image on a terminal side onto a target object on a real object side is known.

  In Patent Document 2, a comment for an image is stored in association with the display position of the comment and identification information (for example, a file name) for identifying the image, and when the image is displayed again, the comment is restored to the original. A technique for displaying at the same position as the position is disclosed.

JP 2005-33756 A JP-A-9-101937

  By the way, in the technique disclosed in Patent Document 1 described above, since it is limited to a captured image from a video camera, an enlarged captured image obtained by enlarging a part of the captured image cannot be displayed. Annotation images cannot be specified. The technique disclosed in Patent Document 2 does not consider the enlargement ratio or reduction ratio between comments and images, and when an image is enlarged and displayed, a comment is displayed correspondingly. It is not something.

  The present invention has been made in view of such circumstances, and enables an instruction of an annotation image with respect to an enlarged captured image obtained by enlarging a part of the captured image, and an enlargement relationship between the enlarged captured image and the annotation image. It is an object of the present invention to provide an instruction system, an instruction program, and an instruction device that can achieve consistency.

  The invention according to claim 1 transmits both the captured image captured by the imaging unit that captures the object and the enlarged captured image captured by the enlarged imaging unit that magnifies and captures the object to the terminal, Control means capable of projecting an annotation image representing an annotation in accordance with an instruction based on the captured image made on the terminal onto an object by the projecting means, and the control means is configured to give an instruction regarding the annotation image to the enlarged captured image. In this case, the instruction system is characterized in that the imaging condition of the enlarged captured image is stored in association with the instruction.

  According to a second aspect of the present invention, in the first aspect of the present invention, when the instruction relating to the annotation image is given to the enlarged captured image, the control unit adds the annotation image to a corresponding portion on the captured image. It is characterized by displaying a mark indicating that it has been performed.

  According to a third aspect of the present invention, in the second aspect of the present invention, when the mark is instructed, the control means restores the enlarged captured image and the annotation image while maintaining the consistency based on the imaging condition. It is characterized by being displayed.

  According to a fourth aspect of the present invention, in the first or second aspect of the present invention, when the control unit is instructed to change the imaging area of the enlarged captured image on which the annotation image is displayed, It is characterized by displaying a corresponding enlarged captured image and annotation image.

  According to a fifth aspect of the invention, in the invention of the fourth aspect, when the annotation image is displayed, the control means displays the annotation image before and after the change while aligning the display positions.

  According to a sixth aspect of the present invention, in the second to fifth aspects of the present invention, when the instruction relating to the annotation image is given to an enlarged captured image that enlarges a part of the captured image, the control means When the enlarged captured image is saved together with the conditions, and when the mark is instructed, the enlarged captured image is further saved according to the imaging condition of the display position of the mark, and there is a difference in the comparison result between the both enlarged captured images In addition, a warning that the image is different from the enlarged captured image displayed when an instruction regarding the annotation image is given is displayed.

  According to the seventh aspect of the present invention, the computer transmits both the captured image captured by the imaging unit that captures the object and the enlarged captured image captured by the enlarged imaging unit that magnifies and captures the object to the terminal. And an annotation image representing an annotation in accordance with an instruction based on the captured image made by the terminal is caused to function as a control unit that can project the annotation on an object by the projecting unit. In the case of the instruction program, the imaging condition of the enlarged captured image is stored in association with the annotation image.

  The invention according to claim 8 is an image pickup means for picking up an object, an enlargement image pickup means for enlarging and picking up an object, a picked-up image picked up by the image pickup means and an enlarged picked-up image picked up by the enlargement image pickup means A projection unit that projects an annotation image representing an annotation in accordance with an instruction based on the object, a captured image and an enlarged captured image are both transmitted to the terminal, and the projection unit is projected in accordance with the instruction made at the terminal, And a control unit that stores an imaging condition of the enlarged captured image in association with the annotation image when an instruction regarding the annotation image is given to the enlarged captured image.

  According to the first aspect of the present invention, the enlarged captured image and the annotation image corresponding to the imaging condition can be reproduced based on the imaging condition when the annotation image is given.

  According to the second aspect of the present invention, it can be visually recognized that the annotation image is given to the enlarged captured image as compared with the case where the present configuration is not provided.

  According to the third aspect of the present invention, the operability for reproducing the enlarged captured image and the annotation image is improved as compared with the case where the present configuration is not provided.

  According to the fourth aspect of the present invention, as compared with the case where the present configuration is not provided, an enlarged captured image and an annotation image corresponding to the change of the imaging region can be displayed.

  According to the fifth aspect of the present invention, the display position of the annotation image is consistent before and after the change as compared with the case where the present configuration is not provided.

  According to the invention described in claim 6, when there is an instruction to restore the magnified captured image and the annotation image, the magnified captured image gives the annotation image compared to the case where the present configuration is not provided. It can be visually recognized that the image is different from the enlarged captured image.

  According to the seventh aspect of the present invention, compared to the case where the present configuration is not provided, the enlarged captured image and the annotation image corresponding to the imaging condition are obtained based on the imaging condition when the annotation image is added. Can be reproduced.

  According to the invention described in claim 8, compared with the case where the present configuration is not provided, the enlarged captured image and the annotation image corresponding to the imaging condition are obtained based on the imaging condition when the annotation image is added. Can be reproduced.

Embodiments of the present invention will be described below with reference to the drawings.
(First embodiment)
FIG. 1 is a configuration diagram of an instruction system according to an embodiment of the present invention, FIG. 2 is a functional block diagram illustrating a main configuration of a server, and FIG. 3 is a functional block diagram illustrating a hardware configuration of the server.

  As shown in FIG. 1, the instruction system includes an object-side device 10 and a terminal 100, and the object-side device 10 and the terminal 100 are connected to each other via a network 300 so that they can communicate with each other. Although only one terminal 100 is shown in FIG. 1, a plurality of terminals 100 and 101 may be connected to a server 50 (described later) of the object side device 10 through the network 300. In addition, the instruction system according to the present embodiment may be configured by a single device, such as being covered by a single housing, or may be configured by a plurality of devices.

  The object side apparatus 10 includes a video camera 21 as an imaging unit, a video camera 22 as an enlargement imaging unit, a projector 40 as a projection unit, a server 50 as a control unit, a half mirror 60, and the like.

  The video cameras 21 and 22 are composed of, for example, a CCD camera or the like. For example, the video camera 21 captures an image of an object 200 such as a white board on which a figure is drawn, and information on the captured image is captured by the server 50. In addition, the video camera 22 magnifies and captures an image of the object 200, and information on the enlarged captured image is captured by the server 50. The video cameras 21 and 22 capture an image transmitted through the half mirror 60.

  The projector 40 is configured by a liquid crystal projector or the like, and projects an annotation image (hereinafter referred to as an annotation image) according to an instruction based on a captured image or an enlarged captured image made on the terminal 100 onto the target object 200. Further, the projector 40 is arranged such that its optical system substantially coincides with the video camera 21, 22 by the half mirror 60. The projector 40 projects the annotation image transmitted from the server 50 onto the object 200 through the half mirror 60 through the optical system. The annotation image from the projector 40 is reflected by the half mirror 60 and projected onto the object 200. The annotation image AN includes all forms of images such as lines, characters, and figures. The reflectance of the half mirror is desirably about 0.3%.

  The server 50 controls the operations of the video cameras 21 and 22 and the projector 40 and exchanges various information with the terminal 100 via the network 300.

  More specifically, as illustrated in FIG. 2, an imaging unit 51 that controls the video cameras 21 and 22, an image transmission unit 52 that transmits captured images and enlarged captured images captured by the video cameras 21 and 22 to the terminal 100, and Is provided. The server 50 also includes a receiving unit 53 that receives a drawing instruction or the like from the terminal 100, and a projection unit 54 that controls the projector 40 and can project an annotation image on the projector 40. Furthermore, a drawing instruction processing unit 55 that performs necessary processing in response to a drawing instruction from the terminal 100 is provided.

  Therefore, the server 50 transmits the captured images and the like captured by the video cameras 21 and 22 to the terminal 100 and projects an annotation image corresponding to an instruction based on the captured images and the like made by the terminal 100 onto the target object 200 by the projector 40. Let

The terminal 100 includes a display device 110 including a liquid crystal display and a CRT display, a computer 120 connected to the network 300, a pointing device (mouse) 130 connected to the computer 120, and the like. For example, a notebook or desktop personal computer corresponds to the terminal 100.
The display device 110 displays an image transmitted from the object side device 10 on the display screen.
The pointing device 130 is used to form an instruction related to an annotation image to be projected onto the object 200 by operating various buttons with a pointer on a display screen on which a captured image or the like is displayed.

  Note that the server 50 and the computer 120 described above are so-called computers, that is, as shown in FIG. 3, a processing device 50 (120) a such as a CPU, an SRAM (Static Random Access Memory), a DRAM (Dynamic RAM), an SDRAM ( Synchronous DRAM), RAM 50 (120) b such as NVRAM (Non Volatile RAM), ROM (Read Only Memory) 50 (120) c such as flash memory, and I / F 50 (120) d for controlling input / output are bus 50 ( 120) Implemented by the hardware configuration connected by e.

  Therefore, the CPU 50 (120) a reads the required program stored in the storage device such as the RAM 50 (120) b or the ROM 50 (120) c, and performs the operation according to the program, thereby realizing the function of each device. Is done. In addition, as such a program, it can be set as the program according to the flowchart mentioned later. Further, if necessary, a hard disk may be connected to the bus 50 (120) e.

Next, the operation of the instruction system configured as described above will be described with reference to FIGS.
4 is a flowchart showing an example of processing in the server 50 of the object side device 10, FIG. 5 is a flowchart showing an example of processing in image formation in the computer 120 of the terminal 100, and FIG. 6 explains an example of operation on the terminal 100 side. It is a figure for doing.

First, the operation of the server 50 will be described.
As shown in FIG. 4, the server 50 on the object side device 10 side first starts capturing captured images and enlarged captured images of the video cameras 21 and 22 (step S11), and requests connection from the computer 120 of the terminal 100. It is determined whether there is any (step S12). Then, when there is a connection request from the computer 120, the server 50 transmits the captured images of the video cameras 21, 22 to the computer 120 of the terminal 100 via the network 300 (step S13).

  Next, the server 50 determines whether or not a drawing instruction is transmitted from the computer 120 (step S14). This drawing instruction is information regarding drawing of an annotation image. When receiving the drawing instruction, the server 50 performs a drawing instruction process according to the contents of the drawing instruction (step S15).

  On the other hand, if the server 50 does not receive the drawing instruction, the server 50 determines whether there is a disconnection request from the computer 120 (step S16). If there is no disconnection request, the server 50 returns to step S13 to obtain a new captured image. Are transmitted to the computer 120 of the terminal 100 through the network 300, and when there is a disconnection request, the transmission of the captured images of the video cameras 21, 22 is stopped (step S17). Then, the server 50 determines whether or not there is a process end request (step S18). If there is no end request, the server 50 returns to step S12 and repeats the above process. finish.

Next, the operation of the terminal 100 will be described.
As shown in FIG. 5, the computer 120 of the terminal 100 makes a connection request to the server 50 (step S21). Next, after the connection is completed, the computer 120 displays the captured image 111a transmitted from the server 50 on the object side on the display screen 111 of the display device 110, for example, as shown in FIG. 6 (step S22).

  Here, the computer 120 determines whether there is an instruction to enlarge a part of the captured image (step S23), and when there is an instruction to enlarge, the computer 120 activates the enlarged display screen 112 according to the instruction (step S23). S24).

  Specifically, the operator of the terminal 100 looks at the captured image 111a in the display screen 111 as shown in FIG. 6 and determines whether or not there is an area AR1 to be enlarged for a part of the displayed captured image 111a. Judge. If it is determined that there is an area AR1 to be enlarged, the pointing area 130 is operated to move the pointer Pt on the display screen 111 and specify and designate the enlarged area AR1 as an imaging area. As a result, the enlarged display screen 112 for enlarging and displaying the enlarged area AR1 is activated.

  As shown in FIG. 6, the enlarged display screen 112 displays an enlarged captured image 112 a captured by the video camera 22. In the enlarged display screen 112, “zoom in” and “zoom out” buttons BT2 and cursor buttons BT3 are arranged. The former button is a button for changing the enlargement ratio of the enlarged captured image, and the latter button is a button for moving the enlargement area AR1. Therefore, the operator of the terminal 100 operates these buttons to specify the position where the annotation image is projected.

  The computer 120 determines whether there is an instruction of the attention area AR2 from the operator for the enlarged captured image 112a displayed on the enlarged display screen 112 (step S23). A corresponding process is performed (step S26).

  Specifically, the operator of the terminal 100 has an area AR2 where an annotation image is desired to be projected on the displayed enlarged captured image 112a while viewing the enlarged captured image 112a in the enlarged display screen 112 as shown in FIG. In this case, the attention area AR2 is specified and designated while the pointer Pt on the display screen 111 is moved by operating the pointing device 130. The attention area AR2 is an area that defines a projection position for projecting the annotation image. At this time, information on the annotation image AN projected on the attention area AR2 is instructed at the same time.

  For example, by operating various buttons BT formed on the display screen 111 using the pointing device 130, graphic information such as a drawn rectangle or circle, a bitmap image prepared in advance, a keyboard, etc. Text information, pointers that reflect the operation of the pointing device 130, and the like. Then, the computer 120 transmits the various types of information specified in step S26 to the server 50 as a drawing instruction (step S27).

  Next, the computer 120 determines whether the instruction by the operator based on the enlarged captured image 112a of the video camera 22 made at the terminal 100 is completed (step S28), and when finished, disconnects from the server 50. A request is made (step S29), and the process is terminated. If the instruction operation by the operator of the terminal 100 has not ended, the process returns to step S22 and the above processing is repeated. Note that a drawing instruction may be directly given to the captured image 111a without specifying the enlarged area AR1.

Next, a drawing instruction process for the enlarged captured image, which is a feature of the present invention in the server 50, will be described with reference to FIGS.
FIG. 7 is a flowchart showing an example of processing for storing the annotation image AN for the enlarged captured image 112a, FIG. 8 shows an example of the enlarged display screen 112, and FIG. 9 shows an example of the display screen 111. The flowchart shown in FIG. 7 can be included in the flowchart showing an example of the operation of the server 50 shown in FIG. 4, and is preferably included in the process of step S15. The same applies to the subsequent processing.

  First, the server 50 determines whether or not the drawing instruction is a drawing instruction made on the enlarged captured image 112a (step S31). As shown in FIG. 8, the determination process is performed based on whether or not there has been a drawing instruction for projecting the annotation image AN on the enlarged captured image 112a.

  If the server 50 determines that the instruction has been given, the server 50 stores the imaging condition of the video camera 22 and the drawing instruction in association with each other (step S32). The imaging conditions include, for example, an enlargement ratio, a reduction ratio, position information, a size, and a shape of the enlargement area AR1.

  Next, the server 50 calculates the coordinates of the annotation image AN in the captured image 111a (step S33). More specifically, the server 50 stores the indicated position of the annotation image AN in the enlarged captured image 112a in a storage device or the like, and determines the correspondence relationship based on the indicated position and the display position of the enlarged area AR1. While maintaining, the position of the annotation image AN in the captured image 111a is calculated.

  Server 50 then sends the marker command to computer 120. The marker command is executed by the computer 120, and the computer 120 displays the marker M in the captured image 111a. The marker M is a mark indicating that an annotation has been made at a corresponding position on the captured image 112a when an instruction regarding the annotation image AN is given to the enlarged captured image 112a. It is not limited to a figure but may be a character. Therefore, as shown in FIG. 11, when the marker M is displayed on the captured image 111a, the operator of the terminal 100 can understand that the annotation image AN is instructed from the display screen 111 to the enlarged captured image 112a.

(Second Embodiment)
Subsequently, a second embodiment of the present invention will be described with reference to the second embodiment.
FIG. 10 is a flowchart showing an example of the restoration process of the annotation image AN.

  As shown in FIG. 10, the server 50 first determines whether or not the marker M is instructed to be restored (step S <b> 41). In this determination process, for example, as described with reference to FIG. 9, the pointing device 130 is operated by the operator of the terminal 100, the pointer Pt is overlaid on the marker M, and a restoration instruction such as a double click is given. It is done depending on whether or not. Note that if the restoration instruction is not given by the operator, the server 50 ends the process without performing the subsequent process.

  On the other hand, if the server 50 determines that the restoration instruction has been issued, the server 50 then reads the imaging condition and the drawing instruction corresponding to the marker M (step S42). These imaging conditions and the like are imaging conditions of the video camera 22 stored in a storage device such as a RAM or a ROM by the process in step S32 described above.

  Next, the server 50 sets the imaging condition of the video camera 22 based on the readout imaging condition (step S43). As a result, the setting corresponding to the stored imaging condition is performed on the video camera 22, and the server 50 transmits the enlarged captured image to the computer 120. Then, the server 50 transmits the saved drawing instruction (step S44).

  Thus, according to the present embodiment, when a restoration instruction is given to the marker M displayed on the captured image 111a on the display screen 111, the server 50 sets the imaging condition corresponding to the marker M to the video camera 22. The enlarged captured image is transmitted to the computer 120.

  When the computer 120 receives the enlarged captured image 112 a, the computer 120 displays the enlarged display screen 112 in the display screen 111 of the display device 110 and displays the enlarged captured image 112 a in the enlarged display screen 112. When a drawing instruction corresponding to the marker M is further received, an annotation image AN corresponding to the drawing instruction is displayed in the enlarged captured image 112a. As a result, the annotation image AN displayed in the enlarged captured image 112a is restored while maintaining consistency with the enlarged captured image 112a as shown in FIG.

(Third embodiment)
Next, a third embodiment of the present invention will be described with reference to FIGS.
FIG. 11 is a flowchart showing an example of the enlarged area changing process in the server 50, FIG. 12 shows an example of the display screen 111 before the changing process, and FIG. 13 shows an example of the enlarged display screen 112 after the changing process.

  In the instruction system according to the present embodiment, the enlarged captured image 112a and the annotation image AN in the enlarged captured image 112a are changed by an enlarged area changing process such as zoom out performed on the enlarged display screen 112.

  First, as shown in FIG. 11, the server 50 determines whether or not an instruction to change the enlarged area AR1 of the enlarged captured image 112a is instructed (step S51). For example, as shown in FIG. 12, the determination process is performed by operating the pointing device 130 by the operator of the terminal 100, and the pointer Pt is overlaid on the zoom-out button BT2 for lowering the enlargement ratio. This is done depending on whether or not

  The change instruction includes an instruction for the zoom-out button BT2, an instruction for the zoom-in button BT2 for increasing the enlargement ratio, an instruction for the cursor button BT3 for moving the enlargement area AR1 back and forth, and right and left.

  If the server 50 determines that an instruction to change the enlarged area AR1 has been given, the server 50 then determines whether or not the annotation image AN is displayed in the enlarged captured image 112a (step S52). For example, since the annotation image AN is displayed in the enlarged captured image 112a shown in FIG. 12, the server 50 determines that the annotation image AN is displayed.

  Next, the server 50 calculates the display position, size, shape, and the like of the annotation image AN from the imaging conditions (step S53). That is, a new display position, size, and the like in the enlarged captured image 112a of the annotation image AN are calculated according to the imaging condition of the video camera 22 whose setting has been changed due to zoom-out or the like.

  Here, the server 50 determines whether or not the annotation image AN corresponding to the calculation result fits in the enlarged captured image 112a (step S54). For example, when the enlargement area AR1 of the video camera 22 is enlarged by an instruction to change the zoom-in button BT2, and a new annotation image AN cannot be accommodated in the enlarged captured image 112a, or by an instruction to change the cursor button BT3 When the 22 imaging areas AR1 move and the annotation image AN deviates from the enlarged captured image 112a, it is determined that the annotation image AN corresponding to the calculation result does not fit within the enlarged captured image 112a, and the process returns to step S52. . At this time, the annotation image AN corresponding to the calculation result may be erased by an erase instruction or the like.

  When the server 50 determines that the annotation image AN corresponding to the calculation result fits in the enlarged captured image 112a, the server 120 issues a drawing instruction to display the annotation image AN corresponding to the calculated display position, size, and the like. (Step S55).

  As a result, as shown in FIG. 13, an enlarged captured image 112a with a reduced enlargement ratio is displayed in the enlarged display screen 112, and an annotation image with a reduced enlargement ratio is further displayed in the enlarged captured image 112a. The AN is displayed with the consistency of the display position. Thus, according to the present embodiment, when the enlarged area AR1 is changed, the annotation image AN displayed in the enlarged captured image 112a is also changed and displayed according to the change.

(Fourth embodiment)
Next, a fourth embodiment of the present invention will be described with reference to FIGS.
14 is a flowchart showing an example of an enlarged captured image saving process in the server 50, FIG. 15 is a flowchart showing an example of a warning display process in the server 50, FIGS. 16 and 17 are examples of the enlarged display screen 112, and FIG. FIG. 19 is a diagram for explaining a warning display.

  In the present embodiment, when a drawing instruction is given to the enlarged captured image 112a and the annotation image AN is displayed, the display of the enlarged display screen 112 is terminated and the enlarged display screen 112 is displayed again. When the enlarged captured image 112a displayed in the screen 112 is different from the previous enlarged captured image 112a, a warning indicating that the enlarged captured image 112a serving as the background of the annotation image AN has changed is displayed. It is what is displayed.

First, the processing for storing the annotation image AN for the enlarged captured image 112a will be described.
As shown in FIG. 14, the server 50 first performs an annotation image storage process for an enlarged captured image (step S61). The annotation image saving process for the enlarged captured image corresponds to the annotation image AN saving process described with reference to FIG.

  Next, when finishing the process of step S61, the server 50 acquires the enlarged captured image 112a (step S62) and stores it (step S63). For example, when an enlarged captured image 112a as shown in FIG. 16 is displayed on the enlarged display screen 112, the enlarged captured image 112a is acquired and stored.

Next, a case where the enlarged display screen 112 is displayed again by restoration will be described.
First, as shown in FIG. 15, the server 50 performs an annotation image restoration process for an enlarged captured image (step S71). The annotation image restoration process for the enlarged captured image corresponds to the restoration process of the annotation image AN described with reference to FIG. Thereby, the server 50 acquires the enlarged captured image 112a on which the annotation image AN is displayed as shown in FIG.

  Next, the server 50 acquires the enlarged captured image 112a (step S72). That is, the enlarged captured image 112a obtained by the video camera 22 after the enlarged captured image 112a on which the annotation image AN is displayed is restored. For example, the server 50 acquires an enlarged captured image 112a as illustrated in FIG.

  Next, the server 50 compares the enlarged captured image 112a obtained by the process of step S71 with the enlarged captured image 112a obtained by the process of step S72 (step S73). That is, the enlarged captured image 112a shown in FIG. 16 is compared with the enlarged captured image 112a shown in FIG.

  Here, the server 50 determines whether there is a difference between the two enlarged captured images 112a based on the comparison result (step S74). For example, when both of the enlarged captured images 112a are overlapped, as shown in FIG. 18, a difference image is obtained with respect to a rectangular image sandwiched between the rectangular images at both ends among the three rectangular images in the enlarged captured image 112a. IM has occurred. Therefore, in this case, it is determined that there is a difference between both enlarged captured images 112a. Note that the server 50 may determine that there is a difference when the difference is a large difference. In this case, whether or not the difference is large may be determined to be a large difference when the difference is larger than a predetermined threshold. As the threshold value, for example, an image feature amount such as a luminance value or an edge is used.

  If the server 50 determines that there is a difference in the enlarged captured image 112a in the process of step S74, the server 50 transmits a warning display command (step S75). Accordingly, the computer 120 that has received the warning display command causes the display device 110 to display the warning display screen 112c together with the display of the enlarged display screen 112, as shown in FIG. The display position of the warning display screen 112 c is not limited to the position below the enlarged display screen 112, and may be any position in the display device 110, and may be displayed over the enlarged display screen 112.

  As described above, according to the present embodiment, when the enlarged captured image 112a is different between when the annotation image AN is drawn and when the annotation image AN is restored, the operator of the terminal 100 is notified to that effect. Can be visually recognized.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to the specific embodiments according to the present invention, and various modifications are possible within the scope of the gist of the present invention described in the claims.・ Change is possible. For example, the program of the present invention can be provided not only by communication means but also stored in a recording medium such as a CD-ROM.

  In the embodiment described above, the video camera 21 and the video camera 22 are used together. However, when the video camera 21 is equipped with a digital zoom function, only the video camera 21 may be used. Thereby, the structural member of an instruction | indication system can be reduced. Furthermore, a pan function and a tilt function may be configured together.

  Further, the enlargement in the above-described embodiment includes not only an enlargement in which the enlargement ratio exceeds 100% but also an enlargement in which the enlargement ratio is less than 100%, that is, the meaning of reduction.

  According to the present invention, based on the imaging conditions when an annotation image is added, an enlarged captured image and an annotation image corresponding to the imaging condition can be reproduced, and conversation between remote locations is facilitated. Probability is high.

It is a block diagram which shows embodiment of an instruction | indication system. It is a functional block diagram which shows the principal part structure of a server. It is a functional block diagram which shows hardware constitutions, such as a server. It is a flowchart which shows an example of the process in the server of the target side apparatus. It is a flowchart which shows an example of the process in the image formation in the computer of a terminal. It is a figure for demonstrating an example of operation by the side of a terminal. It is a flowchart which shows an example of the preservation | save process of the annotation image with respect to an expansion captured image. It is an example of an enlarged display screen. It is an example of a display screen. It is a flowchart which shows an example of the decompression | restoration process of an annotation image. It is a flowchart which shows an example of the expansion area change process in a server. It is an example of the display screen before a change process. It is an example of the enlarged display screen after a change process. It is a flowchart which shows an example of the enlarged captured image storage process in a server. It is a flowchart which shows an example of the warning display process in a server. It is an example of an enlarged display screen. It is an example of an enlarged display screen. It is a figure for demonstrating a difference image. It is a figure for demonstrating a warning display.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Object side apparatus 21, 22 Video camera 40 Projector 50 Server 50a CPU
50b RAM
50c ROM
50d I / F
50e bus 51 imaging unit 52 image transmission unit 53 reception unit 54 projection unit 55 drawing instruction processing unit 60 half mirror 100 terminal 110 display device 111 display screen 111a captured image 112 enlarged display screen 112a enlarged captured image 112c warning display screen 120 computer 130 pointing Device 200 Object 200a Step 300 Network AN Annotation image Pt Pointer BT1 to BT3 Button IM Difference image

Claims (8)

The captured image captured by the imaging unit that captures the target object and the enlarged captured image captured by the expanded imaging unit that magnifies and captures the target object are both transmitted to the terminal, and the captured image formed by the terminal is transmitted to the terminal. Control means capable of projecting an annotation image representing an annotation according to an instruction based on the object by the projecting means;
The control unit stores an imaging condition of the enlarged captured image in association with the instruction when an instruction regarding the annotation image is given to the enlarged captured image.   The control means displays a mark indicating that the annotation image is added at a corresponding position on the captured image when an instruction regarding the annotation image is given to the enlarged captured image. The instruction system according to claim 1.   3. The control unit according to claim 2, wherein when the mark is instructed, the control unit restores and displays the enlarged captured image and the annotation image while maintaining consistency between the captured image and the annotation image based on the imaging condition. Instruction system.   The control means displays the enlarged captured image and the annotation image corresponding to the imaging region when an instruction to change the imaging region of the enlarged captured image on which the annotation image is displayed is given. The instruction system according to claim 1 or 2.   5. The instruction system according to claim 4, wherein when the annotation image is displayed, the control unit displays the annotation image before and after the change while aligning the display positions of the annotation image.   When the instruction relating to the annotation image is given to an enlarged captured image that enlarges a part of the captured image, the control means stores the enlarged captured image together with the imaging condition, and when the mark is instructed In addition, an enlarged captured image corresponding to the imaging condition at the display location of the mark is further stored, and when there is a difference between the comparison results of the both enlarged captured images, this is displayed when an instruction regarding the annotation image is given. 6. The instruction system according to claim 2, wherein a warning that the image is different from the enlarged captured image is displayed. Computer
The captured image captured by the imaging unit that captures the target object and the enlarged captured image captured by the expanded imaging unit that magnifies and captures the target object are both transmitted to the terminal, and the captured image formed by the terminal is transmitted to the terminal. An annotation image representing an annotation in accordance with an instruction based on the function to function as a control unit capable of projecting onto the object by the projection unit;
When the instruction regarding the annotation image is given to the enlarged captured image, the control means stores the imaging condition of the enlarged captured image in association with the annotation image. An imaging means for imaging an object;
Magnifying imaging means for magnifying and imaging the object;
Projecting means for projecting an annotation image representing an instruction according to an instruction based on a captured image captured by the imaging means and an enlarged captured image captured by the enlarged imaging means on the object;
When both the captured image and the magnified captured image are transmitted to the terminal, and the projection unit is projected according to the instruction made at the terminal, and an instruction regarding the annotation image is given to the magnified captured image Control means for storing the imaging condition of the enlarged captured image in association with the annotation image;
A pointing device characterized by comprising:

Images