JP2013114466A - Display system, display method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To copy an external appearance of an object in a real space and display the object at an arbitrary position in the real space without directly operating a computer or the like.SOLUTION: The display system includes: an imaging apparatus for imaging a real space; a display device for displaying an image in the real space; an operation detection section for detecting the operation of a user on the basis of the image imaging the real space; an identification section for identifying a copy object device designated by the operation of the user, among devices present in the real space; a display control section for controlling the display device to display a device image showing the copy object device; and a function execution section for executing at least a part of functions the copy object device is equipped with.

Description

  The present invention relates to a display system, a display method, and a program.

Patent Document 1 describes a method for accurately selecting a copy target. Patent Document 2 describes a method of inputting information by a finger gesture on a multi-touch panel. Patent Document 3 describes a method for displaying an annotation image in a region including an object.
Patent Document 1 Japanese Patent No. 4056477 Specification Patent Document 2 Japanese Translation of PCT International Publication No. 2010-532055 (WO2009 / 002758)
Patent Document 3 Japanese Unexamined Patent Application Publication No. 2009-290488

  However, conventionally, for example, it has been impossible to copy the appearance of a device such as a clock and a television device and display it at an arbitrary position of the user in the real space.

  In the first aspect of the present invention, a motion detection unit that detects a user's motion based on an image obtained by capturing a real space and a copy target device specified by the user's motion among the devices in the real space are identified. A display unit that controls the display device to display a device image representing the copy target device, and a function execution unit that executes at least some of the functions of the copy target device. Systems and display methods and programs associated with such display systems are provided.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

1 shows a configuration of a display system 10 according to the present embodiment. An example of the functional configuration of the control device 30 according to the present embodiment and the device image 20 displayed when the clock 14 is a copy target is shown. The processing flow at the time of copy and paste in the control apparatus 30 which concerns on this embodiment is shown. The processing flow at the time of operation with respect to the apparatus image 20 in the control apparatus 30 which concerns on this embodiment is shown. An example of the device image 20 displayed by the display system 10 when the remote controller 68 of the television device 66 is a copy target is shown. An example of the device image 20 displayed by the display system 10 when the camera 70 is a copy target is shown. 2 shows an exemplary hardware configuration of a computer 1900 according to the present embodiment.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows a configuration of a display system 10 according to the present embodiment. The display system 10 identifies a device in the real space as a copy target device by a user's predetermined operation (copy operation). Thereafter, the display system 10 displays an image representing the copy target device at the designated position in response to the user designating an arbitrary position in the real space by a predetermined operation (paste operation).

  For example, as shown in FIG. 1, the display system 10 specifies the clock 14 as a copy target device when the user touches the clock 14 in the room 12. Thereafter, the display system 10 displays on the desk 16 a device image 20 obtained by capturing the clock 14 by the user specifying the position on the desk 16. Thereby, the display system 10 can make the user feel as if the function of the designated device is copied and pasted at the designated position.

  The display system 10 includes one or more imaging devices 22, one or more display devices 24, and a control device 30. In the present embodiment, the display system 10 is provided in the room 12. The display system 10 may be provided not only in one room but also across a plurality of rooms.

  Each of the one or more imaging devices 22 images a real space. In the present embodiment, the one or more imaging devices 22 capture each of one or more devices (for example, a clock, a television device, a remote controller, a camera, etc.) existing in the room 12.

  Each of the one or more display devices 24 displays an image in real space. Each of the one or more display devices 24 is, for example, a projector. That is, each of the one or more display devices 24 projects an image onto an object in the room 12 and displays the image on the object. In the present embodiment, one or a plurality of display devices 24 are displayed on a place specified by the user in the room 12 (for example, on a desk, on a desk, a wall, a head portion of a bed, a floor, a ceiling, etc.). Is displayed.

  The control device 30 is a computer and controls the operation of each of the one or more imaging devices 22 and each of the one or more display devices 24. More specifically, the control device 30 controls each of the one or more imaging devices 22 and acquires the captured images captured by the imaging devices 22. In addition, the control device 30 controls one or more display devices 24 to display an image at a position specified by the user.

  Further, the control device 30 executes at least a part of functions of the devices existing in the real space. As an example, the control device 30 communicates with devices (for example, a clock, a television device, a remote, a camera, etc.) existing in the room 12, and controls or adjusts these devices.

  The display system 10 may further include a position detection device that detects the three-dimensional position of the object in the real space. As an example, the position detection device is a stereo camera having a right camera and a left camera. Such a position detection device can represent the subject distance to each object on the screen by the parallax between the image captured by the right camera and the image captured by the left camera.

  FIG. 2 shows an example of the functional configuration of the control device 30 according to the present embodiment and an apparatus image 20 displayed when the clock 14 is a copy target. The control device 30 includes an image acquisition unit 42, an operation detection unit 44, a specification unit 46, a display control unit 50, and a function execution unit 52. The image acquisition unit 42 acquires images captured by each of the one or more imaging devices 22.

  The motion detection unit 44 detects the user's motion based on the image obtained by capturing the real space acquired by the image acquisition unit 42. When the display system 10 includes a position detection device, the operation detection unit 44 may detect the user's operation by further using the position of the user's body detected by the position detection device.

  For example, the operation detection unit 44 detects whether or not the user has performed an operation of designating a device. As an example, the operation detection unit 44 detects that the user has performed an operation of touching the device. Further, as an example, the operation detection unit 44 may detect that the user touches the device and performs an operation such as a predetermined gesture indicating a copy operation.

  For example, the operation detection unit 44 detects whether or not the user has performed an operation of designating an image display position. For example, the motion detection unit 44 detects that the user has touched the surface of any object. Furthermore, as an example, the operation detection unit 44 may detect that the user has touched the display position and has performed an operation such as a predetermined gesture representing a paste operation.

  Further, as an example, the operation detection unit 44 detects whether or not the user has performed any operation on the device image 20 representing the device. For example, the operation detection unit 44 touches a button, a switch, or the like on the device image 20 representing the device and detects whether or not an operation of operating the touched switch has been performed.

  The identifying unit 46 identifies, as a copy target device, a device designated by the user through an operation among devices in the real space. When the user performs an operation to specify a device, the specifying unit 46 copies a device touched by the user among a plurality of devices (such as a clock, a television device, a remote controller, and a camera) existing in the room 12. Identify as equipment.

  In this case, for example, the specifying unit 46 performs a recognition process on an image obtained by capturing the copy target device, and determines the type of the copy target device. For example, the specifying unit 46 may recognize the identification code drawn on the copy target device and determine the type of the copy target device. Thereby, the specific | specification part 46 can specify an apparatus reliably.

  The display control unit 50 controls one or a plurality of display devices 24 to display the device image 20 representing the copy target device at a position specified by the operation by the user. As an example, the display control unit 50 cuts out the range of the copy target device from the image captured by the one or more imaging devices 22, generates the device image 20, and displays it. Instead of this, as an example, the display control unit 50 may generate and display an image representing the copy target device using computer graphics.

  The function execution unit 52 executes at least some of the functions of the copy target device specified by the specifying unit 46. As an example, the function execution unit 52 starts executing the function included in the copy target device in response to the specification of the copy target device by the specifying unit 46.

  As an example, the function execution unit 52 executes processing or control according to the operation content in response to the user performing an operation to operate a button or a switch of the device image 20. Further, the function execution unit 52 communicates with the clock 14 to control the copy target device by remote operation. In addition, when displaying the copy target device represented by computer graphics, the function execution unit 52 may execute the main function of the designated device and reflect it in the device image 20.

  In FIG. 2, the display system 10 shows an example in which the function of the clock 14 provided in the real space (the wall of the room 12) is copied and pasted on the desk 16. In this case, the specifying unit 46 specifies the clock 14 provided in the real space as the copy target device. Further, the display control unit 50 cuts out an image representing the clock 14 from the image captured by the imaging device 22 to generate the device image 20 and displays it on the desk 16. Then, the function execution unit 52 executes a function of setting an alarm and a time for the clock 14 in accordance with a user operation on the device image 20.

  Thereby, the display system 10 can confirm the time represented by the clock 14 even if the user is at a position where the user cannot see the clock 14 provided on the wall, for example. Furthermore, the display system 10 can set the time, alarm, and the like by remotely operating the clock 14.

  FIG. 3 shows a processing flow at the time of copy and paste in the control device 30 according to the present embodiment. The control device 30 executes the processing from step S11 to step S16 in FIG. 3 during copy and paste.

  First, in step S11, the control device 30 determines whether or not the user has performed an operation of specifying a device. As an example, the control device 30 determines whether or not the user touches any device in the room 12 with a hand and makes a predetermined gesture based on an image captured by the imaging device 22. When it is determined that the user has not designated the device (No in step S11), the control device 30 waits for the process in step S11. When it is determined that the user has designated the device (Yes in step S11), the control device 30 advances the process to step S12.

  Subsequently, in step S12, the control device 30 determines the device specified by the user, and specifies the determined device as a copy target device. As an example, the control device 30 determines which type of device is designated by the user from among a plurality of devices present in the room 12. More specifically, the control device 30 is a camera whether the device specified by the user is a clock provided on the wall, a television device, a remote controller of the television device, or the like. And so on.

  Subsequently, in step S13, the control device 30 determines whether or not the user has designated a display position. As an example, the control device 30 determines whether or not the user touches any display position in the room 12 and makes a predetermined gesture based on an image captured by the imaging device 22. . When it is determined that the user has not designated the display position (No in step S13), the control device 30 waits for the process in step S13. When it is determined that the user has designated the display position (Yes in step S13), the control device 30 advances the process to step S14.

  Subsequently, in step S14, the control device 30 generates a device image 20 representing the copy target device. As an example, the control device 30 cuts out an image from the imaging device 22 and generates a device image 20. Further, as an example, the control device 30 may generate the device image 20 by controlling the imaging device 22 to acquire an image zoomed up to the device specified by the user. The control device 30 may generate an image of a device having the same function as the copy target device by computer graphics.

  Subsequently, in step S15, the control device 30 controls one of the display devices 24 or the plurality of display devices 24 to display the device image 20 generated in step S14 at a position designated by the user. As described above, according to the control device 30, the device image 20 representing any device in the room 12 can be displayed at an arbitrary position in the room 12.

  FIG. 4 shows a processing flow when operating the device image 20 in the control device 30 according to the present embodiment. After the copy and paste in FIG. 3 is completed, the control device 30 executes the processing from step S21 to step S23 in FIG.

  First, in step S <b> 21, the control device 30 determines whether or not the user has performed an operation for operating the device image 20. As an example, the control device 30 touches the display position of the button or switch in the device image 20 and determines whether or not a movement for operating the button or switch has been performed. If the control device 30 determines that the user is not performing an operation on the device image 20 (No in step S21), the control device 30 waits for the process in step S21. If the control device 30 determines that the user has operated the device image 20 (Yes in step S21), the control device 30 advances the process to step S22.

  Subsequently, in step S <b> 22, the control device 30 determines the operation content for the device image 20 by the user. As an example, the control device 30 determines which button or switch or the like in the device image 20 the user is touching from the positional relationship between the user's hand or fingertip and the device image 20, It is determined how the switch is operated.

  Subsequently, in step S23, the control device 30 executes a function corresponding to the operation content determined in step S22. As an example, the control device 30 controls the copy target device through communication so as to execute processing corresponding to a button or switch operated by the user. For example, when the copy target device is the clock 14 and the alarm setting button is operated, the control device 30 performs the alarm setting by remotely operating the clock 14. For example, when the copy target device is the clock 14, the control device 30 stops the alarm of the clock 14 when the alarm stop button is operated.

  And control device 30 will return processing to Step S21, and will wait for processing until the next operation, after finishing processing of Step S23. As described above, according to the control device 30, the copy target device can be controlled in accordance with a user operation on the device image 20.

  FIG. 5 shows an example of the device image 20 displayed by the display system 10 when the remote controller 68 of the television device 66 is a copy target. As an example, the display system 10 may copy and paste the function of the remote controller 68 for remotely controlling a device such as the television device 66.

  In this case, the specifying unit 46 of the control device 30 specifies the remote controller 68 as a copy target device in response to the remote controller 68 being designated by the user's operation. Subsequently, the display control unit 50 of the control device 30 displays an image representing the remote controller 68 as, for example, the device image 20 on the desk 16.

  Then, the function execution unit 52 of the control device 30 executes a function of remotely controlling a device (for example, the television device 66) to be controlled by the remote controller 68 in accordance with a user operation on the device image 20. That is, in this case, the function execution unit 52 of the control device 30 controls a device to be controlled by the remote controller 68 instead of controlling the remote controller 68. In this case, when the function execution unit 52 of the control device 30 cannot control the device to be controlled by the remote controller 68, for example, the power source switching operation has been performed on the device image 20. Accordingly, a switch of a power supply line that supplies power to a control target device controlled by the copy target device may be controlled.

  As described above, according to the control device 30, an image representing the remote controller 68 can be displayed at a position desired by the user. Furthermore, according to the control device 30, it is possible to remotely operate a device to be controlled by the remote controller 68 in accordance with an operation on the displayed image of the remote controller 68.

  FIG. 6 shows an example of the device image 20 displayed by the display system 10 when the camera 70 is a copy target. As an example, the display system 10 may copy and paste the function of the camera 70. The camera 70 may be provided outdoors or the like.

  In this case, the one or more imaging devices 22 capture images of the operation surface of the camera 70 and the display surface of the camera 70 as an example. The specifying unit 46 of the control device 30 specifies the camera 70 provided in the real space as a copy target device.

  The display control unit 50 of the control device 30 displays an image representing the operation surface of the camera 70 as the device image 20 based on the image captured by the imaging device 22. As an example, the display control unit 50 displays a device image 20-1 representing an operation surface (for example, an upper surface) provided with a shutter button of the camera 70.

  Further, the display control unit 50 of the control device 30 displays an image representing the display surface of the camera 70 as the device image 20 based on the image captured by the imaging device 22. As an example, the display control unit 50 displays the device image 20-2 representing the display surface (for example, the back surface) on which the display panel of the camera 70 is provided.

  Then, the function execution unit 52 of the control device 30 executes a function of remotely controlling the camera 70 in accordance with a user operation on the device image 20. For example, when an operation of pressing the shutter button is performed on the device image 20-1, the function execution unit 52 remotely controls the camera 70 to cause the camera 70 to image a subject. As described above, according to the control device 30, the camera 70 can be remotely operated while viewing the display screen of the camera 70.

  FIG. 7 shows an example of the hardware configuration of a computer 1900 according to this embodiment. A computer 1900 according to this embodiment is connected to a CPU peripheral unit having a CPU 2000, a RAM 2020, a graphic controller 2075, and a display device 2080 that are connected to each other by a host controller 2082, and to the host controller 2082 by an input / output controller 2084. Input / output unit having communication interface 2030, hard disk drive 2040, and CD-ROM drive 2060, and legacy input / output unit having ROM 2010, flexible disk drive 2050, and input / output chip 2070 connected to input / output controller 2084 With.

  The host controller 2082 connects the RAM 2020 to the CPU 2000 and the graphic controller 2075 that access the RAM 2020 at a high transfer rate. The CPU 2000 operates based on programs stored in the ROM 2010 and the RAM 2020 and controls each unit. The graphic controller 2075 acquires image data generated by the CPU 2000 or the like on a frame buffer provided in the RAM 2020 and displays it on the display device 2080. Instead of this, the graphic controller 2075 may include a frame buffer for storing image data generated by the CPU 2000 or the like.

  The input / output controller 2084 connects the host controller 2082 to the communication interface 2030, the hard disk drive 2040, and the CD-ROM drive 2060, which are relatively high-speed input / output devices. The communication interface 2030 communicates with other devices via a network. The hard disk drive 2040 stores programs and data used by the CPU 2000 in the computer 1900. The CD-ROM drive 2060 reads a program or data from the CD-ROM 2095 and provides it to the hard disk drive 2040 via the RAM 2020.

  The input / output controller 2084 is connected to the ROM 2010, the flexible disk drive 2050, and the relatively low-speed input / output device of the input / output chip 2070. The ROM 2010 stores a boot program that the computer 1900 executes at startup and / or a program that depends on the hardware of the computer 1900. The flexible disk drive 2050 reads a program or data from the flexible disk 2090 and provides it to the hard disk drive 2040 via the RAM 2020. The input / output chip 2070 connects the flexible disk drive 2050 to the input / output controller 2084 and inputs / outputs various input / output devices via, for example, a parallel port, a serial port, a keyboard port, a mouse port, and the like. Connect to controller 2084.

  A program provided to the hard disk drive 2040 via the RAM 2020 is stored in a recording medium such as the flexible disk 2090, the CD-ROM 2095, or an IC card and provided by the user. The program is read from the recording medium, installed in the hard disk drive 2040 in the computer 1900 via the RAM 2020, and executed by the CPU 2000.

  The program installed in the computer 1900 and causing the computer 1900 to function as the control device 30 includes an image acquisition module, an operation detection module, a specific module, a display control module, and a function execution module. These programs or modules work on the CPU 2000 or the like to cause the computer 1900 to function as the image acquisition unit 42, the operation detection unit 44, the specifying unit 46, the display control unit 50, and the function execution unit 52, respectively.

  The information processing described in these programs is read into the computer 1900, whereby the image acquisition unit 42, the operation detection unit 44, and the specific means in which the software and the various hardware resources described above cooperate with each other. Functions as the unit 46, the display control unit 50, and the function execution unit 52. And the specific control apparatus 30 according to the intended use is constructed | assembled by implement | achieving the calculation or processing of the information according to the intended use of the computer 1900 in this embodiment by these specific means.

  As an example, when communication is performed between the computer 1900 and an external device or the like, the CPU 2000 executes a communication program loaded on the RAM 2020 and executes a communication interface based on the processing content described in the communication program. A communication process is instructed to 2030. Under the control of the CPU 2000, the communication interface 2030 reads transmission data stored in a transmission buffer area or the like provided on a storage device such as the RAM 2020, the hard disk drive 2040, the flexible disk 2090, or the CD-ROM 2095, and sends it to the network. The reception data transmitted or received from the network is written into a reception buffer area or the like provided on the storage device. As described above, the communication interface 2030 may transfer transmission / reception data to / from the storage device by a DMA (direct memory access) method. Instead, the CPU 2000 transfers the storage device or the communication interface 2030 as a transfer source. The transmission / reception data may be transferred by reading the data from the data and writing the data to the communication interface 2030 or the storage device of the transfer destination.

  The CPU 2000 is all or necessary from among files or databases stored in an external storage device such as a hard disk drive 2040, a CD-ROM drive 2060 (CD-ROM 2095), and a flexible disk drive 2050 (flexible disk 2090). This portion is read into the RAM 2020 by DMA transfer or the like, and various processes are performed on the data on the RAM 2020. Then, CPU 2000 writes the processed data back to the external storage device by DMA transfer or the like. In such processing, since the RAM 2020 can be regarded as temporarily holding the contents of the external storage device, in the present embodiment, the RAM 2020 and the external storage device are collectively referred to as a memory, a storage unit, or a storage device. Various types of information such as various programs, data, tables, and databases in the present embodiment are stored on such a storage device and are subjected to information processing. Note that the CPU 2000 can also store a part of the RAM 2020 in the cache memory and perform reading and writing on the cache memory. Even in such a form, the cache memory bears a part of the function of the RAM 2020. Therefore, in the present embodiment, the cache memory is also included in the RAM 2020, the memory, and / or the storage device unless otherwise indicated. To do.

  In addition, the CPU 2000 performs various operations, such as various operations, information processing, condition determination, information search / replacement, etc., described in the present embodiment, specified for the data read from the RAM 2020 by the instruction sequence of the program. Is written back to the RAM 2020. For example, when performing the condition determination, the CPU 2000 determines whether the various variables shown in the present embodiment satisfy the conditions such as large, small, above, below, equal, etc., compared to other variables or constants. When the condition is satisfied (or not satisfied), the program branches to a different instruction sequence or calls a subroutine.

  Further, the CPU 2000 can search for information stored in a file or database in the storage device. For example, in the case where a plurality of entries in which the attribute value of the second attribute is associated with the attribute value of the first attribute are stored in the storage device, the CPU 2000 displays the plurality of entries stored in the storage device. The entry that matches the condition in which the attribute value of the first attribute is specified is retrieved, and the attribute value of the second attribute that is stored in the entry is read, thereby associating with the first attribute that satisfies the predetermined condition The attribute value of the specified second attribute can be obtained.

  The program or module shown above may be stored in an external recording medium. As the recording medium, in addition to the flexible disk 2090 and the CD-ROM 2095, an optical recording medium such as DVD or CD, a magneto-optical recording medium such as MO, a tape medium, a semiconductor memory such as an IC card, and the like can be used. Further, a storage device such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet may be used as a recording medium, and the program may be provided to the computer 1900 via the network.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

  The order of execution of each process such as operations, procedures, steps, and stages in the apparatus, system, program, and method shown in the claims, the description, and the drawings is particularly “before” or “prior to”. It should be noted that the output can be realized in any order unless the output of the previous process is used in the subsequent process. Regarding the operation flow in the claims, the description, and the drawings, even if it is described using “first”, “next”, etc. for convenience, it means that it is essential to carry out in this order. It is not a thing.

10 display systems, 12 rooms, 14 clocks, 16 desks, 20 device images, 22 imaging devices, 24 display devices, 30 control devices, 42 image acquisition units, 44 motion detection units, 46 identification units, 50 display control units, 52 functions Execution unit, 66 television device, 68 remote controller, 70 camera, 1900 computer, 2000 CPU, 2010 ROM, 2020 RAM, 2030 communication interface, 2040 hard disk drive, 2050 flexible disk drive, 2060 CD-ROM drive, 2070 input / output Chip, 2075 graphic controller, 2080 display device, 2082 host controller, 2084 input / output controller, 2090 flexible disk, 2095 CD-ROM

Claims (13)

A motion detector that detects a user's motion based on an image of real space;
Among the devices in the real space, a specifying unit that specifies a copy target device specified by the user's operation,
A display control unit for controlling a display device to display a device image representing the copy target device;
A function execution unit that executes at least some of the functions of the copy target device;
A display system comprising: An imaging device for imaging real space;
A display device for displaying an image in the real space;
The display system according to claim 1, further comprising: The display control unit
The display system according to claim 2, wherein a part of the copy target device is cut out from an image captured by the imaging device and displayed as the device image. The display system according to claim 1, wherein the display control unit displays the device image at a position specified by a user operation. The specifying unit performs recognition processing on an image obtained by imaging the copy target device, determines a type of the copy target device,
The display system according to claim 1, wherein the function execution unit executes a function of the identified type of device. The display system according to claim 5, wherein the specifying unit recognizes an identification code drawn on the copy target device and determines a type of the copy target device. The function execution unit controls a switch of a power supply line that supplies power to a control target device controlled by the copy target device in response to a power source switching operation performed on the device image. The display system according to any one of 1 to 6. The specifying unit specifies a clock provided in the real space as the copy target device;
The display control unit displays an image representing the clock as the device image,
The display system according to claim 1, wherein the function execution unit executes a function of setting the clock according to a user operation on the device image. The specifying unit specifies a remote controller for remotely controlling a device as the copy target device,
The display control unit displays an image representing the remote controller as the device image,
The display system according to claim 1, wherein the function execution unit executes a function of remotely controlling a device to be controlled by the remote controller in accordance with a user operation on the device image. The specifying unit specifies a camera provided in the real space as the copy target device,
The display control unit displays an image representing an operation surface of the camera as the device image,
The display system according to claim 1, wherein the function execution unit executes a function of remotely controlling the camera according to a user operation on the device image. A position detection device for detecting a three-dimensional position of the object in the real space;
The display system according to claim 1, wherein the motion detection unit detects the motion of the user based on a position of the user's body detected by the position detection device. A motion detection step for detecting a user's motion based on an image of real space;
A specifying step of specifying a copy target device specified by the user's operation among devices in the real space;
A display control step of controlling a display device to display a device image representing the copy target device;
A function execution step for executing at least a part of functions of the copy target device;
A display method comprising: Computer
A motion detector that detects a user's motion based on an image of real space;
Among the devices in the real space, a specifying unit that specifies a copy target device specified by the user's operation,
A display control unit for controlling a display device to display a device image representing the copy target device;
A function execution unit that executes at least some of the functions of the copy target device;
A program for functioning as a control device.

Images