JP2019117649A - Image display apparatus and image display method - Google Patents

Abstract

To provide an image display apparatus capable of assisting input to a mobile terminal and expanding an image display function of the mobile terminal in synchronization with the mobile terminal.SOLUTION: An image display apparatus according to the present invention includes a display unit and a detection unit. An image display surface of the display unit is arranged horizontally. The detection unit is disposed on the image display surface of the display unit, and detects a touched position of the image display surface. The display unit displays a screen on the image display surface for selection as to whether to display the image of an input device for inputting information to an external device in response to such detection by the detection unit that the external device is placed on the detection unit, and the display unit displays the image of the input device on the image display surface in response to the detection unit detecting that the display of the image of the input device is selected.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a table-type image display device in which an image display surface of a touch panel display is horizontally arranged, and in particular, assisting in input to a mobile terminal and extending an image display function of the mobile terminal in synchronization with the mobile terminal. The present invention relates to an image display device that enables this.

  In recent years, portable information processing apparatuses have become widespread. As the user interface device, a display (hereinafter referred to as a touch panel display) in which a touch panel is arranged on an image display surface is adopted. When it is necessary not only to directly touch the object displayed on the touch panel display but also to input characters and numbers, it is common to display a software keyboard on the screen and operate it.

  Also in a computer, a touch panel display is mounted, and an operating system (OS) adopting a user interface assuming a touch operation is also provided. There is also provided a portable terminal device called a tablet PC or a tablet terminal etc. which is not equipped with either a conventional computer keyboard (hereinafter simply referred to as a keyboard) and a computer mouse as standard.

  With regard to touch panel displays, the increase in screen size has progressed, and it is expected that touch panel displays will be used for various applications in the future. For example, it has been proposed that the image display surface be arranged horizontally and used as a table (such as an office table or a conference table) that can be operated by touch, from the original use of an electronic blackboard or the like.

  With the spread of such touch panel displays, there is a demand for improvement in touch operations. For example, Patent Document 1 below discloses a technology for displaying a keyboard as an input unit on a touch panel in an office equipment integrated desk for improving an office environment. Specifically, the top plate of the desk is provided with a personal computer function unit, and the personal computer function unit is provided with a screen unit for displaying a display screen and a keyboard touch panel for displaying a keyboard. Further, Patent Document 2 below discloses a technology in which the upper surface of a table is configured by a touch panel and connected to an external computer to display a display screen and a keyboard on the upper surface of the table. The table is provided with a foot switch or push switch for turning on / off the touch panel function.

JP-A-8-89346 JP, 2012-138030, A

  However, in the technology disclosed in Patent Document 1, due to the restriction of the size of the liquid crystal panel, it is only possible to provide a touch panel display on a part of the desk top, which is mainly limited to personal use, and by multiple persons simultaneously There was a problem that can not be used.

  As described above, although the screen size of touch panel displays has progressed, the table-type image display device capable of touch operation using an image display surface arranged horizontally as a table has not been fully utilized. , The possibility has not been fully considered. For example, in patent document 2, the function of the touch panel which comprises the upper surface of a table can only be turned on / off.

  In recent years, wireless communication environments such as Bluetooth (registered trademark) and Wi-Fi have become widespread. Portable terminals equipped with these wireless communication functions as standard are also in widespread use, and cooperation between the portable terminal and the table-type image display device is expected to improve operability, improve convenience, and create new functions. Ru.

  Therefore, the present invention is a table-type image display device in which the image display surface of the touch panel display is arranged horizontally, and in synchronization with the portable terminal, assisting the input to the portable terminal and the image display function of the portable terminal. An object of the present invention is to provide an image display device that enables expansion.

  The image display apparatus according to the present invention includes a display unit in which an image display surface is horizontally disposed, and a detection unit disposed on the image display surface of the display unit and detecting a touched position. The display unit receives the detection that the external device has been placed on the detection unit, and the external device is provided in the vicinity of the area on the image display surface corresponding to the detected placement area. Display the image of the input device for.

  Preferably, the image display apparatus transmits the instruction information corresponding to the operation on the input device to the external device in response to the detection unit detecting that the touch to the position corresponding to the image of the input device is detected by the detection unit. Further includes

  More preferably, the display unit moves and displays the image of the input device to the newly instructed position in response to the detection unit being instructed to change the display position of the image of the input device.

  More preferably, the display unit magnifies and displays the image data displayed on the image display surface of the external device, which is received from the external device in response to the detection of the touch detected by the detection unit. .

  Preferably, the display unit is an image displayed on the image display surface of the external device received from the external device in response to the detection of the touch detected by the detection unit in a state where the external device is placed. The data is enlarged and displayed in the area including the portion corresponding to the mounting area, and in the enlarged display state, the detection unit receives that the external device is removed from above the detection unit, and The partial image displayed in the corresponding part is changed to a different partial image.

  More preferably, the display unit is displayed on the image display surface of the external device received from the external device in response to detection of the touch by the detection unit in a state where the external device is placed. The image data is enlarged and displayed in an area including a portion corresponding to the placement area, and in the enlarged state, even if the detection unit detects that the external device is removed from above the detection unit, the enlargement is performed. Keep the display.

  More preferably, in response to the detection unit detecting that the plurality of external devices are placed on the detection unit, the display unit further detects each of the positions corresponding to the plurality of detected placement positions. Display the image of the input device in the vicinity.

  Preferably, the display unit is a boundary of a region on which each of the plurality of external devices is to be placed in response to the detection unit receiving an instruction to display the input device to each of the plurality of external devices. Display

  According to the present invention, in the table type image display device, when the terminal device is placed on the image display surface, the image of the input device such as the keyboard is displayed in the vicinity of the terminal device, thereby the user Input to the device can be easily performed.

  Further, by displaying the image displayed on the display screen of the terminal device on the image display device with a larger screen, the user can easily and efficiently perform a meeting, a presentation, and the like.

FIG. 1 is a block diagram showing a schematic configuration of an image display device according to a first embodiment of the present invention. It is a figure which shows an example of the detection method of touch input. FIG. 1 is a block diagram showing a schematic configuration of a terminal device cooperating with an image display device according to a first embodiment of the present invention. It is a flowchart which shows the control structure of the program run in an image display apparatus, in order to implement | achieve cooperation with the image display apparatus which concerns on the 1st Embodiment of this invention, and a terminal device. It is a flowchart which shows the control structure of the program performed in a terminal device, in order to implement | achieve cooperation with the image display apparatus and terminal device which concern on the 1st Embodiment of this invention. It is a top view which shows the image display apparatus of the state on which the selection screen was displayed. It is a top view which shows the image display apparatus of the state on which the message for providing an input assistance function was displayed. It is a top view which shows the image display apparatus of the state in which the terminal device was mounted and the image of the keyboard and touch pad was displayed. It is a top view which shows the image display apparatus of the state in which the message for providing a display assistance function was displayed. It is a top view which shows the image display apparatus of the state on which the image of the screen of a terminal device was displayed. It is a top view which shows the image display apparatus of the state in which the display position of the keyboard image and touch pad image was changed from the state of FIG. It is a top view which shows the drag operation with respect to a terminal device and an image display apparatus. It is a top view which shows the drag operation with respect to a terminal device and an image display apparatus following FIG. FIG. 6 is a plan view showing the image display apparatus in a state in which the terminal device is placed, and a keyboard image, a touch pad image, and an image of a magnified terminal device screen are displayed. It is a flowchart which shows the control structure of the program run in an image display apparatus, in order to implement | achieve cooperation with the image display apparatus which concerns on the 2nd Embodiment of this invention, and a terminal device. It is a flowchart which shows the control structure of the program run in a terminal device in order to implement | achieve cooperation with the image display apparatus and terminal device which concern on the 2nd Embodiment of this invention. It is a top view which shows the image display apparatus of the state in which the image of the screen of a terminal device was displayed by the drag operation on a terminal device. It is a top view which shows the state from which the terminal device was removed from the top of an image display apparatus. FIG. 19 is a plan view showing a state in which the terminal device is removed from above the image display device, which is different from FIG. 18; It is a top view which shows the state in which the screen of the terminal device was changed, after the terminal device was removed from on an image display apparatus. It is a flowchart which shows the control structure of the program run in an image display apparatus, in order to implement | achieve cooperation with the image display apparatus which concerns on the 3rd Embodiment of this invention, and a terminal device. It is a flowchart which shows the control structure of the program run in a terminal device, in order to implement cooperation with a graphics display device and a terminal device concerning a 3rd embodiment of the present invention. It is a top view which shows the image display apparatus of the state in which the message for providing the input assistance function of 2 units | sets of terminal devices was displayed. It is a top view which shows the image display apparatus of the state in which the key for 2 units was touched. It is a top view which shows the image display apparatus of the state in which one terminal device was mounted. It is a top view which shows the image display apparatus of the state in which two terminal devices were mounted. FIG. 27 is a plan view showing an image display apparatus in a state in which two terminal devices are placed, which is different from FIG. 26.

  In the following embodiments, the same parts are given the same reference numerals. Their names and functions are also identical. Therefore, detailed description about them will not be repeated.

  In the following, “touch” means that the detection device of the input position is in a state capable of detecting the position, and when touching and pressing the detection device, the case of touching without pressing and without touching Including close proximity to The detection device of the input position is not limited to the contact type, and a non-contact type device can also be used. In the case of the non-contact type detection device, "touch" means a state in which the input position is close to the detection device to a detectable distance.

First Embodiment
Referring to FIG. 1, an image display apparatus 100 according to the first embodiment of the present invention includes an arithmetic processing unit (hereinafter referred to as a CPU) 102, a read only memory (hereinafter referred to as a ROM) 104, and can be rewritten. A memory (hereinafter referred to as a RAM) 106, a recording unit 108, a communication unit 110, a touch detection unit 112, a display unit 114, a display control unit 116, a video memory (hereinafter referred to as a VRAM) 118, and a bus 120 . The CPU 102 controls the entire image display apparatus 100.

  As described later, the touch detection unit 112 and the display unit 114 constitute a touch panel display, the image display surface of the touch panel display is disposed horizontally, and the image display apparatus 100 is used as a table.

  The ROM 104 is a non-volatile storage device, and stores programs and data necessary to control the operation of the image display apparatus 100. The RAM 106 is a volatile storage device in which data is erased when the power supply is interrupted. The recording unit 108 is a non-volatile storage device that holds data even when the power supply is interrupted, and is, for example, a hard disk drive or a flash memory. The recording unit 108 may be configured to be removable. The CPU 102 reads a program from the ROM 104 onto the RAM 106 via the bus 120, and executes the program using a part of the RAM 106 as a work area. The CPU 102 controls each part constituting the image display apparatus 100 in accordance with a program stored in the ROM 104.

  The CPU 102, the ROM 104, the RAM 106, the recording unit 108, the communication unit 110, the touch detection unit 112, the display control unit 116, and the VRAM 118 are connected to the bus 120. The exchange of data (including control information) between the units is performed via the bus 120.

  The display unit 114 is a display panel (liquid crystal panel or the like) for displaying an image. The display control unit 116 includes a drive unit for driving the display unit 114, reads out the image data stored in the VRAM 118 at a predetermined timing, generates a signal for displaying as an image on the display unit 114, and performs display. Output to the part 114. The image data to be displayed is read from the recording unit 108 by the CPU 102 and transmitted to the VRAM 118.

  The communication unit 110 enables wireless communication with a portable terminal device described later. The wireless communication scheme is optional, and for example, Bluetooth (registered trademark) or Wi-Fi can be used. In the case of using Wi-Fi, the communication unit 110 may be connected by wire to a network to which a wireless router is connected.

  The touch detection unit 112 is, for example, a touch panel, and detects a touch operation by the user. The touch detection unit 112 is disposed so as to overlap the display screen of the display unit 114. The touch on the touch detection unit 112 is an operation of designating a point of the image displayed on the display screen, which corresponds to the touch position. Therefore, in the present specification, in the case where the touch on the image displayed on the display unit 114 is described in order to eliminate the redundancy described, this description indicates the position on the corresponding touch detection unit 112. Means a touch of

  Detection of a touch operation in the case of using a touch panel as the touch detection unit 112 will be described with reference to FIG. FIG. 2 shows an infrared cut-off detection type touch panel (touch detection unit 112). The touch panel is disposed in a row so as to face light emitting diode rows (hereinafter referred to as LED rows) 140 and 142 arranged in two rows on adjacent two sides of the rectangular writing surface and to the respective LED rows 140 and 142 And two photodiode rows (hereinafter referred to as PD rows) 144 and 146. Infrared rays are emitted from the LEDs of the LED rows 140 and 142, and the infrared rays are detected by the PDs of the opposing PD rows 144 and 146, respectively. In FIG. 2, infrared rays from the LEDs of the LED strings 140 and 142 are indicated by arrows pointing upward and to the left.

  The touch panel includes, for example, a microcomputer (hereinafter, referred to as a microcomputer) (an element including a CPU, a memory, an input / output circuit, and the like), and controls light emission of each LED. Each PD outputs a voltage according to the intensity of the received light. The output voltage of PD is amplified by an amplifier. In addition, since signals are simultaneously output from a plurality of PDs in each of the PD columns 144 and 146, the output signals are temporarily stored in a buffer and then output as serial signals according to the arrangement order of the PDs and transmitted to the microcomputer Ru. The order of the serial signals output from the PD column 142 represents the X coordinate. The order of serial signals output from the PD column 146 represents Y coordinates.

  When the user 900 touches on the touch panel with a finger, infrared light is blocked at the touched position. Therefore, the output voltage of the PD that has received the infrared light decreases until it is shut off. Since the signal portion from the PD corresponding to the touched position (XY coordinates) decreases, the microcomputer detects a portion where the signal levels of the two received serial signals are lowered, and obtains the touched position coordinates. . The microcomputer transmits the determined position coordinates to the CPU 102. Since the process of detecting the touch position is repeated at a predetermined detection cycle, if the same point is touched for a longer time than the detection cycle, the same coordinate data is repeatedly output. The microcomputer does not transmit the position coordinates unless touched anywhere.

  As the above-described touch position detection technique is known, the further description will not be repeated. In addition, as the touch detection unit 112, a touch panel other than the infrared blocking method (capacitive method, surface acoustic wave method, resistive film method, or the like) may be used. In the electrostatic capacitance method, if it is close to the sensor, the position can be detected without contact.

  The image display apparatus 100 is configured as described above, and the user can operate the image display apparatus 100 in the same manner as a computer. That is, the user performs a touch operation on a user interface (operation button or the like) on the screen displayed on the display unit 114 via the touch detection unit 112 to activate an application program, and is displayed by the activated application. Can be performed via the touch detection unit 112.

  Referring to FIG. 3, a portable terminal device (hereinafter simply referred to as a terminal device) cooperating with image display device 100 is configured similarly to image display device 100. That is, the terminal device 200 includes a CPU 202, a ROM 204, a RAM 206, a recording unit 208, a communication unit 210, a touch detection unit 212, a display unit 214, a display control unit 216, a VRAM 218, and a bus 220. The CPU 202 controls the entire terminal device 200. The touch detection unit 212 and the display unit 214 constitute a touch panel display.

  The ROM 104 stores programs and data necessary to control the operation of the terminal device 200. The CPU 202 reads a program from the ROM 204 onto the RAM 206 via the bus 220, and executes the program using a part of the RAM 206 as a work area. The CPU 202 controls each unit constituting the terminal device 200 in accordance with a program stored in the ROM 204.

  The display unit 214 is a display panel (liquid crystal panel or the like) for displaying an image. The display control unit 216 includes a drive unit for driving the display unit 214, reads out the image data stored in the VRAM 218 at a predetermined timing, generates a signal for displaying as an image on the display unit 214, and performs display. Output to the unit 214. The CPU 202 reads out the image data to be displayed from the recording unit 208 and transmits the image data to the VRAM 218.

  The communication unit 210 enables wireless communication with the image display apparatus 100 by Bluetooth (registered trademark), Wi-Fi, or the like.

  The touch detection unit 212 is, for example, a touch panel, and detects a touch operation by the user. The touch detection unit 212 is disposed so as to overlap the display screen of the display unit 214. The touch on the touch detection unit 212 is an operation of designating a point of the image displayed on the display screen, which corresponds to the touch position. The detection of the touch operation in the case of using the touch panel as the touch detection unit 212 is as described above with reference to FIG.

  Hereinafter, with reference to FIGS. 4 and 5, a method of realizing the input assistance function and the display assistance function of the terminal device 200 by cooperation of the image display device 100 and the terminal device 200 will be specifically described. Here, in the state where the program for realizing cooperation is activated in image display apparatus 100, the user brings terminal apparatus 200 at which the program for realizing cooperation is activated and approaches image display apparatus 100. I suppose. Communication between the image display device 100 and the terminal device 200 is assumed to be directly performed by Bluetooth (registered trademark).

  The operation of the image display apparatus 100 will be described. Referring to FIG. 4, in step 400, CPU 102 determines whether or not a pre-registered terminal device 200 to be provided with the cooperation function has been detected. If it is determined that it has been detected, control transfers to 402. If not, the control proceeds to step 432, and the CPU 102 determines whether termination has been instructed. The end instruction is issued, for example, by turning off the image display device 100. If termination is instructed, this program is terminated. Otherwise, control returns to step 400. Thus, the detection process is repeated until the corresponding terminal device 200 is detected.

  When the terminal device 200 is detected, in step 402, the CPU 102 displays a function selection screen on the display unit 114, and waits for the user's selection. For example, on the screen 600 of the display unit 114, a window for selecting the function of the input assistance and the display assistance as shown in FIG. 6 is displayed.

  In step 404, the CPU 102 determines whether a selection has been made. Specifically, the CPU 102 determines whether or not either the “input assist” key or the “display assist” key is touched. If it is determined that either the "input assist" key or the "display assist" key is touched, the control moves to step 406. Otherwise, step 404 is repeated.

  In step 406, the CPU 102 transmits the selection result detected in step 404 to the terminal device 200 as selection information. For example, when it is detected that the "input auxiliary" key is touched, the CPU 102 transmits "0" as the selection information, and when it is detected that the "display auxiliary" key is touched. Send "1" as the selection information.

  At step 408, the CPU 102 branches the control according to the selection detected at step 404. If it is determined that the "input assist", the control proceeds to step 410. If it is determined that the "display assistance", the control proceeds to step 422.

  In step S <b> 410, the CPU 102 determines whether the terminal device 200 has been placed on the touch detection unit 112. At this time, the CPU 102 preferably displays a message on the display unit 114. For example, as shown in FIG. 7, a message prompting the user to place the terminal device on the image display apparatus 100 is displayed.

  The CPU 102 can determine that the terminal device 200 has been placed on the touch detection unit 112 from the touch position coordinates acquired as described above. For example, when a plurality of touches are simultaneously detected in an area of a predetermined area or more, it is determined that the terminal device is placed on the image display device 100. If it is determined that the terminal device has been placed, control proceeds to step 412. Otherwise, step 410 is repeated.

  In step 412, the CPU 102 displays a keyboard image and a touch pad image near the area of the display unit 114 located below the area on the touch detection unit 112 where the terminal device 200 is placed. The keyboard image and the touch pad image are displayed by the normal display function of the image display device 100 without performing any treatment such as rotation. For example, as shown in FIG. 8, a keyboard image 610 is displayed below the terminal device 200, and a touch pad image 612 is displayed on the right side thereof. Further, the CPU 102 displays a key for instructing the end of the cooperation function. In FIG. 8, an end key 620 is displayed at the upper right of the screen. In this state, when the user touches the touch detection unit 112, the CPU 102 can specify the touched position as described above.

  In step 414, the CPU 102 determines whether an input operation has been performed. Specifically, it is determined whether any one of the keyboard image 610, the touch pad image 612, and the end key 620 is touched. If it is determined that the input operation has been performed, the control proceeds to step 416. Otherwise, step 414 is repeated.

  In step 416, the CPU 102 determines whether the operation detected in step 414 is a touch on the end key 620. If it is determined that the touch on the end key 620 is made, the control proceeds to step 420, and the display image (keyboard image 610, touch pad image 612 and end key 620) is erased, and information indicating the end (hereinafter, end) Information) is transmitted to the terminal device 200. Control then returns to step 400. If it is determined that the touch on the end key 620 is not made, then the control moves to step 418.

  In step 418, the CPU 102 specifies an instruction corresponding to the touch position from the touch position coordinates acquired in step 414, and transmits corresponding data (such as a character code) to the terminal device 200. Specifically, when the key of the keyboard image 610 is touched, the CPU 102 specifies a character code corresponding to the touched key from the touched position coordinates. When the touch pad image 612 is touched, the CPU 102 specifies data corresponding to the touched position. Since the touch pad image 612 shown in FIG. 8 includes the pad area 618, the left click key 614, and the right click key 616, when the inside of the pad area 618 is touched, the CPU 102 detects the touch detection unit The position coordinates in the pad area 618 corresponding to the position coordinates acquired from 112 are specified. When the left click key 614 or the right click key 616 is touched, the CPU 102 specifies the corresponding code.

  The data transmitted from the image display device 100 is received by the terminal device 200 as described later, and an input process to the terminal device 200 is executed.

  If it is determined in step 408 that “display assistance” is set, in step 422, the CPU 102 causes the display unit 114 to display a predetermined message for executing the display assistance function and a key for instructing the end of the cooperation function. indicate. For example, as shown in FIG. 9, a message and end key 620 are displayed.

  As described later, when the user who sees the above message performs a drag operation on the touch detection unit 212 of the terminal device 200, the terminal device 200 transmits the image data of the displayed screen to the image display device 100. Do.

  In step S 424, the CPU 102 determines whether a touch operation on the touch detection unit 112 has been made. If it is determined that a touch operation has been made, control proceeds to step 426. Otherwise, step 424 is repeated. That is, when the user performs the drag operation on the touch detection unit 212 of the terminal device 200 and then touches the touch detection unit 112 of the image display device 100, the control proceeds to step 426.

  In step 426, the CPU 102 determines whether the operation detected in step 424 is a touch on the end key 620. If it is determined that the touch on the end key 620 is made, the control proceeds to step 420, and after the display screen is erased and the end information is transmitted, the control returns to the step 400. If it is determined that the touch on the end key 620 is not made, the control proceeds to step 428.

  In step 428, the CPU 102 determines whether or not image data has been received from the terminal device 200. If it is determined that it has been received, the control moves to step 430. Otherwise, step 428 is repeated.

  In step 430, the CPU 102 displays, for example, the image data received in step 428 at the touch position detected in step 424 in an enlarged manner. As a result, as shown in FIG. 10, the same image 632 as the image on the screen of the display unit 214 of the terminal device 200 is displayed on the display unit 114 of the image display device 100.

  Thereafter, control returns to step 428. Therefore, each time image data is received from the terminal device 200, the image display device 100 can display it on the screen 600. For example, when the user performs a drag operation while the user operates the terminal device 200 to change the screen of the terminal device 200, new image data is transmitted, and the image display device displays the received image data. . Therefore, for example, after one page of a plurality of page images displayed on the screen of the terminal device 200 is dragged and displayed on the image display device 100, another page is displayed on the screen of the terminal device 200. If you do, two pages can be displayed at the same time.

  The operation of the terminal device 200 will be described. Referring to FIG. 5, in step 500, CPU 202 determines whether or not image display apparatus 100 to be cooperated is detected. If it is determined that it has been detected, control transfers to 502. Otherwise, control proceeds to step 522, where the CPU 202 determines whether termination has been instructed. The termination instruction is issued, for example, by turning off the terminal device 200. If termination is instructed, this program is terminated. Otherwise, control returns to step 500. Thus, the detection process is repeated until the corresponding image display device 100 is detected.

  If the image display apparatus 100 is detected, the CPU 202 determines in step 502 whether or not selection information has been received. As described above, the selection information is information indicating which of the input assistance and the display assistance is selected, which is transmitted from the image display apparatus 100 in step 406. If selection information has been received, control proceeds to step 504. Otherwise, step 502 is repeated.

  In step 504, the CPU 202 branches the control according to the selection information received in step 502. If it is determined that the received selection information corresponds to “input assist” (selection information = 0), the control proceeds to step 506. If it is determined that the received selection information corresponds to “display assistance” (selection information = 1), the control proceeds to step 512.

  In step 506, the CPU 202 determines whether data has been received from the image display apparatus 100. In step 418 described above, data (a character code or the like corresponding to the touched key on the keyboard image 610) corresponding to the user's operation on the image display device 100 is transmitted from the image display device 100. If it is determined that it has been received, the control proceeds to step 508. Otherwise, step 506 is repeated.

  In step 508, the CPU 202 determines whether the data received in step 506 is end information. The end information is transmitted from the image display apparatus 100 in step 420 described above. If it is determined to be end information, control returns to step 500. Otherwise, control transfers to step 510.

  At step 510, the CPU 202 executes processing in accordance with the received data. That is, the CPU 202 passes the received data as input data to the selected application. Control then returns to step 506.

  Therefore, when a touch operation is performed on the keyboard image 610 or the touch pad image 612 displayed on the image display device 100, input processing corresponding to the touch operation is performed on the terminal device 200. For example, when the terminal device 200 receives a character code by operating the keyboard image 610 displayed on the image display device 100 while word processing software is activated and selected in the terminal device 200, the character The code is passed to word processing software, and the corresponding characters are displayed on the display unit 214 by the word processing software. Further, when the touch pad image 612 displayed on the image display device 100 is operated (touched), the received data is passed to the selected application, and the processing corresponding to the operation of the touch pad image 612 is Executed by the selected application.

  When it is determined in step 504 that “display assistance” is to be performed, in step 512, the CPU 202 determines whether a touch operation on the touch detection unit 212 has been performed. If a touch operation is detected, control transfers to step 514. If not, control transfers to step 518 which determines whether termination information has been received. The end information is transmitted from the image display apparatus 100 in step 420 described above. If it is determined to be end information, control returns to step 500. Otherwise, control returns to step 512.

  In step 514, the CPU 202 determines whether a drag operation has been performed on the touch detection unit 212. If a drag operation is detected, control transfers to step 516. Otherwise, control transfers to step 520.

  As described above, when a message as shown in FIG. 9 is displayed on the image display device 100 and the user wants to display the screen of the terminal device 200 on the image display device 100, the user uses the terminal device 200. Do a drag operation.

  In step 516, the CPU 202 reads out the image data displayed on the screen of the display unit 214 from the VRAM 218 and transmits the image data to the image display apparatus 100. The transmitted image data is received by the image display apparatus 100 and displayed on the display unit 114 (see step 428 and step 430).

  Thereafter, the control returns to step 514, and each time a drag operation is performed on the touch detection unit 212, the image data displayed on the screen of the display unit 214 is transmitted to the image display apparatus 100.

  If the operation detected in step 512 is not a drag operation, in step 520, the CPU 202 executes processing corresponding to the operation. Control then returns to step 512.

  When the display auxiliary function is selected, the user can operate the terminal device 200 to change the screen in steps 512 to 520, and if the drag operation is performed after the screen is changed, the screen is newly displayed. The displayed image data is transmitted and displayed on the image display device 100.

  As described above, the image display device 100 can provide the input assistance function and the display assistance function to the terminal device 200 by cooperation between the image display device 100 and the terminal device 200. That is, the user can use the entire screen of the terminal device 200 for display without displaying a keyboard image or the like on the small screen of the terminal device 200 and using a part of the screen for input by the input assistance function. can do. Since a keyboard or the like of a normal size is displayed on the large screen of the image display apparatus 100, the user can easily perform an input operation. Further, the keyboard or the like may not be carried along with the terminal device 200. Furthermore, since the screen of the terminal device 200 can be enlarged and displayed on the image display device 100 by the display assistance function, the user can efficiently perform meetings, presentations, etc. while operating the terminal device 200. it can.

  In the above, in the input assistance function, only the input processing has been described, but the keyboard image 610 and the touch pad image 612 may be movable to any position. For example, if the keyboard image 610 or a particular portion of the touch pad image 612 (eg, the top of the image) is touched and dragged while the touch is maintained, the image of the subject may be moved. FIG. 11 shows a state in which the positions of the keyboard image 610 and the touch pad image 612 have been changed from the arrangement of FIG. This enables the user to perform input operations more easily and efficiently.

  Similarly, in the display assistance function, the display position of the image 632 of the screen of the terminal device 200 displayed on the image display device 100 may be changed by a drag operation or the like on the touch detection unit 112.

  Although the case where cooperation is ended when the end key 620 displayed on the image display device 100 is touched has been described above, the present invention is not limited to this. For example, when the terminal device 200 is removed from above the touch detection unit 112 of the image display device 100, the cooperation may be ended.

  In the above, in the display assistance function, after the user performs a drag operation on the terminal device 200, an image of the screen of the terminal device 200 is displayed at the touch position by performing a touch operation on the image display device 100. Although the case has been described, it is not limited thereto. For example, when the user performs a drag operation with a finger on the touch detection unit 212 of the terminal device 200, moves the finger to the touch detection unit 112 of the image display device 100 as it is, and then performs a drag operation. The image of the screen of the terminal device 200 may be displayed on the image display device 100 in response to the drag operation. Specifically, as shown in FIG. 12, the user touches the right end portion of the terminal device 200 on the left side of the screen of the image display device 100 (the upper surface of the touch detection unit 112). A drag operation (arrow on the left) is performed in the right direction, the finger is moved in the right direction, the image display apparatus 100 is touched, and a drag operation is performed while the touch is maintained (arrow on the right). According to this operation, the CPU 202 transmits the image data of the screen of the terminal device 200 to the image display device 100 by dragging on the terminal device 200. The CPU 102 displays the right end portion of the received image at the touch position (end of the right arrow) and moves the right end of the image to the current touch position (tip of the right arrow) according to the continuation of the drag operation thereafter. A portion 630 of the received image is displayed in the range up to the first touch position (end of the right arrow). When the dragging is further continued, as shown in FIG. 13, an image 632 of the entire screen of the terminal device 200 is displayed on the image display device 100. By such an operation, cooperation between the image display device 100 and the terminal device 200 can be more effectively produced, and the other party can be impressed.

  Further, although the case where the image data of the screen of the terminal device 200 is transmitted by the user performing the drag operation in the display assist function has been described above, the operation may be an operation other than the drag operation. In addition, whenever the screen display of the terminal device 200 is changed, the image data of the screen of the terminal device 200 may be automatically transmitted without the user performing a drag operation. In this way, the user's operation can be reduced and it is more efficient.

  Although the case has been described above where the user selects and uses one of the input assistance function and the display assistance function, both functions may be simultaneously available. For example, in FIG. 4, if the user selects the input assistance in step 404, a window for selecting whether to use the display assistance function after displaying the keyboard image 610 and the touch pad image 612 in step 412 is displayed. You may display it. If the user selects to use the display assistance function, steps 422 to 430 may be executed. Similarly, if the user selects display assistance in step 404, after displaying the screen image of the terminal device in step 430, a window may be displayed to select whether to use the input assistance function. . If the user selects to use the input assistance function, steps 410 to 418 may be executed. Then, as shown in FIG. 14, the user performs an input by performing a touch operation on the keyboard image 610 and the touch pad image 612 in a state where the screen of the terminal device 200 is enlarged and displayed. be able to. At this time, as described above, if the image data of the screen of the terminal device 200 is automatically transmitted whenever the screen display of the terminal device 200 is changed, the keyboard image 610 and the touch pad image 612 can be transmitted. The touch operation is automatically reflected on the image 632 displayed on the image display device 100, which is efficient.

  Moreover, although the case where the terminal device 200 was mounted on the image display apparatus 100 so that the screen of the terminal device 200 may become parallel with the screen of the image display apparatus 100 above was demonstrated, it is not limited to this. For example, the terminal device 200 may be placed on the touch detection unit 112 of the image display device 100 using a stand or the like for arranging the terminal device 200 upright.

  Although the case of displaying the keyboard image and the image of the touch pad has been described above, the present invention is not limited to this. Only one of the keyboard image and the touch pad image may be displayed. In that case, the user may select which of the keyboard image and the touch pad image to display.

Second Embodiment
In the first embodiment, the same image as the screen image of the terminal device is enlarged and displayed on the image display device, for example. However, in the second embodiment, the screen image of the terminal device is , Configure a part of the image of the screen of the image display device. That is, the screen of the terminal device complements the screen of the image display device.

  The image display device and the terminal device according to the second embodiment are configured in the same manner as the image display device and the terminal device according to the first embodiment shown in FIGS. 1 to 3. In the following, the same reference numerals as in FIGS. 1 to 3 are used. The program executed by the image display device and the terminal device according to the second embodiment is different from the first embodiment.

  The cooperation between the image display device 100 and the terminal device 200 according to the second embodiment will be described with reference to FIGS. 15 and 16. The flowchart shown in FIG. 15 differs from the flowchart shown in FIG. 4 only in the addition of steps 700 to 704. In FIG. 15, for the sake of convenience, steps 422 to 430 in FIG. 4 are collectively described as step 706 (display assistance processing). Further, the flowchart shown in FIG. 16 is different from the flowchart shown in FIG. 5 only in the addition of steps 720 to 726. In FIG. 16, for convenience, steps 512 to 520 in FIG. 5 are collectively described as step 728 (display auxiliary terminal side process).

  Referring to FIG. 15, after input assistance is selected and keyboard image 610 and touch pad image 612 are displayed (step 412), if an input operation is performed (the determination result in step 414 is YES), step 700. The CPU 102 determines whether the predetermined area has been touched. The predetermined area is an area where the keyboard image 610, the touch pad image 612, and the end key 620 are displayed. If it is determined that the touch on the predetermined area is made, the control proceeds to step 416. Otherwise, control transfers to step 702.

  In step 702, the CPU 102 determines whether image data has been received from the terminal device 200. Reception of image data is performed by another program being executed in parallel with the present program, and the received image data is stored in the RAM 106 or the recording unit 108. If image data has been received, control proceeds to step 704. If not, control returns to step 414. Transmission of image data from the terminal device 200 is instructed by a drag operation on the screen of the terminal device 200, as described later.

  In step 704, the CPU 102 displays the received image data on the display unit 114 so as to include the area in which the terminal device 200 is mounted. For example, as shown in FIG. 17, the image 640 is displayed so that the area where the terminal device 200 is placed is included in the lower left portion. Control then returns to step 414.

  17, the image data of the screen of the terminal device 200 is transmitted to the image display device 100 by performing the drag operation as indicated by the arrow on the screen of the terminal device 200, and the display unit 114 of the image display device 100. The same image 640 as the screen of the terminal device 200 is displayed by touching the area on the touch detection unit 112 corresponding to the area where nothing is displayed.

  Referring to FIG. 16, after data corresponding to the input assistance is received (determination result in step 504 is YES), when data from image display device 100 is not received (determination result in step 506 is NO). In step 720, the CPU 202 determines whether a touch operation has been performed on the touch detection unit 212. If a touch operation is detected, control transfers to step 722. Otherwise, step 720 is repeated.

  In step 722, the CPU 202 determines whether a drag operation has been performed on the touch detection unit 212. If a drag operation is detected, control transfers to step 724. Otherwise, control transfers to step 726.

  In step 724, the CPU 202 reads the image data displayed on the screen of the display unit 214 from the VRAM 218 and transmits the image data to the image display apparatus 100. The transmitted image data is received by the image display apparatus 100 and displayed on the display unit 114 as described above (see step 702 and step 704). Control then returns to step 506.

  If the operation detected in step 720 is not a drag operation, in step 726, the CPU 202 executes processing corresponding to the operation. Control then returns to step 506.

  Therefore, when the input assistance is selected (the determination result in step 408 is YES), the user operates terminal apparatus 200 to change the display of the screen according to steps 700 to 704 and steps 720 to 726. If a drag operation is performed after the screen display is changed, image data of a new screen is transmitted and displayed on the image display apparatus 100.

  Even if the terminal device 200 is removed from above the image display apparatus 100 from the state of FIG. 17 as shown in FIG. 18 (only the left end portion of the removed terminal device 200 is shown), the terminal device The display of the same image 640 as the image of the 200 screen is maintained. Therefore, the user can perform the touch operation on the keyboard image 610 and the touch pad image 612 while looking at the image 640 displayed by enlarging the screen of the terminal device 200, so that the input operation can be performed efficiently. be able to.

  Although the case where the display of the same image 640 as the screen of the terminal device 200 is maintained even if the terminal device 200 is removed from the top of the image display device 100 has been described above, as shown in FIG. When 200 is removed from the top of the image display device 100, the portion where the terminal device 200 was placed may not be displayed. That is, when displaying the received image data in step 704, the data of the partial image displayed in the area where the terminal device 200 is placed among the image data is another data (for example, black etc.). (Displayed on the display unit 114). In FIG. 19, the portion 644 where the terminal device 200 was placed is displayed in single color (grayed out). In this way, for example, as shown in FIG. 20, after removing the terminal device 200, the user can display an image 802 which the user does not want to show to the other party on the terminal device 200, and can examine it. In FIG. 20, confidential information is displayed in the window of the application being executed by the terminal device 200, but the information is not displayed on the image display device 100. In this state, even if a drag operation is performed on the terminal device 200 and image data of a new screen of the terminal device 200 is transmitted, the image display device as long as the position of the window in which the confidential information is displayed is not changed. The image data of the new screen of the terminal device 200 is displayed on the screen 100 with the confidential information removed.

Third Embodiment
The image display apparatus according to the first and second embodiments cooperates with one terminal device to provide an input assistance function to the terminal device. The image display apparatus according to the third embodiment cooperates with a plurality of terminal devices to provide an input assistance function to each terminal device.

  The image display apparatus according to the third embodiment is configured in the same manner as the image display apparatus according to the first embodiment shown in FIG. Further, each of the plurality of terminal devices is configured in the same manner as the terminal device shown in FIG. Programs executed by the image display device and the terminal device according to the third embodiment are different from the first and second embodiments.

  The case where the image display apparatus 100 according to the third embodiment provides an input assistance function to two terminal apparatuses will be described with reference to FIGS. 21 and 22. In the following, the same reference numerals as in FIG. 1 are used for the image display device. The two terminal devices use the reference numerals in FIG. 3 after specifying the first terminal device and the second terminal device as described later. In FIGS. 21 and 22, steps given the same reference numerals as in FIGS. 4 and 5 are the same as the steps in FIGS. 4 and 5.

  Referring to FIG. 21, after the target terminal device providing the cooperation function is detected in step 400, in step 740, CPU 102 provides display unit 114 with the input assistance function for the two terminal devices. Display an image that contains a message indicating that it is possible. Specifically, as shown in FIG. 23, on the screen 600, a window 660 including a message and a key 662 for two machines are displayed.

  At step 742, the CPU 102 determines whether or not two units are selected. The use of two units is selected by touching the two unit key 662. If it is determined that the two units are selected, the control proceeds to step 744. Otherwise, control transfers to step 410.

  The image display apparatus 100 needs to identify two terminal devices. For example, after power on, activation of a program, and placement on the image display apparatus 100 for one terminal apparatus, power on, activation of a program, and placement on the image display apparatus 100 for another terminal apparatus. If placement is performed, the image display apparatus 100 can identify two terminal devices and associate the two terminal devices with the displayed two sets of keyboard images and touch pad images. In addition, when two terminal devices are turned on and the dedicated program is activated and communication with the image display apparatus 100 becomes possible, when the terminal devices are installed, the two terminal devices are identified. The terminal device may be made to correspond to the keyboard image and the touch pad image by displaying the information (device name and the like) to be selected by the user.

  In step 744, the CPU 102 displays a boundary dividing the screen 600 into two. For example, as shown in FIG. 24, a dashed border 664 is displayed. In FIG. 24, the two touched keys 662 are displayed with inverted luminance to indicate that they are selected.

  In step 746, the CPU 102 determines whether the terminal device is placed on the touch detection unit 112. When it is placed, the CPU 102 adds “1” to the counter (initial value is “0”), and the control proceeds to step 748. If not, step 746 is repeated. Hereinafter, the terminal device placed first is referred to as a first terminal device.

  At step 748, the CPU 102 displays a keyboard image and a touch pad image in the vicinity of the placed first terminal device. Specifically, the keyboard image and the touch pad image are displayed between the side on which the first terminal device is placed parallel to the boundary 664 and the position where the first terminal device is placed. For example, as shown in FIG. 25, when the first terminal device 222 is placed, the keyboard image 670 and the touch pad image 672 are displayed.

  In step 750, the CPU 102 determines whether all the two terminal devices have been placed. When the second terminal device (hereinafter referred to as the second terminal device) is placed, step 746 is executed again, the addition processing of the counter is executed, and the value of the counter becomes “2”. It is determined whether or not "2". If it is determined that all the sheets have been placed, the control proceeds to step 414. Otherwise, control returns to step 746.

  The state in which the two terminal devices are placed is as shown in FIG. In FIG. 26, as described above, the keyboard image 670 and the touch pad image 672 are displayed in the area 674 in which the first terminal device 222 is placed, and corresponds to the area 676 in which the second terminal device 224 is placed. A keyboard image 678 and a touch pad image 680 are displayed.

  As a result of the determination in step 742, if the two-unit key 662 is not touched, the CPU 102 determines whether or not the terminal device is placed on the touch detection unit 112 in step 410.

  In step 414, the CPU 102 determines whether an input operation has been performed on the keyboard image and the touch pad image. If the input operation has been performed, the CPU 102 corresponds to the touch position from the touch position coordinates acquired in step 414 in step 418. The instruction is specified, and the corresponding data (such as a character code) is transmitted to the corresponding one of the first terminal 222 and the second terminal 224. In the case where step 414 is executed without the two keys 662 being touched (this can be determined by whether or not the value of the counter is the initial value "0"), in the following, with respect to one terminal device Processing is performed.

  In step 752, the CPU 102 determines whether the terminal device has been removed from the top of the image display device. If it is determined that it has been removed, then control transfers to step 754. If not, control returns to step 414.

  In step 754, the CPU 102 deletes the keyboard image, the touch pad image, the boundary line, and the two keys 662 from the area where the terminal device has been removed, and transmits the end information to the terminal device. At this time, if the counter is not "0", the CPU 102 subtracts the counter by "1".

  In step 756, the CPU 102 determines whether all the terminal devices have been removed. This is determined by whether the counter is an initial value ("0"). If it is determined that two terminals have been removed, control returns to step 400. If not, control returns to step 414.

  Referring to FIG. 22, each step of a program executed by the first terminal device and the second terminal device is the same as the step related to the input assistance function among the steps shown in FIG. That is, as described above, after the communication with the image display apparatus 100 is established, the first terminal apparatus and the second terminal apparatus receive the input data addressed to the own apparatus from the image display apparatus 100 and respond to the input data. Execute the process.

  In this manner, if one terminal device is placed in a state where the two-unit key 662 is not touched (NO at step 742 and YES at step 410), 1 as in the first embodiment, 1 It is possible to provide an input assistance function to one or more terminal devices. On the other hand, when the two device keys 662 are touched and the two terminal devices are placed, the image display apparatus 100 can provide the input assistance function to the two terminal devices. When one of the mounted terminals is removed, the corresponding keyboard image and touch pad image are erased. In that state, if there is a terminal device mounted, the input assistance function for that terminal device is maintained. When all terminals are removed. The image is erased from the screen of the image display device 100, and the initial state is restored.

  Although the case where the user has provided the input assistance function to the two terminal devices by touching the two-piece key 662 has been described above, the present invention is not limited to this. For example, the image display device establishes communication with the terminal device each time the terminal device is placed on the image display device without displaying the screen as shown in FIG. 23, and a keyboard image and a touch are displayed in the vicinity of the terminal device. If there is a space for displaying pad images, they may be displayed.

  Further, the method of dividing the screen 600 of the image display device is not limited to the dividing method as shown in FIG. For example, as shown in FIG. 27, the diagonal position of the screen 600 may be used.

  Further, if the screen of the image display device is large, three or more terminal devices may be provided with the input assistance function.

  The present invention has been described above by describing the embodiment, but the above-described embodiment is an exemplification, and the present invention is not limited to the above-described embodiment, and various modifications may be made. be able to.

100 image display devices 102 and 202 arithmetic processing unit (CPU)
104, 204 Read Only Memory (ROM)
106, 206 Rewritable Memory (RAM)
108, 208 recording unit 110, 210 communication unit 112, 212 touch detection unit 114, 214 display unit 116, 216 display control unit 118, 218 video memory (VRAM)
120, 220 bus 600 screen 610, 670, 678 keyboard image 612, 672, 680 touch pad image 620 end key

Claims (14)

A display unit in which the image display surface is arranged horizontally,
A detection unit disposed on the image display surface of the display unit and detecting a touched position of the image display surface;
The display unit receives the fact that the detection unit detects that the external device has been placed on the detection unit, and displays the image of the input device for inputting information to the external device. Display a screen for selecting the image on the image display surface,
The image display apparatus, wherein the display unit displays an image of the input device on the image display surface in response to detection that the detection unit has selected to display the image of the input device.   In response to the detection unit detecting that the display unit has selected to display the image of the input device, the display unit is arranged on an area different from the area where the external device is placed on the image display surface, The image display apparatus according to claim 1, wherein an image of the input device is displayed.   The display unit enlarges the image data displayed on the image display surface of the external device, which is received from the external device in response to detection that the detection unit detects that the image display surface is touched. The image display apparatus according to claim 1, wherein the image display is performed. The display unit is
Image data displayed on the image display surface of the external device received from the external device in response to the detection unit detecting that the image display surface is touched in a state where the external device is mounted In an area including a portion corresponding to the placement area on which the external device is placed;
In response to the detection unit detecting that the external device has been removed from above the detection unit in the magnified state, the partial image displayed in the portion corresponding to the placement area is displayed. The image display apparatus according to any one of claims 1 to 3, wherein the image is changed to a different partial image. The display unit is
Image data displayed on the image display surface of the external device received from the external device in response to the detection unit detecting that the image display surface is touched in a state where the external device is mounted In an area including a portion corresponding to the placement area on which the external device is placed;
The enlarged display according to any one of claims 1 to 3, wherein the enlarged display is maintained even if the detection unit detects that the external device is removed from above the detection unit in the enlarged display state. Image display device.   The display unit mounts each of the plurality of external devices in response to the detection unit detecting that each of the plurality of external devices receives an instruction to display the input device. The image display apparatus according to any one of claims 1 to 5, wherein a boundary of the area is displayed.   The image display apparatus according to any one of claims 1 to 6, wherein the image indicating the input device includes a keyboard image or a touch pad image. An image display method for displaying an image on a horizontally arranged image display surface, comprising:
An external device detection step of detecting the position of the external device placed on the image display surface;
A selection screen display step of displaying a screen for selecting whether or not to display an image of an input device for inputting information to the external device on the image display surface after detecting the position of the external device;
An input device image selection detection step of detecting that the display of the image of the input device is selected on the image display surface;
An input device image display step of displaying an image indicating the input device after detecting that the display of the image of the input device is selected.   The image display according to claim 8, wherein an image of the input device is displayed on the image display surface in an area different from the area where the external device is placed on the image display surface in the input device image display step. Method. A touch detection step of detecting a touch on the image display surface;
In response to detecting that the image display surface is touched in the touch detection step, the image data displayed from the external device and displayed on the image display surface of the external device is enlarged and displayed. The image display method according to claim 8, further comprising: a display unit display step. The external device received from the external device in the display unit display step in response to the detection that the image display surface is touched in the touch detection step in a state where the external device is placed. Image data displayed on the image display surface of the image display area is enlarged and displayed in an area including a portion corresponding to the position at which the external device is mounted,
In the enlarged display state, the external device detection step detects that the external device has been removed from the top of the image display surface, and the display unit displays the external device in the display step. The image display method according to claim 10, wherein a partial image displayed in a portion corresponding to a position at which is placed is changed to a different partial image. In the state where the external device is placed, receiving that the image display surface has been touched in the touch detection step, and in the display unit display step, the external device received from the external device Image data displayed on the image display surface is enlarged and displayed in an area including a portion corresponding to the position at which the external device is mounted;
Even if it is detected in the external device detection step that the external device has been removed from the image display surface in the enlarged display state, the enlarged display is maintained in the display unit display step. The image display method according to claim 10. A touch detection step of detecting a touch on the image display surface;
In the touch detection step, in response to an instruction to display the input device to each of the plurality of external devices, a boundary line of a region on which each of the plurality of external devices is to be placed is displayed. The image display method according to claim 8, further comprising: a boundary line display step.   The image display method according to any one of claims 8 to 13, wherein the image indicating the input device includes a keyboard image or a touch pad image.

Images