JP2013114593A - Image display device, control method thereof, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a new image display device that maintains usability even when handwriting and operation are frequently switched therebetween, and to provide a control method thereof and a program.SOLUTION: In the present invention, an ON/OFF switch is provided at a pen device used for drawing and operation of an electronic blackboard. If the switch of the pen device is turned off, a mouse event is transferred to a handwriting drawing section. If the switch is turned on and a coordinate where the mouse event occurs is not in a region to inset and display a screen of a user's PC, the mouse event is transferred to a GUI control unit. If the switch is turned on and the coordinate where the mouse event occurs is in the region to inset and display the screen of the user's PC, the coordinate of the mouse event is converted into a screen coordinate system of the user's PC and transferred to the user's PC.

Description

  The present invention relates to an image display apparatus, a control method thereof, and a program.

  In recent years, so-called “electronic blackboards” using large displays have been used in conferences and the like. Japanese Patent Laying-Open No. 2006-331263 (Patent Document 1) discloses an electronic blackboard device that allows a user to handwrite letters, numbers, figures, etc. on a background image displayed on a large display. Currently, the screen image of the user's notebook computer can be transferred to a large display and displayed, and the user's laptop computer can be remotely operated by touching the displayed screen image of the user PC. Blackboard products are provided.

  However, when switching between the handwritten drawing mode and the other operation modes, it is necessary for the conventional electronic blackboard to instruct the mode switching through the operation panel or the like, so that both handwriting and operation are frequently performed. In such a case, there was a problem in usability in that the pen tip had to be moved each time the switch was made.

  The present invention has been made in view of the above problems in the prior art, and the present invention is a novel image display device that does not impair usability even in a case where both handwriting and operation are frequently performed. It is an object to provide a control method and program thereof.

  As a result of intensive studies on a novel image display device that does not impair usability even when the handwriting and the operation are frequently performed, the present inventor has conceived the following configuration and led to the present invention. It was.

  That is, according to the present invention, there is provided an image display control device that controls a display device in which a pointing device is disposed on a display device, and a screen instruction device having an ON / OFF switch contacts or approaches the pointing device. In response to this, a pointing device event is generated, an event generating unit that generates an ON / OFF event based on an ON / OFF signal received from the screen instruction device, and a graphic based on the pointing device event A handwritten drawing unit that displays a background image on the display device, captures a screen screen of an information processing apparatus connected to the outside, displays the screen image in a predetermined inset area in the background image, and the background image or the predetermined image Inset A video processing unit that superimposes and displays the graphics generated by the handwritten drawing unit, and a GUI that displays a GUI in the background image or the predetermined inset area and receives a predetermined operation based on the pointing device event An event for receiving the pointing device event and the ON / OFF event from the control unit and the event generation unit, and transferring the pointing device event to any one of the handwriting drawing unit, the GUI control unit, and the information processing apparatus An image display control device including a distribution unit is provided.

1 is a diagram illustrating an appearance of an image display device according to an embodiment. 1 is a hardware configuration diagram of an image display apparatus according to an embodiment. The block diagram of the pen device in this embodiment. 1 is a functional block diagram of an image display apparatus according to an embodiment. The flowchart which shows the process which an event distribution part performs. The conceptual diagram for demonstrating drawing mode. The conceptual diagram for demonstrating the judgment process which an event distribution part performs. The conceptual diagram for demonstrating GUI operation mode. The conceptual diagram for demonstrating the coordinate transformation process which an event distribution part performs. The conceptual diagram for demonstrating PC operation mode. 1 is a diagram illustrating an appearance of an image display device according to an embodiment. The flowchart which shows the process which an event distribution part performs. The conceptual diagram for demonstrating the judgment process which an event distribution part performs. The conceptual diagram for demonstrating PC operation mode.

  Hereinafter, although this invention is demonstrated with embodiment, this invention is not limited to embodiment mentioned later. In the drawings referred to below, the same reference numerals are used for common elements, and the description thereof is omitted as appropriate.

  FIG. 1 shows an appearance of an image display apparatus 100 according to an embodiment of the present invention. The image display device 100 according to the present embodiment is a device referred to as a so-called electronic blackboard, and includes a large display device 102 and an image display control device (not shown) for controlling the display device 102. Has been.

  As shown in FIG. 1, in the display apparatus 102 of the present embodiment, a background image (in the example, a uniform color background) is displayed as a desktop on the screen screen 104, and the desktop is provided as a drawing canvas.

  On the other hand, when using the image display apparatus 100, the user is provided with a dedicated screen instruction device 200. The screen instruction device 200 according to the present embodiment is not limited to a specific form as long as the user can hold the desired position on the screen screen 104 with the hand, and is preferably a pen-type device. It can be a shape. In the following, a case where a screen instruction device 200 having a pen shape (hereinafter referred to as the pen device 200) is employed will be described as an example.

  When the user moves the pen tip of the pen device 200 while making contact with the screen screen 104, in response to this, the movement locus is displayed on the desktop as a drawing line. That is, as shown in FIG. 1A, the user can freely write handwritten characters and figures using the pen device 200 on the screen screen 104 that looks like a blackboard.

  On the other hand, a GUI control component 106 for operating the image display device 100 is displayed at the right corner of the desktop displayed on the display device 102, and the user can use the GUI control component 106 with the pen tip of the pen device 200. By designating, a desired operation can be performed. Note that the GUI control component 106 shown in FIG. 1 is merely an example, and in this embodiment, any form of GUI control component that can be assumed by those skilled in the art may be employed.

  Further, as shown in FIG. 1B, the image display apparatus 100 according to the present embodiment captures drawing data for drawing a screen screen 302 of a user's personal computer 300 (hereinafter referred to as PC 300). It can be displayed on the screen screen 104 of the display device 102. Specifically, the image display device 100 displays the same image as the image displayed on the screen screen 302 of the PC 300 on the screen screen 104 based on the VGA signal transmitted from the PC 300 via the VGA cable 304. It can be displayed in a predetermined size.

  FIG. 2 is a hardware configuration diagram of the image display apparatus 100 according to the present embodiment. The image display device 100 includes a processor 150, a ROM 151, a RAM 152, a hard disk device (HDD) 153, a graphics board 154, a display device 155, a coordinate detection device 156, a wireless communication interface 157, and an external device connection. And an interface 158.

  The processor 150 is a device that calculates the processing executed by the image display apparatus 100, and various processors such as a CPU such as PENTIUM (registered trademark) and Athlon, and an MPU can be employed. The ROM 151 is a memory that stores a boot program such as BIOS.

  The RAM 152 is a memory that provides an execution space for executing the program of the present embodiment. The image display apparatus 100 expands and executes a program written in a program language such as assembler, C, C ++, Java (registered trademark), Java (registered trademark) Script, PERL, RUBY, or PYTHON under the management of the OS. Thus, each functional means described later is realized.

  The hard disk device 153 includes various operating systems (such as WINDOWS (registered trademark) series, UNIX (registered trademark), LINUX (registered trademark), Android (registered trademark), ITRON, and μITRON that control the entire system of the image display device 100. OS), various device drivers, application programs, and the like.

  The external device connection interface 158 is an interface for connecting an external device such as the user's PC 300, and is a physical interface such as a VGA input connector or a USB port.

  The graphics board 154 is hardware that processes drawing data to be drawn on the display device 155, and includes a VRAM that holds an image and a connection terminal of the display device 155. The display device 155 is a device that draws and displays drawing data generated by the graphics board 154.

  The coordinate detection device 156 is a device for detecting the position coordinates at which the pen device 200 is in contact with or approaching (hereinafter referred to as contact or the like). In the present embodiment, the coordinate detection device 156 is a touch panel using a capacitance sensor that detects a capacitance that changes with contact of an object or the like, or a touch panel that uses a pressure sensor that detects the contact pressure of an object. Can be configured.

  Furthermore, in the present embodiment, the coordinate detection device 156 can be configured using an optical technique. As an apparatus using an optical technique, for example, a coordinate input / detection apparatus disclosed in a previous application (Japanese Patent Laid-Open No. 2008-176802) of the present applicant can be cited. In the coordinate input / detection device, the coordinate parallel to the display surface is detected by irradiating light parallel to the display surface and detecting that the light is blocked by the pen.

  Note that the coordinate detection device 156 in the present embodiment may take any form as long as it is a panel-like pointing device that can detect the coordinates of the position touched by the pen device 200. The configuration of the coordinate detection device 156 is not limited.

  The image display apparatus 100 according to the present embodiment performs wireless communication with the pen device 200, and the wireless communication I / F 157 is an interface for receiving a wireless signal transmitted from the pen device 200. As the wireless communication method, for example, infrared communication can be employed, and other suitable methods such as Bluetooth (registered trademark) and WiFi may be employed.

  FIG. 3 shows a configuration diagram of the pen device 200 in the present embodiment. The pen device 200 includes, for example, a pen-type housing, and a mechanical push button 202 is disposed at an appropriate position on the outer peripheral surface thereof. In the present embodiment, the push button 202 is preferably disposed at an appropriate position where the push device 202 can be easily pressed without re-gripping the pen device 200. The signal generation unit 204 generates a “true” signal when the push button 202 is pressed, and generates a “false” signal when the push button 202 is not pressed. The generated signal is transmitted to the display apparatus 102 by a signal transmission unit 208 referred to as a wireless transmitter.

  Note that the push button 202 shown in FIG. 3 is merely an example, and the pen device 200 according to the present embodiment is essentially provided with a mechanism for defining binary information such as ON / OFF. Instead of the push button 202, a slide switch or a rotary switch can be adopted, and a switch using various sensors such as an optical sensor or a pressure sensor can also be adopted. In the following description, a true / false signal generated according to whether or not the push button 202 is pressed is referred to as an ON / OFF signal.

  The hardware configurations of the image display apparatus 100 and the pen device 200 according to the present embodiment have been described above. Next, the display apparatus 102 that constitutes the image display apparatus 100 and the image display control apparatus that controls the display apparatus 102 will be described. This will be described with reference to FIG. In the following description, the coordinate detection device 156 is referred to as the touch panel 156.

  FIG. 4 shows a functional block diagram of the display device 102 and the image display control device 120. The display apparatus 102 includes a display device 155, a touch panel 156 disposed on the display device 155 so as to cover the entire screen screen 104, and a signal receiving unit 110 for receiving a wireless signal transmitted from the pen device 200. It is configured to include. The signal receiving unit 110 is not limited to the inside of the housing of the display apparatus 102 as long as it can receive a wireless signal transmitted from the pen device 200, and may be disposed at any position. The display control device 120 may be disposed in the housing.

On the other hand, the image display control device 120 includes a controller unit 121 and a VGA for receiving a VGA signal transmitted from the PC 300.
The controller unit 121 includes an I / F 128 and a USB I / F 129 for transferring an event signal to the PC 300. The controller unit 121 includes a touch panel driver unit 122, an operating system 123 that controls the entire controller unit 121, The video processing unit 124, the handwriting drawing unit 125, the GUI control unit 126, and the event distribution unit 130 are configured.

  The touch panel 156 generates a signal indicating the position coordinates on the touch panel 156 in response to the touch of the pen device 200, and transmits the signal to the touch panel driver unit 122. On the other hand, the signal receiving unit 110 transmits the ON / OFF signal received from the pen device 200 to the touch panel driver unit 122.

  The touch panel driver unit 122 converts a signal indicating the position coordinates received from the touch panel 156 into a predetermined pointing device event (hereinafter referred to as a touch panel event). Further, the touch panel driver unit 122 converts the ON / OFF signal received from the signal receiving unit 110 into a corresponding predetermined event (hereinafter referred to as an ON / OFF event). The ON / OFF event can be assigned as a function key key event. The touch panel driver unit 122 associates the converted touch panel event with the ON / OFF event and sends it to the operating system 123.

  The operating system 123 generates a predetermined pointing device event (hereinafter referred to as a mouse event) indicating the coordinates of the screen coordinate system of the display device 155 based on the touch panel event received from the touch panel driver unit 122. That is, the operating system 123 functions as an event generation unit. When the operating system 123 generates a mouse event, the operating system 123 sends it to the event distribution unit 130 together with the received ON / OFF event.

The video processing unit 124 receives the VGA from the PC 300.
Based on the VGA signal transmitted via the I / F 128, a layer for displaying the screen screen 302 of the PC 300 is generated and superimposed on a desktop (drawing campus) layer displayed on the screen screen 104. As a result, the screen screen 302 of the PC 300 is displayed in a predetermined position in the desktop.

  The handwriting drawing unit 125 is a functional unit referred to as a drawing application, generates graphics such as a handwritten line based on a mouse event transferred from the event distribution unit 130 described later, and passes the graphics to the video processing unit 124. The video processing unit 124 generates a layer for displaying the received graphics, and superimposes it on the other layers (the layer for displaying the desktop or the layer for displaying the screen screen 302 of the PC 300).

  The GUI control unit 126 is a functional unit that is referred to as a GUI program, displays a GUI control component on the desktop, and accepts a user operation via the GUI control component. The GUI control unit 126 waits for a mouse event transferred from the event distribution unit 130, which will be described later, and responds to the occurrence of a mouse event that selects a predetermined GUI control component, in response to a predetermined event associated with the GUI control component. The operation of is accepted.

  The event distribution unit 130 includes an external input image management unit 132, an ON / OFF determination unit 134, a coordinate acquisition unit 136, and a coordinate conversion unit 138.

  The external input image management unit 132 manages the coordinates (the coordinate information of the screen coordinate system of the display device 155) of the area in which the screen screen 302 of the PC 300 transmitted from the PC 300 is displayed. The external input image management unit 132 holds the value when the area in which the screen screen 302 is fitted and displayed is fixed, that is, when the coordinate of the area to be displayed is a fixed value, and the area to be fitted and displayed. Is variably set, the coordinate information of the area is inquired and acquired from the setting source (for example, the video processing unit 124).

  The ON / OFF determination unit 134 refers to the ON / OFF event received from the operating system 123, and determines whether or not the push button 202 of the pen device 200 is pressed (that is, whether the switch is ON / OFF). On the other hand, the coordinate acquisition unit 136 acquires coordinate information (mouse event occurrence coordinates) included in the mouse event received from the operating system 123.

  Based on the mouse event and the switch ON / OFF event received from the operating system 123 and the coordinate information of the area in which the screen screen of the PC 300 is inserted and displayed, the event distribution unit 130 is based on the handwriting drawing unit 125, the GUI control unit 126, and the PC 300 ( Any one of the externally connected PCs) is determined as the transfer destination of the mouse event. When the PC 300 determines the transfer destination of the mouse event, the event distribution unit 130 transfers the mouse event to the PC 300 via the USB I / F 129.

  The functional units constituting the controller unit 121 of the image display control apparatus 120 of the present embodiment have been described above. Next, the processing executed by the event distribution unit 130 is specifically described based on the flowchart shown in FIG. explain. In the following description, the image display device 100 is referred to as the electronic blackboard 100.

  When the image display control device 120 is activated, the event distribution unit 130 monitors a mouse event generated by the operating system 123 (No in step 101). When the operating system 123 detects that a mouse event has been generated (step 101, Yes), the mouse event and an ON / OFF event associated therewith are received, and the process proceeds to step 102.

  In step 102, the ON / OFF determination unit 134 refers to the ON / OFF event, and determines whether the switch of the pen device 200 is ON. If it is determined that the switch of the pen device 200 is not turned on (No at Step 102), the process proceeds to Step 103, where the event distribution unit 130 transfers the mouse event received from the operating system 123 to the handwriting drawing unit 125. Thereafter, the process returns to the mouse event monitoring loop (step 101).

  The handwriting drawing unit generates graphics such as a handwritten line based on the mouse event received from the event distribution unit 130. In other words, in the present embodiment, while the pen device 200 is switched off, the electronic blackboard 100 always operates in the drawing mode, and the user performs handwriting drawing using the entire area of the desktop screen as the drawing canvas. be able to. Since the graphics generated by the handwritten drawing unit are displayed on the uppermost layer, as shown in FIG. 6A, the user writes handwritten characters or the like in the pixel area where the background color is displayed. Of course, as shown in FIG. 6B, handwritten characters and the like are overwritten on the pixel area 105 (hereinafter referred to as the PC screen display area 105) in which the screen screen of the PC 300 is displayed. You can also

  On the other hand, if it is determined in step 102 that the switch of the pen device 200 is ON (step 102, Yes), the process proceeds to step 104. In step 104, the coordinate acquisition unit 136 acquires the mouse event occurrence coordinates received from the operating system 123 (step 104).

  In subsequent step 105, the event distribution unit 130 determines whether or not the mouse event occurrence coordinates are within the PC screen display area 105. Hereinafter, the determination processing in step 105 will be described based on the conceptual diagram shown in FIG.

The event distribution unit 130 determines the maximum value [x _min ] and minimum value [x _max ] of the X coordinate of the PC screen display region 105 and the Y coordinate from the coordinate information of the inset display region held or acquired by the external input image management unit 132. After obtaining the maximum value [y _min ] and the minimum value [y _max ], the following inequalities (1) and (2) are simultaneously established for the mouse event occurrence coordinates (x, y) acquired by the coordinate acquisition unit 136. Judge whether to do.
x _min <x <x _max (or x _min ≦ x ≦ x _max ) (1)
y _min <y <y _max (or y _min ≦ y ≦ y _max ) (2)

  As a result, if it is determined that the inequalities (1) and (2) do not hold at the same time, it is determined that the mouse event occurrence coordinates (x, y) are not in the PC screen display area 105 (No in Step 105). In step 106, the detected mouse event is transferred to the GUI control unit 126, and the process returns to the mouse event monitoring loop (step 101).

  When the GUI control unit 126 determines that the mouse event received from the event distribution unit 130 selects a predetermined GUI control component, the GUI control unit 126 receives a predetermined operation associated with the selected GUI control component. For example, when the pen device 200 designates the GUI control component “C” shown in FIG. 8A, the GUI control unit 126 defines a predetermined one defined by the touched GUI control component “C”. Execute the process. FIG. 8B illustrates a case where an operation for deleting the screen screen of the PC 300 is executed in response to the GUI control component [C] being specified by the pen device 200.

  On the other hand, if it is determined in step 105 that the inequalities (1) and (2) are simultaneously established, it is determined that the mouse event occurrence coordinates (x, y) are within the PC screen display area 105 (step 105, Yes), go to step 107. In step 107, the coordinate conversion unit 138 converts the mouse event occurrence coordinates received from the operating system 123 into the screen coordinate system of the PC 300 by an appropriate method. Hereinafter, an example of the coordinate conversion processing executed by the coordinate conversion unit 138 will be described based on the conceptual diagram shown in FIG.

  The coordinate conversion unit 138 converts the mouse event occurrence coordinates (x, y) in the screen coordinate system of the electronic blackboard 100 into the mouse event coordinates (x ′, y ′) in the screen coordinate system of the PC 300 based on the following conversion formula (3). ).

In the above formula (3), [x _min ] and [x _max ], and [y _min ] and [y _max ] are respectively the maximum and minimum values of the X coordinate and the Y coordinate of the PC screen display area 105. The maximum value and the minimum value are indicated, and [x ′ _max ] and [y ′ _max ] indicate the maximum value of the X coordinate and the maximum value of the Y coordinate of the screen coordinate system of the PC 300, respectively.

  Here, when the PC screen display area 105 and the mouse event occurrence coordinates are in the state shown in FIG. 9A, the mouse event occurrence coordinates (300, 200) in the screen coordinate system of the electronic blackboard 100 are as shown in FIG. As shown in (b), it is converted into coordinates (180, 140) in the screen coordinate system of the PC 300.

  When the event distribution unit 130 converts the mouse event occurrence coordinates in the screen coordinate system of the electronic blackboard 100 into the coordinates of the screen coordinate system of the PC 300 in step 107, the event distribution unit 130 proceeds to step 108 and transfers the mouse event in which the occurrence coordinates are converted to the PC 300. After that, the process returns to the mouse event monitoring loop (step 101).

  The PC 300 determines whether or not a predetermined GUI control component has been selected based on the generated coordinates (after conversion) of the mouse event received from the electronic blackboard 100. For example, as shown in FIG. 10A, when the pen device 200 touches a button “→” meaning “go to the next page” displayed in the PC screen display area 105, the response occurred in response to the touch. After the coordinates of the mouse event are converted into the coordinates of the screen coordinate system of the PC 300 (that is, the coordinates corresponding to the button “→” displayed on the screen screen 302 of the PC 300), the mouse event is transferred to the PC 300. As a result, as shown in FIG. 10B, the PC 300 switches the screen screen 302 of the PC 300 to the display of the next page, and transmits a VGA signal reflecting the change in display contents to the electronic blackboard 100. As a result, the display of the PC screen display area 105 is switched in synchronization with the screen screen 302 of the PC 300.

  The electronic blackboard 100 of the present embodiment has been described based on an example in which a screen screen is captured from one user PC. However, the electronic blackboard 100 of the present embodiment captures screen screens from a plurality of user PCs. You can also. FIG. 11 shows an embodiment in which the screen screen 302 a of the PC 300 a (PC 1) and the screen screen 302 b of the PC 300 b (PC 2) are simultaneously displayed on the screen screen 104 of the electronic blackboard 100. Hereinafter, the processing executed by the event distribution unit 130 when two PCs are connected to the electronic blackboard 100 will be described with reference to the flowchart shown in FIG.

  In FIG. 12, Step 101 to Step 104 perform the same processing as the content described in FIG.

  In step 105 following step 104, it is determined whether or not the mouse event occurrence coordinates are within the PC screen display area 105a of the PC 300a (PC1), and the mouse event occurrence coordinates (x, y) are the PC screen display area. If it is determined that the mouse event is within 105a (step 105, Yes), the process proceeds to step 107, where the detected coordinates of the mouse event are converted into the screen coordinate system of the PC 300a. After transferring to the PC 300a, the process returns to the mouse event monitoring loop (step 101).

  On the other hand, when it is determined in step 105 that the mouse event occurrence coordinates are not within the PC screen display area 105a of the PC 300a (PC1) (step 105, No), the process proceeds to step 109, where the mouse event occurrence coordinates are PC 300b (PC2). ) In the PC screen display area 105b.

  If it is determined in step 109 that the mouse event occurrence coordinates (x, y) are not within the PC screen display area 105 b (step 109, No), the process proceeds to step 106, and the detected mouse event is sent to the GUI control unit 126. After the transfer, returns to the mouse event monitoring loop (step 101). On the other hand, if it is determined that the mouse event occurrence coordinates (x, y) are within the PC screen display area 105b (step 109, Yes), the process proceeds to step 110, and the detected mouse event occurrence coordinates are set to the screen coordinate system of the PC 300b. In step 111, the mouse event whose generated coordinates have been converted is transferred to the PC 300b, and the process returns to the mouse event monitoring loop (step 101).

  Here, the determination processing in step 105 and step 109 will be exemplarily described based on the conceptual diagram shown in FIG.

The event distribution unit 130 determines the maximum value [x1 _min ] / minimum value [x1 _max ] of the X coordinate of the PC screen display area 105a of the PC1 and Y from the coordinate information of the inset display area held or acquired by the external input image management unit 132. Acquire the maximum value [y1 _min ] / minimum value [y1 _max ] of the coordinate, the maximum value [x2 _min ] / minimum value [x2 _max ] of the X coordinate of the PC screen display area 105b of PC2, and the maximum value of the Y coordinate [y2 _min ] / minimum value [y2 _max ] is acquired. Then, in step 105, it is determined whether or not the following inequalities (4) and (5) are simultaneously established for the mouse event occurrence coordinates (x, y).
_{x1 min <x} <x1 _{max (or, x1 min ≦ x ≦ x1 max} ) (4)
_{y1 min <y} <y1 _{max (or, y1 min ≦ y ≦ y1 max} ) (5)

  As a result, if the inequalities (4) and (5) hold simultaneously, it is determined that the mouse event occurrence coordinates (x, y) are within the PC screen display area 105a (step 105, Yes). Otherwise, it is determined that the mouse event occurrence coordinates (x, y) are not within the PC screen display area 105a (step 105, No).

On the other hand, in step 109, it is determined whether or not the following inequalities (6) and (7) are simultaneously established for the mouse event occurrence coordinates (x, y).
_{x2 min <x} <x2 _{max (or, x2 min ≦ x ≦ x2 max} ) (6)
_{y2 min <y} <y2 _{max (or, y2 min ≦ y ≦ y2 max} ) (7)

  As a result, if the inequalities (6) and (7) hold simultaneously, it is determined that the mouse event occurrence coordinates (x, y) are within the PC screen display area 105b (step 109, Yes). Otherwise, it is determined that the mouse event occurrence coordinates (x, y) are not in the PC screen display area 105b (step 109, No).

  For example, as shown in FIG. 14A, when the pen device 200 touches a “pie graph” displayed in the PC screen display area 105b of the PC 300b (PC2), the coordinates of the mouse event generated in response thereto are displayed. After being converted into coordinates in the screen coordinate system of the PC 300b (that is, coordinates corresponding to the “pie graph” displayed on the screen screen 302b of the PC 300b), the mouse event is transferred to the PC 300b. As a result, as shown in FIG. 14 (b), the PC 300 b pops up another window 303 on the screen screen 302 b of the PC 300 b and transfers a VGA signal reflecting the change in display contents to the electronic blackboard 100. As a result, the display of the PC screen display area 105b is switched in synchronization with the display switching of the screen screen 302b of the PC 300b.

  As described above, according to the present invention, the user can perform three operations of handwriting drawing, operation of the electronic blackboard, and operation of the externally connected PC with very little movement (ON / OFF of the button of the pen device). It is possible to maintain high usability even in a case where both handwriting and operation are frequently performed.

  Each function of the above-described embodiment can be realized by a device-executable program written in an object-oriented programming language such as C, C ++, C #, Java (registered trademark), and the program of this embodiment is a hard disk device. , CD-ROM, MO, DVD, flexible disk, EEPROM, EPROM and the like can be stored and distributed in a device-readable recording medium, and can be transmitted via a network in a format that other devices can.

  As described above, the present invention has been described with the embodiment. However, the present invention is not limited to the above-described embodiment, and as long as the operations and effects of the present invention are exhibited within the scope of embodiments that can be considered by those skilled in the art. It is included in the scope of the present invention.

100 ... Image display device (electronic blackboard)
DESCRIPTION OF SYMBOLS 102 ... Display apparatus 104 ... Screen screen 105 ... PC screen display area 106 ... GUI control component 110 ... Signal receiving part 120 ... Image display control apparatus 121 ... Controller part 122 ... Touch panel driver part 123 ... Operating system 124 ... Video processing part 125 ... Handwritten drawing unit 126 ... GUI control unit 128 ... VGA I / F
129 ... USB I / F
DESCRIPTION OF SYMBOLS 130 ... Event distribution part 132 ... External input image management part 134 ... ON / OFF judgment part 136 ... Coordinate acquisition part 138 ... Coordinate conversion part 150 ... Processor 151 ... ROM
152 ... RAM
153: Hard disk device 154: Graphics board 155 ... Display device 156 ... Coordinate detection device (touch panel)
157 ... Wireless communication interface 158 ... External device connection interface 200 ... Screen instruction device (pen device)
DESCRIPTION OF SYMBOLS 202 ... Push button 204 ... Signal generation part 208 ... Signal transmission part 300 ... Personal computer 302 ... PC screen screen 303 ... Pop-up window 304 ... VGA cable

JP 2006-331263 A

Claims (8)

An image display control device for controlling a display device in which a pointing device is disposed on a display device,
A pointing device event is generated in response to a screen pointing device having an ON / OFF switch contacting or approaching the pointing device, and an ON / OFF event is generated based on an ON / OFF signal received from the screen pointing device. An event generator to generate;
A handwriting drawing unit that generates graphics based on the pointing device event;
A background image is displayed on the display device, a screen screen of an information processing apparatus connected to the outside is captured and displayed in a predetermined inset area in the background image, and the background image or the predetermined inset area is over the A video processing unit that superimposes and displays the graphics generated by the handwriting drawing unit;
A GUI control unit that displays a GUI in the background image or the predetermined inset area and receives a predetermined operation based on the pointing device event;
An event distribution unit that receives the pointing device event and the ON / OFF event from the event generation unit and transfers the pointing device event to any one of the handwriting drawing unit, the GUI control unit, and the information processing apparatus; Including image display control device. The event distribution unit
Based on the ON / OFF event, determine ON / OFF of the switch of the screen instruction device,
When the switch is OFF, the pointing device event is transferred to the handwriting drawing unit,
When the switch is ON and the pointing device event occurrence coordinates are not within the fitting area, the pointing device event is transferred to the GUI control unit,
When the switch is ON and the coordinate of the pointing device event is within the fitting area, the coordinate of the pointing device event is converted into the screen coordinate system of the information processing apparatus, and the information processing Transfer to device,
The image display control apparatus according to claim 1.   The image display control device according to claim 1, wherein the video processing unit captures screen screens of the plurality of information processing devices and displays the screen images in a predetermined inset area in the background image.   The image display control device according to claim 1, wherein the ON / OFF switch is a button switch. A computer-executable method for causing a computer to control a display device having a pointing device disposed on a display device, the computer comprising:
A pointing device event is generated in response to a screen pointing device having an ON / OFF switch contacting or approaching the pointing device, and an ON / OFF event is generated based on an ON / OFF signal received from the screen pointing device. Event generation means for generating,
Handwriting drawing means for generating graphics based on the pointing device event;
A background image is displayed on the display device, a screen screen of an information processing apparatus connected to the outside is captured and displayed in a predetermined inset area in the background image, and the background image or the predetermined inset area is over the Video processing means for superimposing and displaying the graphics generated by the handwriting drawing unit;
GUI control means for displaying a GUI in the background image or the predetermined inset area, and receiving a predetermined operation via the GUI based on the pointing device event;
A method for realizing an event distribution unit that receives the generated pointing device event and the ON / OFF event and transfers the pointing device event to any one of the handwritten drawing unit, the GUI control unit, and the information processing apparatus . The event distribution means includes:
Based on the ON / OFF event, determine ON / OFF of the switch of the screen instruction device,
When the switch is OFF, the pointing device event is transferred to the handwriting drawing means,
When the switch is ON and the pointing device event occurrence coordinates are not within the fitting area, the pointing device event is transferred to the GUI control means;
When the switch is ON and the coordinate of the pointing device event is within the fitting area, the coordinate of the pointing device event is converted into the screen coordinate system of the information processing apparatus, and the information processing Transfer to device,
The method of claim 5.   The method according to claim 5, wherein the video processing unit captures a plurality of screen images of the information processing apparatus and displays the screen images in a predetermined inset area in the background image.   The computer executable program for making a computer perform the method as described in any one of Claims 5-7.