JP5808712B2 - Video display device - Google Patents

Description

  The present invention relates to an image display apparatus that projects an image on a whiteboard or the like and can input writing on the whiteboard, and more particularly to an image display apparatus that performs an operation instruction using an electronic pen and an image operation method thereof.

  2. Description of the Related Art In recent years, with the enlargement of monitors and the widespread use of projector devices, electronic pens have been provided that allow a computer to be used to add data to the data-displayed display surface in the same form as writing letters and figures on a white board with a writing instrument. The electronic pen is required to notify the computer of trajectory information and to notify the computer of operation information such as writing pressure and tilt of the electronic pen.

  For example, Patent Document 1 shows an example of an electronic pen. Specifically, an electronic pen disclosed in Patent Document 1 includes a substantially cylindrical casing that is gripped by a user, a pen tip portion formed on a tip side of the casing, and a three-dimensional configuration of the pen tip section. It is disclosed that the apparatus includes a detecting unit that detects a target displacement and calculates a writing pressure or an inclination of the pen tip portion.

  And an electronic pen, a coordinate position detection device that detects a coordinate position touched on the touch panel surface of the display device by the electronic pen, and information corresponding to a detection result of the coordinate position of the coordinate position detection device. In the touch panel device including a display control device to be displayed, the electronic pen includes a substantially cylindrical housing that is gripped by a user, a pen tip portion formed on a tip side of the housing, and the pen tip portion. Detecting means for detecting the three-dimensional displacement of the electronic pen, and calculating the writing pressure of the electronic pen, the display control device according to the writing pressure of the electronic pen calculated by the detecting means, It is disclosed that the display mode of information displayed on a display device is changed.

  An electronic board system using the above electronic pen as an input device is disclosed in Patent Document 2, for example. In the electronic board system disclosed in Patent Document 2, a coordinate detection device is attached to the board, pen position information on the electronic pen board is detected, and writing data is projected onto the board from the switch information of the pen tip and the pen position information. doing.

JP 2006-92410 A JP 2010-79834 A

  According to the electronic pen described in Patent Document 1, the writing pressure of the electronic pen and the inclination of the pen shaft can be detected. However, in the display device such as the electronic board system disclosed in Patent Document 2, the conventional pen is not available. The writing function of the writing instrument is merely electronic, and it has not yet reached the point of providing a new input device function.

  The purpose of the present invention is to detect pen pressure and tilt pen status using an electronic pen that can detect the detection of pen pressure and tilt detection with higher accuracy, and improve usability based on this. Another object of the present invention is to provide a video display device having a new usage method.

  In order to solve the above problems, a video display device operated with a pen according to the present invention includes a display unit that displays a graphical user interface, a pen position detection unit that detects a position of the pen on the display surface of the video display device, A pen position detected by the pen position detecting means, comprising: pen state information input means including presence / absence of contact of the pen tip with the display surface of the video display device and inclination with respect to the surface with which the pen tip contacts. The graphical user interface is specified by information, and the display content of the graphical user interface is changed according to the pen state information.

  According to the present invention, since the user can smoothly switch operations such as line drawing and line deletion performed on the GUI using the pen-type input device by the user, an image of an electronic board system or the like can be obtained. The effect of improving the operability of the display device is brought about.

It is a figure which shows an example of the external appearance in which a user operates the video display apparatus in Example 1. FIG. It is a figure which shows an example of the external appearance in which a user operates the video display apparatus in Example 1. FIG. 1 is a diagram illustrating an example of a configuration of a video display device in Embodiment 1. FIG. 10 is a diagram illustrating an example of processing in Embodiment 1. FIG. 6 is a diagram illustrating an example of an operation in Embodiment 1. FIG. 6 is a diagram illustrating an example of an operation in Embodiment 1. FIG. 6 is a diagram illustrating an example of an operation in Embodiment 1. FIG. 6 is a diagram illustrating an example of an operation in Embodiment 1. FIG. 1 is a diagram illustrating an example of a configuration of a video display device in Embodiment 1. FIG. 1 is a diagram illustrating an example of a configuration of a video display device in Embodiment 1. FIG. 10 is a diagram illustrating an example of an operation in Embodiment 2. FIG. 10 is a diagram illustrating an example of an operation in Embodiment 2. FIG. 10 is a diagram illustrating an example of an operation in Embodiment 2. FIG. FIG. 10 is a diagram illustrating an example of an operation in the third embodiment. FIG. 10 is a diagram illustrating an example of an operation in the third embodiment. FIG. 10 is a diagram illustrating an example of an operation in the third embodiment. FIG. 10 is a diagram illustrating an example of an operation in the third embodiment. FIG. 10 is a diagram illustrating an example of an operation in the third embodiment. FIG. 10 is a diagram illustrating an example of an operation in the fourth embodiment. FIG. 10 is a diagram illustrating an example of an operation in the fourth embodiment. FIG. 10 is a diagram illustrating an example of an operation in the fourth embodiment. FIG. 10 is a diagram illustrating an example of an operation in the fifth embodiment. FIG. 10 is a diagram illustrating an example of an operation in the fifth embodiment. FIG. 10 is a diagram illustrating an example of an operation in the fifth embodiment. It is a figure which shows the structural example of the electronic pen of an Example. Electronic pen strain detection process flow It is a table | surface which shows the relationship between the state of an electronic pen, and a distortion detected value.

  Prior to the description of the embodiment, an outline of the pen-type input device (electronic pen) of the present embodiment will be described with reference to FIGS. FIG. 25 is a diagram illustrating an example of the structure of the pen-type input device according to the embodiment. FIG. 26 is a table showing the correspondence between the state of the pen-type input device and the detected value.

  As shown in FIG. 25, the structure of the pen-type input device of this embodiment is such that a strain sensor (2501, 2502) whose output value changes according to the amount of strain in one direction such as a semiconductor strain sensor is the surface of the pen shaft. It is the structure installed in. When the user has a pen shaft gripping part and performs a writing operation on a writing surface such as a board or a screen, the strain sensor detects the amount of deformation of the pen shaft as a strain amount. In this embodiment, the contact state of the pen tip of the pen-type input device, the pressure of the pen tip, and the tilt direction of the pen are determined from the distortion amount.

  Here, although two semiconductor strain sensors (2501, 2502) are described in FIG. 25, the number of sensors is not limited as long as the amount of strain can be detected. The inner surface of the pen shaft or the inner surface may be used.

  Although not described in the following embodiments, in addition to the pen tip, by installing a strain sensor at the center of the pen shaft or at the base of the pen shaft, or by installing it at multiple locations, It is possible to determine a twisting operation, a pen shaft pulling / pushing, a pen shaft holding state, and the like, and it is possible to indicate operation instruction information other than the embodiment.

  FIG. 26 shows a processing flow for determining the pen tip contact state, pen tip pressure, and pen tilt direction based on the strain amount ε detected by the strain sensor. Here, ε0 means a sensor output value in an equilibrium state of the strain sensor, and E means a sensor threshold value for processing.

  FIG. 27 shows the specific processing contents of S503 in the processing flow of FIG. In the processing shown in FIG. 27, the amount of change of the strain output from the strain sensor from the equilibrium state is compared with the threshold value, and the contact state of the pen tip with the writing surface and the pen tilt direction are determined based on the magnitude.

  Embodiments of the present invention will be described below with reference to the drawings.

  Next, Example 1 will be described. In the first embodiment, the operation of the projected image of the projector using the pen-type input device will be described.

  FIG. 1 shows an appearance of an example in which a user operates a projected image of a projector using the pen-type input device. Moreover, FIG. 2 shows an external appearance when viewed from the side with the same device configuration as FIG. In FIGS. 1 and 2, 100 is the pen type input device, 2100 is a projector, 2101 is a projected image, 2102 is projection light, 2103 is a wall surface, and 2104 is a pen position detection sensor.

  The projector 2100 is attached to the wall surface 2103, and the projection image 2101 is displayed on the wall surface 2103 by outputting the projection light 2102 based on a predetermined image signal such as GUI.

  The pen position detection sensor 2104 is a sensor provided in the projector 2100, and has a function of detecting the position of the pen tip of the pen-type input device 100 with respect to the projection image 2101. For example, an ultrasonic sensor, a camera, Sensor devices such as an electric field sensor, an infrared distance sensor, a laser distance sensor, and a distance image sensor are used. The sensor used for the pen position detection sensor 2104 is not particularly limited, and the pen-type input on the projection plane of the projected image 2101 is performed in combination with a microcomputer that processes sensor data and software that operates on the microcomputer. It is only necessary to have a function of obtaining the position of the pen tip of the device 100.

  Further, the projector 2100 calculates coordinates corresponding to the projected image 2101 from the position of the pen tip of the pen type input device 100 obtained by the pen position detection sensor 2104, and further calculates the position and movement of the coordinates. Accordingly, the position, movement, configuration, etc. of the graphic in the video displayed on the projected video 2101 are changed. As a result, the user can perform an operation on the projected image 2101 using the pen-type input device 100.

  FIG. 3 shows an example of the configuration of the pen-type input device 100 and the projector 2100. In FIG. 3, the configuration of the pen-type input device 100 is the same as that described above. In the projector 2100 of FIG. 3, 2300 is a pen position detection unit, 2301 is a calculation unit, 2302 is a pen communication unit, 2303 is an operation unit, 2304 is a central processing unit, 2305 is a memory, 2306 is a video control unit, and 2307 is a video. It is a projection unit.

  The pen position detection unit 2300 includes the pen position detection sensor 2104, a microcomputer that processes sensor data of the pen position detection sensor 2104, software that operates on the microcomputer, and the like, and the pen-type input for the projection image 2101 The position of the pen tip of the device 100 is detected.

  The arithmetic unit 2301 is composed of a circuit board or the like, and the pen-type input device obtained via the pen-tip position of the pen-type input device 100 obtained by the pen-position detecting unit 2300 or the pen communication unit 2302 described later. Information such as the inclination of 100, how to be gripped, and the contact state with other objects is processed to determine the state of various operations of the user using the pen-type input device 100.

  The pen communication unit 2302 includes a wireless communication device such as an ultrasonic unit or an infrared communication device, or a wired communication device such as a universal serial bus (USB), and is an interface that can communicate with the pen-type input device 100. The pen communication unit 2302 receives information from the pen-type input device 100 such as tilt, how to hold, and contact state with other objects. The operation unit 2303 includes buttons, switches, an operation panel, and the like, and is an interface that can accept input from the user.

  The central processing unit 2304 is composed of a semiconductor chip such as a central processing unit (hereinafter referred to as CPU), software such as an operating system (hereinafter referred to as OS), and various user operation states obtained from the arithmetic unit 2301. Based on a user input from the operation unit 2303, input / output of information to / from a memory 2305, which will be described later, and each unit of the projector 2100 such as a video control unit 2306, a video projection unit 2307, which will be described later, are controlled.

  The memory 2305 is composed of a semiconductor or the like, and stores information necessary for calculation and control of the central processing unit 2304, video information to be displayed as the projected video 2101, and the like. The video control unit 2306 is composed of a circuit board or the like, performs arithmetic processing necessary for drawing video information in accordance with the control of the central processing unit 2304, and has a format suitable for input to the video projection unit 2307 described later. Rendering information consisting of a set of pixels is output.

  The image projection unit 2307 is composed of a light source such as a lamp, optical components such as a lens and a reflection mirror, a liquid crystal panel, and the like, and draws information obtained from the image control unit 2306 by modulating a light beam emitted from the light source. Image light corresponding to the above is formed, and the image light is enlarged and projected on a projection surface such as a screen.

  In addition, although each part of FIG. 3 is independent, you may comprise by one or several structural requirements as needed. For example, 2301, 2304, 2305, and 2306 may be configured to perform the processing using one or a plurality of semiconductor chips (System-on-a-chip: SoC, etc.).

  Next, referring to FIG. 4, the projector 2100 detects the position of the pen tip of the pen type input device 100, information on whether or not the pen tip is in contact, information on stress applied to the pen tip, and the pen type input device. The flow of processing when changing the position, movement, configuration, etc. of the graphic in the video displayed in the projected video 2101 in accordance with the tilt information of the projection video 2101 is shown.

FIG. 4 is a processing flow showing a flow of control mainly performed by the central processing unit 2304 in the projector 2100.
In S2401, the detection state of the position of the pen tip of the pen-type input device 100 obtained by the pen position detection unit 2300 is determined, and if it can be detected, the process proceeds to S2402.

  In S2402, the pen tip contact state, the pen tip pressure, and the pen tilt direction detection state obtained by the pen communication unit 2302 are determined and detected, and the process proceeds to S2403.

  If the position of the pen tip of the pen-type input device 100 cannot be detected in S2401 and S2402, the processes in S2401 and S2402 are repeated.

  In step S2403, the position of the graphic in the image displayed on the projected image 2101 according to the position of the pen tip of the pen type input device 100, the pen tip contact state, the pen tip pressure, and the pen tilt direction detection state. Change movement, composition, etc. Details of changes in the position, movement, configuration, etc. of the graphic will be described later with reference to another drawing.

  In S2404, it is determined whether the user has instructed to end the operation of the pen-type input device 100 by a predetermined operation via the operation unit 2303. If the end is not instructed, The processes after S2401 are continuously performed.

  Next, referring to FIGS. 5 to 8, the projected image 2101 according to the pen tip position, the pen tip contact state, the pen tip pressure, and the pen tilt direction detection state of the pen input device 100. A description will be given of changes in the position, movement, composition, etc. of the graphic in the video displayed in FIG.

  FIG. 5 shows an example of the configuration of the graphic in the video displayed as the projected video 2101 and the appearance when the user operates the graphic using the pen-type input device 100. In FIG. 5, 2500 is a strain sensor output graph, 2501 is a feedback GUI, 2502 is an icon, and 2503 is a linear graphic.

  A strain sensor output graph 2500 shows the output of the strain detection unit 101 of the pen-type input device 100, as in FIG. The strain sensor output graph 2500 shows an example of the pen-type input device 100 in which the strain sensor 200 is arranged in the same manner as in FIG.

  Accordingly, when Δε <E1 is satisfied in the strain sensor output graph 2500, the pen-type input device 100 is in contact with the wall surface 2103 and tilted to the left as viewed from the user. When Δε> Er is satisfied, the pen-type input device 100 is similarly inclined to the right when viewed from the user.

  The feedback GUI 2501 is a graphic displayed on the projected video 2101. An icon 2502 and a linear graphic 2503 are graphic parts constituting the feedback GUI 2501. Further, the feedback GUI 2501 is a display for allowing the user to visually recognize the inclination recognition state of the pen-type input device 100 in the projector 2100.

  As shown in FIG. 5A, when the user tilts the pen-type input device 100 to the left when viewed from the user, the output from the strain detection unit 101 of the pen-type input device 100 satisfies Δε <εl. The projector 2100 performs display in which the icon 2502 is moved to the left end of the linear graphic 2503 in the feedback GUI 2501. Similarly, as shown in FIG. 5C, when the user tilts the pen-type input device 100 to the right when viewed from the user, the icon 2502 is displayed at the right end of the linear graphic 2503 in the feedback GUI 2501. The display is moved.

As described above, the projector 2100 changes the display of the feedback GUI 2501 in accordance with the tilt recognition state of the pen type input device 100, so that the user can change the display of the pen type input device 100. When performing the operation, the effect of being able to operate reliably while visually confirming the operation status is obtained.
In FIG. 5, it has been described that the projector 2100 recognizes the tilt of the pen-type input device 100 in three stages of FIG. 5 (a), (b), and (c). In order to determine the tilt of the pen-type input device 100, a plurality of threshold levels may be provided, and the same operation as described above may be realized by recognizing the tilt of the pen-type input device 100 in a plurality of steps. .

Next, an example of an operation pattern performed on the projection image 2101 by the user using the pen-type input device 100 will be described.
FIG. 6 shows the appearance of the projected image 2101 and the appearance of the user's operation when the user performs an operation of drawing a line on the projected image 2101 using the pen-type input device 100. In FIG. 6, 2600 is an eraser icon.
An eraser icon 2600 is a graphic displayed on the projected image 2101.

  As shown in FIG. 6A, the user moves the pen-type input device 100 while holding the pen-type input device 100 at a predetermined writing angle and keeping the pen tip of the pen-type input device 100 in contact with the wall surface 2103. Then, the projector 2100 draws a line on the projected image 2101 along the position of the pen tip of the pen-type input device 100.

  On the other hand, as shown in FIG. 6B, the user holds the pen-type input device 100 by tilting it to the left as viewed from the user with respect to a predetermined writing angle, and the pen of the pen-type input device 100. When the projector 2100 moves in a state in which the tip is in contact with the wall surface 2103, the projector 2100 displays the eraser icon 2600 on the projection image 2101, and at the same time, projects the projection along the pen tip position of the pen-type input device 100. The line in the video 2101 is deleted. When the user returns the pen-type input device 100 to a predetermined writing tilt, the projector 2100 turns off the display of the eraser icon 2600 of the projected image 2101 and (a) of FIG. ).

  The projector 2100 estimates the inclination during normal writing from the change in the inclination of the pen-type input device 100 over time in the operation of the user using the pen-type input device 100. The predetermined writing inclination may be dynamically determined.

  FIG. 7 shows the display of the projected image 2101 when the user performs an operation of drawing a line on the projected image 2101 using the pen-type input device 100 by a method different from that shown in FIG. The external appearance of a user's operation is shown.

  FIG. 7A shows an operation when the user deletes a line in the projected image 2101. As shown in (a-1) of FIG. 7, when the user tilts the pen-type input device 100 once more to the left as viewed from the user than a predetermined writing angle, the projector 2100 displays the projected image. At the same time that the eraser icon 2600 is displayed on 2101, the line in the projected image 2101 is deleted along the position of the pen tip of the pen-type input device 100. Further, as shown in (a-2) of FIG. 7, unlike FIG. 6 described above, the eraser icon 2600 is displayed even after the user returns the pen-type input device 100 to a predetermined writing inclination. Is displayed, and the state in which the line in the projected image 2101 is deleted continues.

  On the other hand, FIG. 7B shows an operation when the user draws a line on the projected image 2101. As shown in (b-1) of FIG. 7, when the user tilts the pen-type input device 100 once more to the right as viewed from the user than the tilt at the time of writing, the projector 2100 displays the projected image. The display of the eraser icon 2600 2101 is erased, and a line is drawn on the projected image 2101 as in FIG. 6A described above.

  Further, as shown in FIG. 8, as in (a-1) of FIG. 7, when the user tilts the pen-type input device 100 once more to the left side as viewed from the user than the predetermined writing angle, The projector 2100 thickens the line to be drawn on the projected image 2101, and the user sees the pen-type input device 100 from the user rather than a predetermined writing inclination, as in (b-1) of FIG. When the projector 2100 is tilted to the right once, the line drawn on the projected image 2101 may be thinned.

  As described above, the projector 2100 switches the drawing operation of the line on the projection image 2101, the deletion operation of the line, and the thickness of the line to be drawn according to the inclination recognition state of the pen-type input device 100. Accordingly, an effect is obtained in which the user can switch the operation on the projection image 2101 with a simple operation of tilting the pen-type input device 100.

  The projector 2100 described with reference to FIG. 3 includes all the components 2300 to 2307 inside the projector 2100. For example, as illustrated in FIG. 9, one or more configurations of the projector 2100 are provided. An equivalent function may be realized by combining the projector 2100 and the PC 2900 by configuring the requirements in a personal computer (hereinafter, PC) 2900 connected to the projector 2100.

  Further, for example, as shown in FIG. 10, the projector 2100 described in FIG. 3 is a device that displays an image on a display such as a tablet terminal 3000, and the tablet terminal 3000 includes the pen position detection unit 2300 and the calculation unit. 2301 and the pen communication unit 2302 may be provided to realize an equivalent function in the video of the display.

  Next, Example 2 will be described. In the second embodiment, in the operation of the projected image 2101 of the projector 2100 using the pen-type input device 100 described in the first embodiment, the projection image 2101 is displayed according to the tilt of the pen-type input device 100. The graphic display / non-display switching operation and the operation for changing the display position will be described.

  FIG. 11 shows the appearance of the projected image 2101 and the user's operation when the user performs an operation for displaying a menu on the projected image 2101 using the pen-type input device 100. In FIG. 11, 3100 is a menu GUI.

  A menu GUI 3100 is a graphic displayed on the projected video 2101. The menu GUI 3100 includes operation buttons that allow a user to select and execute a predetermined operation. The predetermined operation is, for example, a text selected by a user in advance, as in a general PC OS. Operations such as cutting and copying for graphics.

  As shown in FIG. 11 (a), when the user tilts the pen-type input device 100 once more to the right than the predetermined writing angle, the projector 2100 moves the pen-type input device. The menu GUI 3100 is displayed at the position of the 100 nib. Further, as shown in FIG. 11B, the user can select an operation button of the displayed menu GUI 3100.

  On the other hand, as shown in FIG. 11 (c), in a state where the menu GUI 3100 is displayed, the user once increases the pen-type input device 100 to the left as viewed from the user with respect to a predetermined writing inclination. When tilted, the menu GUI 3100 is not displayed.

  FIG. 12 shows the appearance of the projected image 2101 and the user's operation when the user performs an operation of displaying a menu on the projected image 2101 using the pen-type input device 100. In FIG. 12, unlike the above-described FIG. 11, the display position of the menu GUI differs depending on whether the user holds the pen-type input device 100 with his / her left or right hand. In FIG. 12, 3200 is a pointer.

  The pointer 3200 is a graphic displayed on the predetermined projected image 2101, and is a graphic for visually confirming the position indicated by the pen tip of the pen-type input device 100 on the projected image 2101.

  In the operation shown in FIG. 12, the projector 2100 dynamically determines the predetermined writing inclination as described above. Also, as shown in FIG. 12, when the predetermined writing inclination satisfies Δε <E0 in the strain sensor output of the pen-type input device 100, the projector 2100 has the user's left hand as the pen-type input. It is determined that the device 100 is held. Similarly, when Δε> E0 is satisfied, it is determined that the user has the pen-type input device 100 with the right hand. Furthermore, when the user displays the menu GUI 3100 by the same operation as described with reference to FIG. 11, the projector 2100 determines that the user has the pen-type input device 100 with the left hand. (A-1), when the menu GUI 3100 is displayed on the right side of the pen tip position of the pen-type input device 100 and it is determined that the user has the pen-type input device 100 with the right hand As shown in FIG. 12A-2, the menu GUI 3100 is displayed on the left side of the position of the pen tip of the pen-type input device 100.

  In addition, in other graphics displayed on the projected image 2101 such as the pointer 3200 shown in FIG. 12B, or in the operation of the first embodiment described above or other embodiments described later, Similar to the description of FIG. 12A, the projector 2100 is effective in operating the projector 2100 depending on whether the user has the pen input device 100 with the left or right hand. The inclination, the display position and angle of the graphic, the graphic type, and the like may be changed.

  FIG. 13 shows the appearance of the projected image 2101 and the user's operation when the user performs an operation for displaying a menu on the projected image 2101 using the pen-type input device 100. In FIG. 13, unlike FIGS. 11 and 12 described above, the graphic moves in the projected image 2101 by the user tilting the pen-type input device 100. In FIG. 13, 3300 is a panel GUI, and 3301 indicates the position of the pen tip of the pen-type input device 100.

  The panel GUI 3300 is a graphic displayed on the projected image 2101 and has a graphic structure for selecting and operating an arbitrary panel from a plurality of arranged panels.

  As shown in (1-a) of FIG. 13, the user points the position of the panel 3 at the upper right of the projected image 2101 with the pen tip of the pen type input device 100, and the left direction as viewed from the user. When the pen-type input device 100 is tilted, the projector 2100 moves the position of the panel GUI 3300 so that the panel 3 is positioned at the center of the projected image 2101 as shown in FIG. To move. Similarly, as shown in (1-c) of FIG. 13, the user indicates the position of the panel 1 at the upper left of the projected image 2101 with the pen tip of the pen-type input device 100, and is viewed from the user. When the pen-type input device 100 is tilted in the right direction, the projector 2100 is arranged such that the panel 1 is positioned at the center of the projected image 2101 as shown in (2-c) of FIG. The position of the GUI 3300 is moved.

  As described above, the projector 2100 switches the display and non-display of the GUI displayed on the projected image 2101 and changes the display position in accordance with the tilt and position recognition state of the pen-type input device 100. As a result, the user can smoothly operate the projected image 2101 with a simple operation of tilting the pen-type input device 100.

  Next, Example 3 will be described. In the third embodiment, in the operation of the projection image 2101 of the projector 2100 using the pen-type input device 100 described in the first embodiment, the operation according to the continuous change in the tilt of the pen-type input device 100. explain.

  FIG. 14 shows an example of the configuration of the graphic in the video displayed as the projected video 2101 and the appearance when the user operates the graphic using the pen-type input device 100, as in FIG. In FIG. 14, the change in display of the feedback GUI 2501 is different from that in FIG. 25 described above, and in the feedback GUI 2501 according to the continuous change of Δε indicated by the strain sensor output of the strain sensor output graph 2500, An icon 2502 moves continuously over the linear graphic 2503.

Next, an example of an operation pattern performed on the projected image 2101 by using a continuous change in the tilt of the pen-type input device 100 will be described.
FIG. 15 shows the appearance of the projected image 2101 and the user's operation when the user operates the graphic of the projected image 2101 using the pen-type input device 100. In FIG. 15, 3500 is a page turning GUI, 3501 is a scroll GUI, 3502 is a moving image content, and 3503 is a time bar GUI.

  The page turning GUI 3500 is a graphic displayed on the projected video 2101, and the displayed page is switched by the movement of the graphic that turns the page left and right.

  The scroll GUI 3501 is a graphic displayed on the projected image 2101. The panels are arranged in a horizontal row, and the display position of the panel is changed by the movement of the graphic that the panel moves to the left and right.

  The moving image content 3502 is a moving image such as a so-called movie displayed on the projected image 2101. A time bar GUI 3503 is a graphic displayed on the projected video 2101, and indicates a playback position of the moving image content 3502 by a change in the position, size, and color of the graphic in the time bar.

  As shown in FIGS. 15A and 15A, when the user tilts the pen-type input device 100 to the left as viewed from the user, the projector 2100 changes to Δε indicated by the strain sensor output. In response, the page being displayed on the page turning GUI 3500 is moved to the left of the projected video 2101. Further, when the user further tilts the pen-type input device 100 to the left when viewed from the user, for example, when Δε <El is satisfied in the strain sensor output graph 2500 of FIG. 14, the projector 2100 The page being displayed on the page turning GUI 3500 is moved to the outside in the left direction of the projected image 2101, and the page being displayed on the page turning GUI 3500 is switched. The same applies when the user tilts the pen-type input device 100 to the right as viewed from the user, as shown in FIGS.

  As shown in FIGS. 15A and 15B, when the user operates the scroll GUI 3501, the operation is the same as the operation of the page turning GUI 3500, and the user tilts the pen-type input device 100. The projector 2100 moves the panel of the scroll GUI 3501 according to the direction. The speed at which the panel moves may change according to the change in Δε indicated by the strain sensor output.

  As shown in FIGS. 15A and 15C, when the user operates the time bar GUI 3503, the operation is the same as the operation of the page turning GUI 3500, and the user tilts the pen-type input device 100. The projector 2100 moves the playback position of the moving image content 3502 indicated by the time bar GUI 3503 in accordance with the selected direction. Note that the speed at which the reproduction position moves may change according to a change in Δε indicated by the strain sensor output.

  As shown in FIG. 16, when the user operates the projection image 2101 using the pen type input device 100, the pointer 3200 is moved with respect to the position of the pen tip of the pen type input device 100. When the position is shifted and the position of the pointer 3200 is corrected by so-called calibration or the like, the user changes the inclination of the pen-type input device 100 as described with reference to FIG. You may correct | amend by moving a position.

  FIG. 17 shows the appearance of the projected image 2101 and the user's operation when the user operates the graphic of the projected image 2101 using the pen-type input device 100. In FIG. 17, 3700 is an image display GUI, and 3701 is an image list GUI.

  The image display GUI 3700 is a graphic displayed on the projected video 2101, displays image data such as photographs and illustrations, and can browse the image data while performing operations such as enlargement, reduction, and movement.

  The image list GUI 3701 is a graphic displayed on the projected image 2101 and displays a plurality of image data such as photographs and illustrations at the same time, and performs operations such as enlarging, reducing, and moving all or part of the plurality of image data. You can browse while doing.

  As shown in FIGS. 17A and 17A, when the user tilts the pen-type input device 100 to the left as viewed from the user, the projector 2100 changes to Δε indicated by the strain sensor output. Accordingly, the image data being displayed on the image display GUI 3700 is reduced. As shown in FIGS. 17C and 17A, when the user tilts the pen-type input device 100 to the right side when viewed from the user, the image data being displayed on the image display GUI 3700 is enlarged. To do.

  It should be noted that the rate of change in enlargement or reduction of the image data being displayed on the image display GUI 3700 may change in accordance with the change in Δε indicated by the strain sensor output. In FIG. 17A, the image data being displayed on the image display GUI 3600 is enlarged or reduced while being positioned at the center of the projected image 2101. As shown in FIG. The image data being displayed on the image display GUI 3700 may be enlarged or reduced about the pen tip position 3301 of the pen-type input device 100.

  As shown in FIGS. 17 (a) and (3), when the user tilts the pen-type input device 100 to the left when viewed from the user, the projector 2100 changes to Δε indicated by the strain sensor output. Accordingly, the entire display of the image data being displayed in the image list GUI 3701 is reduced. As shown in FIGS. 17C and 17C, when the user tilts the pen-type input device 100 to the right side when viewed from the user, the image data displayed on the image list GUI 3701 The image data corresponding to the pen tip position 3301 of the pen-type input device 100 is enlarged.

  As shown in FIG. 18, when the user draws a figure such as a circle 3800 on the projected video 2101 using the pen input device 100, the pen input is performed in the same manner as the image display GUI 3700. Depending on the inclination of the apparatus 100, the figure may be enlarged or reduced.

  As described above, the projector 2100 performs an operation on the GUI displayed on the projection image 2101 in accordance with a continuous change in the tilt of the pen-type input device 100, so that the user can perform the pen-type operation. With the simple operation of tilting the input device 100, an effect can be obtained in which the operation on the projected image 2101 can be performed smoothly.

  Next, Example 4 will be described. In the fourth embodiment, in the operation of the projection image 2101 of the projector 2100 using the pen-type input device 100 described in the first embodiment, an operation according to the stress applied to the pen tip of the pen-type input device 100 will be described. .

  FIG. 19 shows an example of the configuration of the graphic in the video displayed as the projected video 2101 and the appearance when the user operates the graphic using the pen-type input device 100. In FIG. 19, 3900 is the operation layer A, and 3901 is the operation layer B.

  The operation layer A 3900 and the operation layer B 3901 are virtual operation areas in the projected image 2101, and the user independently configures graphics so that the user can operate the operation layer A 3900 or the operation layer B 3901. One of these can be selected by a predetermined operation, and an operation such as graphic drawing can be performed on the selected operation layer. Further, as shown in FIG. 19A, the operation layer A 3900 and the operation layer B 3901 can be displayed in an overlapped manner in the projection image 2101, and the user has operated each operation layer. The result can be viewed as a single video.

  FIG. 19B shows that the user can select either the operation layer A 3900 or the operation layer B 3901 according to the strength of the force pressing the pen-type input device 100 against the wall surface 2103. Yes. Accordingly, in the strain sensor output of the strain sensor output graph 2500, when | Δε | ≦ Es is satisfied, the projector 2100 enables the user to operate the operation layer A 3900. Similarly, | Δε | If> Es is satisfied, the projector 2100 allows the user to operate the operation layer B3901.

  FIG. 20 shows that when the projector 2100 linearly displays a line drawn by the user on the projected image 2101 with the curvature of the line corrected, the user places the pen-type input device 100 on the wall surface 2103. It shows a state where the presence or absence of correction of the curve of the line is switched depending on the strength of the pressing force.

  In the operation shown in FIG. 20, when the strain sensor output of the pen-type input device 100 satisfies | Δε |> Es, the projector 2100 does not correct the curve of the line and satisfies | Δε | ≦ Es. In such a case, the bending of the line is corrected. Accordingly, as shown in FIG. 20A, when the user presses the pen-type input device 100 against the wall surface 2103 with a strong force, the projected image 2101 is moved along the pen tip of the pen-type input device 100. Drawn lines are drawn. On the other hand, as shown in FIG. 20B, when the user moves the pen-type input device 100 against the wall surface 2103 with a weak force, the trajectory of the pen-tip of the pen-type input device 100 on the projected image 2101. A line with a linear correction is drawn.

  As shown in FIG. 21, when the user presses the pen-type input device 100 against the wall surface 2103 too much, for example, the strain sensor output of the pen-type input device 100 satisfies | Δε |> Emax. In this case, the projector 2100 may alert the user by displaying a graphic such as a warning icon 4100 on the projected video 2101. Further, for example, when the strain sensor output of the pen-type input device 100 satisfies | Δε | <Emax, the user's force to press the pen-type input device 100 against the wall surface 2103 is too weak, or the pen-type input Assuming that the pen tip of the apparatus 100 has moved away from the wall surface 2103, the projector 2100 may alert the user with the warning icon 4100 or a predetermined warning sound.

  As described above, the projector 2100 performs an operation on the GUI displayed on the projection image 2101 in accordance with the stress applied to the pen tip of the pen input device 100, so that the user can input the pen input. An effect that the operation on the projected image 2101 can be smoothly performed by a simple operation of simply changing the force pressing the apparatus 100 against the wall surface 2103 is obtained.

  Next, Example 5 will be described. In the fifth embodiment, in the operation of the projection image 2101 of the projector 2100 using the pen-type input device 100 described in the first embodiment, an operation according to stresses other than the pen tip of the pen-type input device 100 will be described. .

  FIG. 22 shows an example of the configuration of graphics in the video displayed as the projected video 2101 and the appearance when the user operates the graphics using the pen-type input device 100.

  The pen-type input device 100 shown in FIG. 22 can detect the contact of the pen base of the pen-type input device 100 as in FIG. 8 described above. Accordingly, as shown in FIG. 22A, when the user operates the projection image 2101 with the pen tip of the pen-type input device 100, as in FIG. A line is drawn on the projected image 2101 along the trajectory of the pen tip of the pen-type input device 100. Further, as shown in FIG. 22B, when the user operates the projection image 2101 at the pen base of the pen-type input device 100, the same as in FIG. A line previously drawn in the projected image 2101 is deleted along the pen source trajectory of the pen-type input device 100.

  FIG. 23 shows an example of the configuration of a graphic in the video displayed as the projected video 2101 and an external appearance when the user operates the graphic using the pen-type input device 100.

  The pen-type input device 100 shown in FIG. 23 can detect the force with which the user holds the grip portion of the pen-type input device 100, as in FIG. 13 described above. Accordingly, as shown in FIG. 23A, when the projected image 2101 is operated by gripping a portion close to the pen tip of the pen-type input device 100 as shown in FIG. Similarly to FIG. 6A, a line is drawn on the projected image 2101 along the trajectory of the pen tip of the pen-type input device 100 as writing means. Further, as shown in FIG. 23B, when the user grips a portion near the pen base of the pen-type input device 100 and operates the projection image 2101 of the user, the force gripped by the user Cannot be detected. In this case, similarly to the above-described FIG. 32B, the pointer 3200 is displayed at the position of the pen tip of the pen-type input device 100. For example, the selection of the button displayed on the projected image 2101 is selected. As an instruction means such as this, an operation according to the position of the pen tip of the pen-type input device 100 can be performed.

  FIG. 24 shows an example of the graphic configuration in the video displayed as the projected video 2101 and the appearance when the user operates the graphic using the pen-type input device 100.

  The pen-type input device 100 shown in FIG. 24 can detect the force with which the user holds the grip portion of the pen-type input device 100, as in FIG. 13 described above. Accordingly, as shown in FIG. 24A, the user points to a graphic such as an icon displayed on the projected image 2101 at the position of the pen tip of the pen-type input device 100, and the user When the pen-type input device 100 is gripped with a strong force, the graphic position such as the icon can be moved according to the position of the pen tip of the pen-type input device 100 like a so-called drag operation.

  Note that when the graphic position such as the icon is moved, as shown in FIG. 24B, the user holds the pen-type input device 100 with a strong force, and the pen-type input device. The pen-type input device 100 is moved in a state where the pen tip of 100 does not contact the wall surface 2103, and the pen tip of the pen-type input device 100 is again moved to the wall surface 2103 as shown in FIG. Further, the movement of the graphic position such as the icon may be completed when the force of gripping the pen-type input device 100 is weakened.

  In addition, the projector 2100 detects the contact of the pen base of the pen-type input device 100 described above, or detects the force with which the user holds the grip portion of the pen-type input device 100. You may perform operation similar to an Example.

  As described above, the projector 2100 is displayed on the projection image 2101 according to the stress applied to the pen base of the pen-type input device 100 and the position and force with which the user holds the pen-type input device 100. By executing the operation on the GUI, an effect can be obtained in which the user can smoothly operate the projection image 2101 with a simple operation using the pen-type input device 100.

  In addition, this embodiment mentioned above is an illustration for description of this invention, and is not the meaning which limits the scope of the present invention only to embodiment.

200: Semiconductor strain sensor, 201: Sensor sensitivity axis,
2100 ... Projector, 2101 ... Projected image, 2102 ... Projected light, 2103 ... Wall surface,
2104 ... Pen position detection sensor, 2500 ... Strain sensor output graph,
2501 ... Feedback GUI, 2502 ... Icon, 2503 ... Linear graphic,
2600 ... Eraser icon, 2900 ... PC, 3000 ... Tablet terminal 3100 ... Menu GUI, 3200 ... Pointer, 3300 ... Panel GUI,
3301 ... Pen tip position, 3700 ... Image display GUI, 3701 ... Image list GUI,
3800 ... circle, 3900 ... operation layer A, 3901 ... operation layer B,
4100 ... Warning icon Warning icon

Claims (4)

A video display device operated with a pen,
Display means for displaying a graphical user interface;
Pen position detecting means for detecting the position of the pen on the display surface of the video display device;
The presence or absence of contact of the pen tip of the pen with the display surface of the video display device, and input means for pen state information including an inclination with respect to the surface with which the pen tip contacts,
The graphical user interface is identified by the pen position information detected by the pen position detecting means,
Change the display content of the graphical user interface according to the pen status information ,
A standard pen tilt is obtained from a change in the tilt of the pen status information with respect to the surface with which the pen tip contacts, and the tilt of the input pen status information with respect to the surface with which the pen tip contacts differs from the standard pen tilt In such a case, the display content of the graphical user interface is changed . A video display device operated with a pen,
Display means for displaying a graphical user interface;
Pen position detecting means for detecting the position of the pen on the display surface of the video display device;
Comprising the presence or absence of contact with the display surface of the pen nib of the video display device, an input means of the force of including pen state information for gripping the pen user, and
An image display device characterized in that the graphical user interface is specified by pen position information detected by the pen position detecting means, and the display content of the graphical user interface is changed in accordance with the pen state information. The video display device according to claim 2 ,
When the force for gripping the pen of the pen status information is equal to or larger than a predetermined value,
An image display apparatus characterized by moving a graphic element specified by pen position information detected by the pen position detecting means. The video display device according to claim 2 ,
When the force for gripping the pen of the pen status information has been detected, it performs path display with the pen position information said pen position detecting means detects a force for gripping the pen of the pen status information can not be detected In some cases, an image display apparatus is configured to instruct selection of a graphic element based on pen position information detected by the pen position detecting means.

Images