JP2014203131A - Input device, input processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mechanism that allows a user to easily perform handwriting input using a display with high transmissivity.SOLUTION: An input device is arranged on a document to receive input to the document, and includes switching means for switching, at a predetermined timing, display on display means from transmission display displaying a document transmitted through the display means and arranged on a rear face side of the display means to non-transmission display displaying a virtual image virtually indicating the transmission display of the document by the display means, and input receiving means for receiving input to the document from a user.

Description

  The present invention relates to an input device, an input processing method, and a program.

Conventionally, augmented reality is known for displaying information superimposed on an image acquired from a camera. For example, Patent Document 1 discloses a technique for displaying information related to a photographed document in an overlapping manner. There has been a request to input information by handwriting on a document photographed in this way.
In addition, there is a demand to input information to a document through the apparatus after the apparatus is overlaid on the document using a highly transparent touch panel display disclosed in Patent Document 2. . When inputting into a document through a highly transmissive display device, it is desirable to enable handwriting input in a state where the document existing on the back side of the device is transmitted from the device.

JP 2011-8752 A JP 2000-353049 A

  In general, the size of a document is larger than the size of a display device with high transparency as described above. For this reason, when handwritten input covers a wide range, the user has to move the display device in accordance with a position where input is desired, which is troublesome.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a mechanism capable of easily performing handwriting input using a highly transmissive display.

  Therefore, the present invention is an input device that is arranged on a document and receives an input to the document, and at a predetermined timing, the display unit displays the display unit through the display unit and the back side of the display unit Switching means for switching from transparent display for displaying the document placed on the display to non-transparent display for displaying a virtual image virtually showing the transparent display of the document by the display means, and input for accepting an input from the user for the document And receiving means.

  ADVANTAGE OF THE INVENTION According to this invention, the structure which can perform the handwritten input using a highly transmissive display easily can be provided.

It is a figure which shows an input device. It is a figure which shows a data processing part. It is a figure which shows an input device. It is a figure which shows the positional relationship of an input device and a document. It is a figure for demonstrating the operation | movement of writing to a document. It is a flowchart which shows a display switching process. It is a flowchart which shows arrangement | positioning determination processing. It is a figure for demonstrating a display switching process. It is a flowchart which shows a display switching process.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
(First embodiment)
FIG. 1 is a diagram illustrating an input device. The input device according to the present embodiment is a tablet terminal having a display screen. The CPU 101 controls the entire apparatus by executing a control program stored in the ROM 102. The ROM 102 stores a program executed by the CPU 101. The functions and processes of the input device described later are realized by the CPU 101 reading out a program stored in the ROM 102 and executing this program. The RAM 103 temporarily stores various data from each component. The program is expanded and executed by the CPU 101.
The reading unit 104 is, for example, a scanner, and reads a document to form a still image when the apparatus is placed on the document. The image capturing unit 105 records an image captured by the image sensor as digital data.

The storage device 106 is a device that stores various data. As the storage device 106, a flash memory, an HDD, or the like can be used. The input unit 107 includes operation buttons, a touch panel, and the like, and accepts inputs such as instructions from the user. The display unit 108 includes a liquid crystal panel and displays various data.
The display unit 108 is a highly transmissive display and can perform transmissive display and non-transmissive display. In the transmissive display, the display unit 108 transmissively displays an object or the like existing on the back side of the display unit 108. When the input device is placed over the document, the display unit 108 transparently displays the document existing on the back side of the display unit 108. In non-transparent display, the display unit 108 displays a virtual image instead of performing transparent display on the back side. Here, the virtual image is image information that virtually shows a transparent display of a document existing on the back side. The transmissive display and the non-transmissive display will be described later. The input unit 107 and the display unit 108 are integrally formed.

The communication unit 109 connects to the Internet by wireless or the like. The communication unit 109 communicates with the external server 110, for example. The external server 110 has a document information database 111. The document information database 111 stores management information of data to be processed by the input device. The document information database 111 stores, for example, document information indicating a document and input information input from the user with respect to the document information in association with each other.
In the input device according to the present embodiment, the display switching process described with reference to FIG. 6 is realized by software using the CPU 101. However, as another example, part or all of the process is performed by an electronic circuit or the like. You may make it implement | achieve with the hardware of.

FIG. 2 is a diagram illustrating a data processing unit of the input device. The arrangement determination unit 112 receives information from the imaging unit 105 and determines the arrangement state of the input device. There are two arrangement states, a hollow state and a grounded state. The hollow state is a state in which the input device is held hollow by a human hand or the like. The grounding state is a state where the input device is placed on a document or a desk. That is, in the grounding state, the input device is in contact with some object. The arrangement determination process will be described later with reference to FIG. When it is determined that the grounding state is established, the reading unit 104 reads the grounding surface.
The overlap determination unit 113 receives information from the reading unit 104 and determines whether or not the document overlap state. Here, the document overlapping state is a state in which the input device is placed on the document. If the document is overlapped, the communication unit 109 transmits information obtained by the reading unit 104 to the external server 110.
Based on the information received from the reading unit 104, the external server 110 identifies a document that overlaps the input device, and returns document identification information that identifies the identified document to the input device.

The image generation unit 114 generates a virtual image corresponding to a document that is overlapped with the input device. Specifically, the image generation unit 114 transmits document identification information to the external server 110 and receives document information identified by the document identification information from the external server 110. The image generation unit 114 generates a virtual image by processing the received document information into a displayable state.
The input information combining unit 115 combines input information. The input information synthesizing unit 115 performs processing for displaying the input information input by the user via the input unit 107 so as to be superimposed on the transparent display or the non-transparent display. In non-transparent display, the input information synthesis unit 115 synthesizes a character input by the user, a locus of handwritten drawing by the user, and the like as input information into a virtual image. The display switching unit 116 switches the display on the display unit 108 to either transmissive display or non-transparent display at a predetermined timing.

FIG. 3 is a diagram illustrating an appearance of the input device. The outer frame 201 is constructed of, for example, plastic or metal. The panel 202 is constructed of an organic EL display or the like having sufficient transparency, and can be viewed through the object on the back side from the front side of the panel 202. The panel 202 functions as the input unit 107 and the display unit 108.
FIG. 4 is a diagram for explaining the positional relationship between the input device and the document. FIG. 4A is a diagram illustrating a state in which the input device is placed in a place that does not overlap the document 300. At this time, the arrangement determination unit 112 determines that the grounding state. FIG. 4B is a diagram illustrating a state where the input device is lifted by the user's hand. At this time, the arrangement determination unit 112 determines the hollow state. The display unit 108 displays the document 300 in a transparent manner. FIG. 4C is a diagram illustrating a state in which the input device is separated from the user's hand and placed on the document 300. At this time, the overlap determination unit 113 determines that the document overlap state. Further, at this time, the display unit 108 appropriately performs transmissive display or non-transmissive display.

FIG. 5 is a diagram for explaining a writing operation on a document by the user using the input device. FIG. 5A is a diagram illustrating a state in which the user enters a writing character 402 using the touch pen 401. Note that the panel 202 shown in FIG. 5A transparently displays the document 300 arranged on the back side of the input device, and the user is browsing the transparently displayed document 300.
In this state, the user uses the touch pen 401 to trace on the panel 202. In response to this, the input unit 107 acquires the trajectory of the touch pen 401 as input information. Then, the input information combining unit 115 superimposes a writing character 402 corresponding to writing by the touch pen 401 on the transparent display, and displays it on the panel 202.
FIG. 5B is a diagram illustrating a state in which the user continuously enters the writing character 403. The writing character 403 has reached the end of the panel 202 and cannot be written on a document located outside the panel 202. In this case, the display switching unit 116 switches the display on the panel 202 from the transparent display to the non-transparent display.

FIG. 5C is a diagram illustrating a display example of the panel 202 after switching to the non-transparent display. A virtual image corresponding to the document 300 is displayed on the panel 202 shown in FIG. Note that in the panel 202 shown in FIG. 5C, written characters 402 and 403 are superimposed on the virtual image. In the example shown in FIG. 5B, it is assumed that the written characters are continued toward the lower direction of the document 300. Therefore, in the example shown in FIG. 5C, a lower image is displayed as compared with the display position of the document 300 that is transparently displayed immediately before.
FIG. 5D is a diagram illustrating a state in which the user newly writes a writing character 404 in an area below the document in the virtual image. In this way, the user can enter characters to be written in an area of the document 300 that does not overlap the input device without moving the input device. As described above, the input device can display not only the transparent display but also the virtual image indicating the transparently displayed document, and can switch between the two displays at an arbitrary timing.

FIG. 6 is a flowchart showing display switching processing by the input device. In step S <b> 501, the CPU 101 uses the imaging unit 105 to image the back surface (back surface) side of the panel 202 to obtain a captured image. Note that the imaging unit 105 is disposed on the back side of the panel 202 as the display unit 108. The captured image may be a still image or a moving image.
Next, in S502, the arrangement state is determined based on the captured image of the arrangement determination unit 112. The process for determining the arrangement state will be described later with reference to FIG. If it is determined in S502 that the grounding state is set, the CPU 101 advances the process to S504. If it is determined in S502 that the state is hollow, the CPU 101 advances the process to S503.

In step S <b> 503, the display switching unit 116 transmits the display on the display unit 108 through the panel 202 so that an object existing on the back side of the input device can be viewed on the panel 202 of the input device. On the other hand, in S504, the overlap determination unit 113 determines whether or not the document overlap state. Specifically, the CPU 101 uses the reading unit 104 to read back side information. Then, the overlap determination unit 113 determines whether or not the read information indicates a document. For example, the overlap determination unit 113 applies an OCR technique and determines whether or not to indicate a document on the basis of whether or not character information is included.
If it is determined in S504 that the document is in an overlapping state, the CPU 101 advances the process to S505. If it is determined in S504 that the document is not overlapped, the CPU 101 advances the process to S503.

In step S <b> 505, the CPU 101 transmits read information determined to be in an overlapping state to the external server 110. At this time, the external server 110 identifies the document indicated by the read information. Then, the external server 110 transmits document identification information for identifying the identified document to the input device. Then, the input device receives the document identification information, and the CPU 101 identifies the document indicated by the read information, that is, the document transparently displayed on the display unit 108 based on the document identification information (document identification processing).
In step S <b> 506, the CPU 101 receives information input from the user via the input unit 107 (input reception process). Then, the CPU 101 determines whether or not the input position by the user is within the preset end region based on the received information. Here, the end region is a region within a certain distance from the end of the panel 202. Note that the end region is an example of a predetermined range.

If the input position is within the end area in S506, the CPU 101 advances the process to S507. If the input position is outside the end area in S506, the CPU 101 advances the process to S503. In step S507, the image generation unit 114 generates a virtual image. Specifically, in step S <b> 505, the image generation unit 114 transmits the document identification information of the identified document to the external server 110 and receives corresponding document information from the external server 110. Then, the image generation unit 114 images the received document information to obtain a virtual image.
In step S508, the input information combining unit 115 combines the input information input by the user with the virtual image. Next, in S509, the display switching unit 116 switches the display on the display unit 108 from transmissive display to non-transparent display (switching process). Thereby, the virtual image combined with the input information is displayed on the display unit 108 in a non-transparent manner.

  As described above, the CPU 101 switches the display on the display unit 108 to the non-transparent display by the processing of S507 to S509 at the timing when the input into the end region is received. Thereby, in a state where the input device is placed on the document, the area of the document that cannot be transparently displayed on the panel 202 can be displayed on the panel 202 as a virtual image using the document information. Therefore, the user can display the entire area of the document to be input on the panel 202 without moving the input device.

FIG. 7 is a flowchart showing detailed processing in the arrangement determination processing described with reference to FIG. First, in S601, the arrangement determination unit 112 determines whether or not it is in a dark state. Specifically, the imaging unit 105 detects illuminance. And the arrangement | positioning determination part 112 determines with it being in a dark state, when illumination intensity is below the preset illumination intensity threshold value.
As another example, the arrangement determination unit 112 may determine whether or not a dark state is detected based on the luminance of the captured image obtained by the imaging unit 105. Specifically, the arrangement determination unit 112 may determine that the state is the dark state when the maximum value or average value of the luminance is equal to or less than a predetermined luminance threshold value. Here, the captured image is a moving image or a still image.

If it is determined in S601 that the image is not a dark image, the arrangement determining unit 112 determines that the image is hollow (S602). On the other hand, if it is determined in S601 that the image is a dark image, the arrangement determining unit 112 determines that the grounding state is set (S603).
As described above, the input device according to the present embodiment can virtually display an area of a document that is not transparently displayed on the panel 202 of the input device. Therefore, the user can display the entire area of the document on the panel 202 without moving the input device, and can input the desired position.

(Second Embodiment)
Next, an input device according to a second embodiment will be described. The input device according to the second embodiment switches between transmissive display and non-transparent display at a timing when an input of a display switching instruction is received from the user.

FIG. 8 is a diagram for explaining the display switching process. FIG. 8A is a diagram illustrating a state in which a user enters a write character 702 using the touch pen 701. In the panel 202 shown in FIG. 8A, the document 300 arranged on the back side of the input device is transparently displayed, and the user is browsing the transparently displayed document. Further, on the panel 202 shown in FIG. 8A, a written character 702 input by the user is displayed superimposed on the transparently displayed document 300.
When the input / movement switching button 703 is pressed by the user, the CPU 101 changes the operation mode of the panel 202. There are two operation modes: an “input mode” in which a user inputs characters and the like, and a “movement mode” in which an input of a display position switching instruction is received from the user. The operation mode of the panel 202 shown in FIG. 8A is set to the input mode.

FIG. 8B is a diagram illustrating a display example of the panel 202 after being switched to the “movement mode”. In the panel 202 shown in FIG. 8A, when the user presses the input / movement switching button 703, the operation mode is switched to the “movement mode”. In this state, as shown in FIG. 8B, when the user traces the panel 202 with the hand 704, the position of the document displayed on the panel 202 is changed. As shown in FIG. 8B, when the panel 202 is traced from the bottom to the top, the CPU 101 receives an input of a switching instruction including an instruction to move the document downward. Then, the display switching unit 116 switches from transparent display to non-transparent display in accordance with the switching instruction.
Then, the display unit 108 non-transparently displays the virtual image in the lower range by an amount corresponding to the movement instruction with respect to the transparently displayed region. FIG. 7C is a diagram illustrating a non-transparent display example displayed in response to the movement instruction. In the non-transparent display shown in FIG. 7C, the input information of the written character 702 is displayed in a superimposed manner. Then, as shown in FIG. 7D, the user can newly enter a write character 705 on the non-transparent display virtual image.
In this way, the user can virtually display a document area that is not transparently displayed on the panel 202 by inputting the switching instruction. Then, the user can input to a desired position while viewing the virtual image.

FIG. 9 is a flowchart illustrating display switching processing in the input device according to the second embodiment. In the input processing according to the second embodiment, after the document is specified in S505, the CPU 101 determines in S520 whether a switching instruction has been received from the user. In S520, when the input of the switching instruction is accepted, the CPU 101 advances the process to S507. In S520, when the input of the switching instruction is not accepted, the CPU 101 advances the process to S503. The remaining configuration and processing of the input device according to the second embodiment are the same as the configuration and processing of the input device according to the first embodiment.
Even when the user has not input to the end of the panel 202, the user may want to write to another distant position. In such a case, in the input device according to the second embodiment, it is possible to switch between transparent display and non-transparent display in accordance with an input from the user. Therefore, the user can display the entire area of the document on the panel 202 without moving the input device, and can input the desired position.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads and executes the program.

  As mentioned above, according to each embodiment mentioned above, handwriting input using a display with high transparency can be performed easily.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

  101 CPU, 102 ROM, 103 RAM, 104 Reading unit, 105 Imaging unit, 106 Storage device, 107 Input unit, 108 Display unit, 109 Communication unit, 110 External server, 111 Document information database

Claims (7)

An input device arranged on a document and accepting input to the document,
At a predetermined timing, the display of the display means is virtually shown as a transparent display of the document by the display means from a transparent display that displays the document disposed on the back side of the display means through the display means. Switching means for switching to non-transparent display for displaying a virtual image;
An input device comprising: an input receiving unit that receives an input from a user for the document.   The input device according to claim 1, wherein the switching unit switches the display of the display unit at the timing when the input receiving unit receives an input to a predetermined range of the display unit. The input receiving means further receives an input of a display switching instruction of the display means from a user;
The input device according to claim 1, wherein the switching unit switches display of the display unit at the timing when the input receiving unit receives an input of the switching instruction. A document specifying means for specifying the transparently displayed document;
The switching unit switches from the transparent display for displaying the document to the non-transparent display for displaying the virtual image of the document specified by the document specifying unit. The input device according to item.   5. The input device according to claim 1, wherein the display unit superimposes and displays the input information received by the input receiving unit on the virtual image. An input processing method executed by an input device disposed on a document and receiving input to the document,
At a predetermined timing, the display of the display means is virtually shown as a transparent display of the document by the display means from a transparent display that displays the document disposed on the back side of the display means through the display means. A switching step for switching to non-transparent display for displaying a virtual image;
And an input receiving step for receiving an input from the user for the document. Computer
At a predetermined timing, the display of the display means is a virtual display that virtually shows the transparent display of the document by the display means from the transparent display that displays the document arranged on the back side of the display means through the display means. Switching means for switching to non-transparent display for displaying an image;
A program for functioning as input receiving means for receiving input from a user for the document.

Images