JP2023162919A - Information processing device - Google Patents

Abstract

To improve a user experience pen.SOLUTION: An information processing device includes: a display unit; a touch sensor unit that is arranged on a screen of the display unit and can detect touch input by contact of an object and pen input by contact of a pen on the screen; and a virtual input processing unit that switches a first mode in which at least a part of an area on the screen is caused to function as a software keyboard on the basis of the touch input to a second mode in which the area is caused to function as a handwriting input device on the basis of the pen input when the pen approaches to the region of the software keyboard within a distance equal to or shorter than a threshold.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device.

In recent years, information processing devices such as notebook personal computers (hereinafter referred to as notebook PCs) and tablet terminals that use the display screen as a virtual input device such as a software keyboard are known (for example, Patent Document 1 reference).

JP 2018-10512 Publication

However, in the above-mentioned conventional information processing devices, when using a software keyboard such as an OSK (On Screen Keyboard) as a virtual input device, it is difficult to input and is not user-friendly compared to a physical keyboard. Therefore, there was a possibility that the user experience pen could not be obtained.

The present invention was made to solve the above problems, and its purpose is to provide an information processing device that can improve the user experience of a pen.

In order to solve the above problem, one embodiment of the present invention includes a display unit, which is arranged on the screen of the display unit, and detects touch input caused by contact of an object on the screen and pen input caused by touch of a pen. and a first mode in which at least a part of the screen functions as a software keyboard based on the touch input, the pen is at a distance within a threshold with respect to the area of the software keyboard. The information processing apparatus includes a virtual input processing unit that switches the area to a second mode to function as a handwriting input device based on the pen input when the area approaches.

Further, one aspect of the present invention includes a display unit, and a touch sensor unit that is arranged on the screen of the display unit and is capable of detecting touch input caused by contact with an object on the screen and pen input caused by contact with a pen. , in a first mode in which at least a partial area of the screen functions as a software keyboard based on the touch input, when the pen approaches the area of the software keyboard to a distance within a threshold value; The area is switched to a second mode in which the area functions as a handwriting input device based on the pen input, and in the second mode, the touch sensor section is configured to switch the area to a second mode in which the area functions as a handwriting input device. a switching processing unit that switches to the first mode when the touch input is detected; and in the second mode, converts the trajectory of the pen input by handwriting into text information, and uses the text information as input information. The information processing apparatus includes a pen input processing section that outputs data.

Further, one aspect of the present invention includes a display unit, and a touch sensor unit that is arranged on the screen of the display unit and is capable of detecting touch input caused by contact with an object on the screen and pen input caused by contact with a pen. , in a first mode in which a first area of at least a portion of the screen functions as a software keyboard based on the touch input, the pen approaches the first area of the software keyboard at a distance within a threshold value; the first area is switched to a second mode in which the first area functions as a handwriting input device based on the pen input, and in the second mode, the touch sensor section a switching processing unit that switches the area to the first mode when the touch input is detected; and a periphery of a character input cursor specified in a second area different from the first area in the second mode; The area is enlarged and displayed in the first area, and based on the pen input, the trajectory of the pen input by hand is displayed in the first area, and the trajectory of the pen is further displayed in the second area. There is an information processing apparatus that includes a pen input processing section that converts the size and displays the pen at a corresponding position, and outputs the converted trajectory of the pen as input information.

According to the above aspects of the invention, the user experience of the pen can be improved.

FIG. 1 is an external view showing an example of a tablet terminal according to a first embodiment. FIG. 2 is a diagram showing an example of the main hardware configuration of the tablet terminal according to the first embodiment. 1 is a block diagram showing an example of a functional configuration of a tablet terminal according to a first embodiment. FIG. FIG. 3 is an image diagram showing an example of mode switching of the tablet terminal according to the first embodiment. 2 is a flowchart illustrating an example of mode switching processing of the tablet terminal according to the first embodiment. 7 is a flowchart illustrating an example of OSK processing of the tablet terminal according to the first embodiment. It is a flowchart which shows an example of handwriting input processing of a tablet terminal by a 1st embodiment. FIG. 2 is a block diagram illustrating an example of a functional configuration of a tablet terminal according to a second embodiment. FIG. 7 is an image diagram showing an example of mode switching of the tablet terminal according to the second embodiment. 12 is a flowchart illustrating an example of mode switching processing of the tablet terminal according to the second embodiment. 12 is a flowchart illustrating an example of handwriting input processing on a tablet terminal according to a second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS An information processing apparatus according to an embodiment of the present invention will be described below with reference to the drawings.

[First embodiment]
FIG. 1 is an external view showing an example of a tablet terminal 1 according to the present embodiment. Note that in this embodiment, a tablet terminal 1 will be described as an example of an information processing device.
As shown in FIG. 1, the tablet terminal 1 has a touch screen 20 installed on one main surface of a housing CS, and uses a pen 30 to execute an application program such as a memo pad.

The touch screen 20 includes a display section 14 and a touch sensor section 33, and the display section 14 displays various information on the screen DF.
The touch sensor unit 33 is arranged to overlap the display unit 14 and is capable of detecting touch input caused by contact with an object (for example, a user's finger, etc.) on the screen DF and pen input caused by contact with the pen 30. It is.
Note that details of the touch screen 20, the display section 14, and the touch sensor section 33 will be described later.

Next, with reference to FIG. 2, the main hardware configuration of the tablet terminal 1 will be described.
FIG. 2 is a diagram showing an example of the main hardware configuration of the tablet terminal 1 according to this embodiment. The tablet terminal 1 is, for example, a terminal device based on a notebook PC whose OS (Operating System) is Windows (registered trademark).

As shown in FIG. 2, the tablet terminal 1 includes a CPU 11, a main memory 12, a video subsystem 13, a display unit 14, a chipset 21, a BIOS memory 22, an SSD 23, a USB connector 24, an audio It includes a system 25, a WLAN card 26, a pen 30, an embedded controller 31, a key input section 32, a touch sensor section 33, a power supply circuit 34, and a sensor section 35.

A CPU (Central Processing Unit) 11 executes various calculation processes under program control and controls the entire tablet terminal 1 .
The main memory 12 is a writable memory used as a reading area for the execution program of the CPU 11 or as a work area for writing processing data of the execution program. The main memory 12 is composed of, for example, a plurality of DRAM (Dynamic Random Access Memory) chips. This execution program includes an OS, various device drivers for operating the hardware of peripheral devices, various services/utilities, application programs (application software), and the like.

The video subsystem 13 is a subsystem for realizing functions related to image display, and includes a video controller. This video controller processes drawing commands from the CPU 11, writes processed drawing information into the video memory, reads out this drawing information from the video memory, and outputs it to the display unit 14 as drawing data (display data).

The display unit 14 is, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display, and displays a display screen based on drawing data (display data) output from the video subsystem 13. The display unit 14 is configured as a part of the touch screen 20, as shown in FIG. 1 described above.

The chipset 21 supports USB (Universal Serial Bus), Serial ATA (AT Attachment), SPI (Serial Peripheral Interface) bus, PCI (Peripheral Component Interconnect) bus, PCI-Express bus, LPC (Low Pin Count) bus, etc. It has a controller and can connect multiple devices. In FIG. 2, as examples of devices, a BIOS memory 22, an SSD 23, a USB connector 24, an audio system 25, a WLAN card 26, and an embedded controller 31 are connected to the chipset 21.

The BIOS (Basic Input Output System) memory 22 is configured of an electrically rewritable nonvolatile memory such as an EEPROM (Electrically Erasable Programmable Read Only Memory) or a flash ROM. The BIOS memory 22 stores the BIOS, system firmware for controlling the embedded controller 31, and the like.

The SSD (Solid State Drive) 23 (an example of a nonvolatile storage device) stores an OS, various drivers, various services/utilities, application programs (hereinafter sometimes referred to as applications), and various data. The USB connector 24 is a connector for connecting peripheral devices using USB.

The audio system 25 records, reproduces, and outputs sound data.
The WLAN (Wireless Local Area Network) card 26 connects to a network via a wireless LAN and performs data communication. For example, when the WLAN card 26 receives data from the network, it generates an event trigger indicating that the data has been received.

The embedded controller 31 is a one-chip microcomputer that monitors and controls various devices (peripheral devices, sensors, etc.) regardless of the system status of the tablet terminal 1. Further, the embedded controller 31 has a power management function for controlling the power supply circuit 34. The embedded controller 31 is composed of a CPU, ROM, RAM, etc. (not shown), and is equipped with a plurality of channels of A/D input terminals, D/A output terminals, timers, and digital input/output terminals. For example, a key input unit 32, a touch sensor unit 33, a power supply circuit 34, a sensor unit 35, etc. are connected to the embedded controller 31 via their input/output terminals, and the embedded controller 31 controls the operation of these units. control.

The key input unit 32 detects, for example, the position of an input device such as a power switch and the contact of an operation medium on the screen DF. For example, the touch sensor unit 33 detects that the pen 30 (pen-shaped medium) approaches the screen DF of the display unit 14 within a predetermined distance (distance within a threshold value), and detects that the pen 30 approaches the screen DF. On the other hand, when the pen 30 approaches within a predetermined distance, the position of the pen 30 on the screen DF can be detected without contact. Note that the touch sensor section 33 and the display section 14 constitute the touch screen 20.

Further, the touch sensor section 33 includes a contact detection section 331 and a pen detection section 332.
The contact detection unit 331 is, for example, a capacitive touch sensor, and detects an operating medium (an object such as the pen 30 or a finger) that has contacted the screen DF of the display unit 14, and detects whether the operating medium is in contact with the screen DF. Detect location.

The pen detection unit 332 is, for example, an electromagnetic induction type touch sensor, and detects the position of the pen 30 on the screen DF of the display unit 14 in a non-contact manner using the resonance circuit LC1 of the pen 30. The pen detection unit 332 can detect, for example, that the pen 30 has approached the screen DF of the display unit 14 at a distance within a threshold value.
In this way, the touch sensor unit 33 can detect touch input caused by contact with an object (for example, a finger) on the screen DF, and pen input caused by contact with the pen 30.

Note that when the pen detection unit 332 detects that the pen 30 approaches the screen DF of the display unit 14 within a distance of a threshold value, the touch sensor unit 33 causes the contact detection unit 331 to By detecting contact, hovering of the pen 30 can be detected. Here, hovering of the pen 30 is a state in which the pen 30 is stopped within a predetermined distance from the screen DF.

The pen 30 is a pen-shaped operating medium, such as a touch pen or a stylus pen. The pen 30 includes a resonant circuit LC1. The pen 30 is configured such that power is supplied by electromagnetic induction to the coil of the resonant circuit LC1, and the position of the pen 30 on the screen DF of the display unit 14 can be detected using the resonant circuit.

The power supply circuit 34 includes, for example, a DC/DC converter, a charging/discharging unit, a battery unit, an AC/DC adapter, etc., and operates the tablet terminal 1 using DC voltage supplied from the AC/DC adapter or the battery unit. Convert to multiple voltages as required. Further, the power supply circuit 34 supplies power to each part of the tablet terminal 1 based on control from the embedded controller 31.

The sensor unit 35 is, for example, an acceleration sensor, a gyro sensor, or the like, and detects the screen display orientation that indicates the usage state of the tablet terminal 1. The sensor unit 35 detects, for example, the direction of gravitational acceleration and the screen display direction of the tablet terminal 1 (display unit 14).

Note that in this embodiment, the CPU 11 and chipset 21 described above correspond to the main control section 10. The main control unit 10 executes processing based on the OS (eg, Windows (registered trademark)).

Next, with reference to FIG. 3, the functional configuration of the tablet terminal 1 according to this embodiment will be described.
FIG. 3 is a block diagram showing an example of the functional configuration of the tablet terminal 1 according to this embodiment.

As shown in FIG. 3, the tablet terminal 1 includes a main control section 10, a touch screen 20, a pen 30, an embedded controller 31, and a storage section 40.
Note that in FIG. 3, only the main functional configurations related to the invention of this embodiment are shown as the configuration of the tablet terminal 1. Furthermore, since the touch screen 20, pen 30, and embedded controller 31 are the same as those shown in FIG. 2 described above, their description will be omitted here. The touch screen 20 includes a display section 14 and a touch sensor section 33, and the touch sensor section 33 includes a contact detection section 331 and a pen detection section 332.

The storage unit 40 is, for example, a storage unit implemented by the main memory 12 or the SSD 23, and stores various information such as OSK input information and pen input information using the pen 30, for example.

The main control unit 10 is a functional unit realized by the CPU 11 and the chipset 21 executing programs stored in the main memory 12, and executes various processes based on the OS. The main control unit 10 includes, for example, a display driver 101, an EC driver 102, an application 103, and a virtual input processing unit 110.

The display driver 101 is a functional unit realized by the CPU 11 and the chipset 21 executing a program stored in the main memory 12, and is a device driver that controls display on the display unit 14. The display driver 101 causes the screen DF of the display unit 14 to display various information (OSK image, handwritten input trajectory, etc.).

The EC driver 102 is a functional unit realized by the CPU 11 and the chipset 21 executing a program stored in the main memory 12, and is a device driver that controls the embedded controller 31. The EC driver 102 controls the embedded controller 31, acquires the touch input and pen input detected by the touch sensor section 33, and supplies them to the virtual input processing section 110, which will be described later.

The application 103 is a functional unit realized by the CPU 11 and the chipset 21 executing a program stored in the main memory 12. The application 103 uses, for example, a virtual input processing unit 110 (described later) to input various information using the OSK and handwritten input using the pen 30.

The virtual input processing unit 110 is a functional unit realized by the CPU 11 and the chipset 21 executing a program stored in the main memory 12, and performs virtual input processing using a virtual input device (handwritten input using an OSK and a pen 30). Execute. The virtual input processing unit 110 can switch between an OSK mode (first mode) in which OSK input processing is executed and a handwriting recognition mode (second mode) in which handwriting input processing using the pen 30 is executed.

Here, the OSK mode is a mode in which at least part of the OSK area AR1 (first area, see FIG. 4 described later) of the screen DF of the display unit 14 functions as a software keyboard based on touch input by the touch sensor unit 33. It is. Furthermore, the handwriting recognition mode is a mode in which the OSK area AR1 functions as a handwriting input device (handwriting recognition device) based on pen input by the touch sensor section 33.

Further, in the OSK mode, when the pen 30 approaches the OSK area AR1 of the software keyboard within a threshold value, the virtual input processing unit 110 switches to the handwriting recognition mode. Note that the virtual input processing unit 110 uses the pen detection unit 332 of the touch sensor unit 33 to determine whether the pen 30 has approached the OSK area AR1 to a distance within a threshold value.

Further, in the handwriting recognition mode, when the touch sensor unit 33 detects a touch input by contacting an object other than a pen (for example, a finger) with respect to the OSK area AR1, the virtual input processing unit 110 operates in the OSK mode Switch to
Further, the virtual input processing section 110 includes an OSK processing section 111, a pen input processing section 112, and a switching processing section 113.

The OSK processing unit 111 executes virtual input processing in OSK mode. The OSK processing unit 111 displays a soft keyboard image (OSK image) in a partial area of the display unit 14 as an OSK area AR1, for example, as in state ST1 in FIG. 4. The OSK processing unit 111 displays the image of the soft keyboard in the OSK area AR1 of the display unit 14 via the display driver 101.

Further, the OSK processing unit 111 acquires the touch input detected by the touch sensor unit 33 (for example, the detected coordinates of the touch input) via the embedded controller 31 and the EC driver 102, and outputs an OSK corresponding to the detected coordinates of the touch input. The key code of the key is output to the application 103 as input information.

The pen input processing unit 112 executes virtual input processing in handwriting recognition mode. The pen input processing unit 112 changes the OSK area AR1 described above into a handwriting input area and displays the trajectory of the pen 30, for example, as shown in state ST2 in FIG. 4. The pen input processing unit 112 displays the trajectory of the pen 30 in the OSK area AR1 via the display driver 101.

Furthermore, in the handwriting recognition mode, the pen input processing unit 112 acquires the pen input detected by the touch sensor unit 33 (for example, the detected coordinates of the pen input) via the embedded controller 31 and the EC driver 102. The pen input processing unit 112 generates a trajectory of the pen 30 input by handwriting based on the acquired pen input (for example, detected coordinates of the pen input), and converts the trajectory of the pen 30 input by handwriting into text information. Then, the text information is output to the application 103 as input information.

The switching processing unit 113 executes switching processing between the OSK mode and the handwriting recognition mode. When the touch sensor unit 33 detects a pen input in the OSK area AR1 in the OSK mode, the switching processing unit 113 switches to the handwriting recognition mode and causes the pen input processing unit 112 to execute virtual input processing in the handwriting recognition mode. . For example, when the pen 30 approaches the OSK area AR1 within a distance of a threshold value (when hovering of the pen 30 is detected), the switching processing unit 113 determines that a pen input is detected in the OSK area AR1. .

In addition, in the handwriting recognition mode, when the touch sensor unit 33 detects a touch input other than the pen 30 in the OSK area AR1, the switching processing unit 113 switches to the OSK mode and inputs a virtual input in the OSK mode to the OSK processing unit 111. Execute the process. Note that when the touch sensor unit 33 detects a touch input (pressing a switching button) to a specific area (for example, an OSK mode switching button) of the OSK area AR1, the switching processing unit 113 It is also possible to switch to OSK mode.

Next, with reference to FIG. 4, switching processing between the OSK mode and the handwriting recognition mode will be described.
FIG. 4 is an image diagram showing an example of mode switching of the tablet terminal 1 according to the present embodiment.

In FIG. 4, a state ST1 indicates an example of the OSK mode state of the tablet terminal 1. In the OSK mode state ST1, the OSK processing unit 111 of the tablet terminal 1 displays an image of a soft keyboard in the OSK area AR1, which is a part of the screen DF of the display unit 14. Furthermore, the display area AR2 (an example of a second area) is an image area of the application 103 to which the virtual input processing unit 110 is linked. The application 103 displays, for example, text information as shown in the display area AR2 on the display unit 14.

In state ST1 of the OSK mode, when the keyboard in the OSK area AR1 is touched, the OSK processing unit 111 converts the key code corresponding to the touched key into, for example, FEP (Front End Processor) or IME (Input Method Editor). The information is output to the application 103 as input information via. The application 103 displays, for example, characters corresponding to the input information in the display area AR2.

Further, state ST2 indicates an example of the state of the handwriting recognition mode of the tablet terminal 1. In the handwriting recognition mode state ST2, the pen input processing unit 112 of the tablet terminal 1 sets a handwriting input area in the OSK area AR1 of the screen DF of the display unit 14.

Furthermore, in state ST2 of the handwriting recognition mode, the pen input processing unit 112 displays the trajectory of handwritten input using the pen 30 in the OSK area AR1, and converts the trajectory of the handwritten input into text information. The pen input processing unit 112 outputs text information to the application 103 as input information via, for example, an FEP (Front End Processor) or an IME (Input Method Editor). The application 103 displays, for example, characters corresponding to the input information in the display area AR2.

Note that in the OSK mode state ST1, the switching processing unit 113 switches to the handwriting recognition mode by detecting the pen 30 by the touch sensor unit 33, and causes a transition to the handwriting recognition mode state ST2.

Furthermore, in the state ST2 of the handwriting recognition mode, the switching processing unit 113 switches to the OSK mode by detecting a touch input from a device other than the pen 30 by the touch sensor unit 33, and causes a transition to the OSK mode state ST1.

Next, the operation of the tablet terminal 1 according to this embodiment will be explained with reference to the drawings.
FIG. 5 is a flowchart illustrating an example of mode switching processing of the tablet terminal 1 according to the present embodiment.

As shown in FIG. 5, the switching processing unit 113 of the tablet terminal 1 first determines whether the tablet terminal 1 is in the handwriting recognition mode (step S101). When the switching processing unit 113 is in the handwriting recognition mode (step S101: YES), the switching processing unit 113 advances the process to step S105. Further, when the handwriting recognition mode is not (OSK mode) (step S101: NO), the switching processing unit 113 advances the process to step S102.

In step S102, the switching processing unit 113 determines whether pen input is detected. The switching processing unit 113 determines whether a pen input has been detected, for example, based on whether the pen 30 has approached the screen DF to a distance within a threshold value by the pen detection unit 332 of the touch sensor unit 33. When the switching processing unit 113 detects pen input (step S102: YES), the switching processing unit 113 advances the process to step S103. Further, if the switching processing unit 113 does not detect pen input (step S102: NO), the switching processing unit 113 returns the process to step S101.

In step S103, the switching processing unit 113 determines whether the detection coordinates of pen detection are in the OSK area AR1. The switching processing unit 113 determines whether the coordinates of the pen 30 on the screen DF detected by the touch sensor unit 33 are within the OSK area AR1. When the detection coordinates of pen detection are in the OSK area AR1 (step S103: YES), the switching processing unit 113 advances the process to step S104. Furthermore, when the detected coordinates of pen detection are not in the OSK area AR1 (step S103: NO), the switching processing unit 113 returns the process to step S101.

In step S104, the switching processing unit 113 switches to handwriting recognition mode. The switching processing unit 113 causes the pen input processing unit 112 to execute virtual input processing in handwriting recognition mode. After the processing in step S104, the switching processing unit 113 returns the processing to step S101.

In step S105, the switching processing unit 113 determines whether a touch input is detected. For example, the switching processing unit 113 determines whether a touch input is detected by the contact detection unit 331 of the touch sensor unit 33, based on whether an object other than the pen 30 (for example, a finger) has touched the screen DF. judge. When the switching processing unit 113 detects a touch input (step S105: YES), the switching processing unit 113 advances the process to step S106. Furthermore, when the switching processing unit 113 does not detect a touch input (step S105: NO), the switching processing unit 113 returns the process to step S101.

In step S106, the switching processing unit 113 switches to OSK mode. The switching processing unit 113 causes the OSK processing unit 111 to execute virtual input processing in OSK mode. After the processing in step S106, the switching processing unit 113 returns the processing to step S101.

Next, details of the OSK processing of the tablet terminal 1 according to the present embodiment will be described with reference to FIG. 6.
FIG. 6 is a flowchart showing an example of the OSK process of the tablet terminal 1 according to the present embodiment.

As shown in FIG. 6, the OSK processing unit 111 of the tablet terminal 1 first displays an image of a soft keyboard in the OSK area AR1 (step S110). The OSK processing unit 111 outputs the soft keyboard image to the display unit 14 via the display driver 101 for display.

Next, the OSK processing unit 111 determines whether a touch input is detected (step S111). The switching processing unit 113 determines whether a touch input has been detected, for example, based on whether the contact detection unit 331 of the touch sensor unit 33 has contacted the screen DF with an object (for example, a finger). When the OSK processing unit 111 detects a touch input (step S111: YES), the OSK processing unit 111 advances the process to step S112. Further, if the OSK processing unit 111 does not detect a touch input (step S111: NO), the process returns to step S111.

In step S112, the OSK processing unit 111 determines whether the detected coordinates of the touch input are in the OSK area AR1. The OSK processing unit 111 determines whether the detection coordinates of the touch input detected by the touch sensor unit 33 are within the OSK area AR1. If the detected coordinates of the touch input are in the OSK area AR1 (step S112: YES), the OSK processing unit 111 advances the process to step S113. Further, if the detected coordinates of the touch input are not in the OSK area AR1 (step S112: NO), the OSK processing unit 111 returns the process to step S111.

In step S113, the OSK processing unit 111 outputs a key code corresponding to the detected coordinates of the touch input to the application 103. After the process in step S113, the OSK processing unit 111 returns the process to step S110.

Next, with reference to FIG. 7, details of the handwriting input process (processing in handwriting recognition mode) of the tablet terminal 1 according to the present embodiment will be described.
FIG. 7 is a flowchart illustrating an example of handwriting input processing of the tablet terminal 1 according to the present embodiment.

As shown in FIG. 7, the pen input processing unit 112 of the tablet terminal 1 first determines whether a pen input is detected (step S120). The pen input processing unit 112 determines whether or not a pen input is detected on the screen DF by the pen detection unit 332 of the touch sensor unit 33, for example. When pen input processing section 112 detects pen input (step S120: YES), it advances the process to step S121. Furthermore, if pen input processing section 112 has not detected pen input (step S120: NO), the process returns to step S120.

In step S121, the pen input processing unit 112 determines whether the detected coordinates of the pen input are in the OSK area AR1. The pen input processing unit 112 determines whether the coordinates of the pen 30 on the screen DF detected by the touch sensor unit 33 are within the OSK area AR1. If the detected coordinates of the pen input are in the OSK area AR1 (step S121: YES), the pen input processing unit 112 advances the process to step S122. Further, if the detected coordinates of the pen input are not in the OSK area AR1 (step S121: NO), the pen input processing unit 112 returns the process to step S120.

In step S122, the pen input processing unit 112 displays the trajectory of the pen input in the OSK area AR1. The pen input processing unit 112 causes the display unit 14 to display the trajectory of the detection coordinates of the pen input detected by the touch sensor unit 33 (the trajectory of the pen input).

Next, the pen input processing unit 112 converts the trajectory of the pen input into text information (step S123). The pen input processing unit 112 converts the trajectory of the pen input into text information using, for example, existing OCR (Optical Character Reader) technology.

Next, the pen input processing unit 112 outputs the text information to the application 103 (step S124). After the process in step S124, the pen input processing unit 112 returns the process to step S120.

As described above, the tablet terminal 1 (information processing device) according to the present embodiment includes the display section 14, the touch sensor section 33, and the virtual input processing section 110. The touch sensor unit 33 is disposed on the screen DF of the display unit 14 and can detect touch input caused by contact with an object on the screen DF and pen input caused by contact with the pen 30. The virtual input processing unit 110 switches to the handwriting recognition mode (second mode) when the pen 30 approaches the OSK area AR1 of the software keyboard within a threshold value in the OSK mode (first mode). Here, the OSK mode (first mode) is a mode in which at least part of the OSK area AR1 (area) of the screen DF functions as a software keyboard based on a touch input. Furthermore, the handwriting recognition mode (second mode) is a mode in which the OSK area AR1 functions as a handwriting input device based on pen input.

Thereby, the tablet terminal 1 according to the present embodiment can seamlessly switch from a software keyboard to handwriting input (handwriting recognition), and can improve usability. Therefore, the tablet terminal 1 according to the present embodiment can improve the user experience pen in virtual input processing.

Further, the tablet terminal 1 (information processing device) according to the present embodiment includes a display section 14, a touch sensor section 33, a switching processing section 113, and a pen input processing section 112. The touch sensor unit 33 is disposed on the screen DF of the display unit 14 and can detect touch input caused by contact with an object on the screen DF and pen input caused by contact with the pen 30. When the pen 30 approaches the OSK area AR1 of the software keyboard within a threshold value in the OSK mode (first mode), the switching processing unit 113 changes the OSK area AR1 to the handwriting recognition mode (second mode). mode). Furthermore, in the handwriting recognition mode, when the touch sensor unit 33 detects a touch input by contacting an object other than a pen with respect to the OSK area AR1, the switching processing unit 113 switches to the OSK mode. In the handwriting recognition mode, the pen input processing unit 112 converts the trajectory of the pen 30 input by handwriting into text information, and outputs the text information as input information.

Thereby, the tablet terminal 1 according to the present embodiment can seamlessly switch between the software keyboard and handwriting input (handwriting recognition), and can further improve usability. In the tablet terminal 1 according to the present embodiment, for example, when inputting characters or symbols that are not on the software keyboard in the OSK mode, it is possible to quickly and easily switch to the handwriting recognition mode, which is seamless handwriting input (handwriting recognition). You can enter characters or symbols that are not on the software keyboard. Therefore, the tablet terminal 1 according to the present embodiment can improve the user experience pen in virtual input processing.

In addition, the control method according to the present embodiment includes a display unit 14, which is arranged on the screen DF of the display unit 14, and is capable of detecting a touch input caused by contact with an object on the screen DF and a pen input caused by contact with the pen 30. This is a method for controlling a tablet terminal 1 (information processing device) including a touch sensor section 33, and includes a switching process step. In the switching processing step, the virtual input processing unit 110 (switching processing unit 113) detects handwriting when the pen 30 approaches the OSK area AR1 of the software keyboard within a threshold value in the OSK mode (first mode). Switch to recognition mode (second mode).

Thereby, the control method according to the present embodiment has the same effect as the tablet terminal 1 described above, can seamlessly switch from the software keyboard to handwriting input (handwriting recognition), and can improve the user experience of the pen.

In addition, the control method according to the present embodiment includes a display unit 14, which is arranged on the screen DF of the display unit 14, and is capable of detecting a touch input caused by contact with an object on the screen DF and a pen input caused by contact with the pen 30. This is a method for controlling a tablet terminal 1 (information processing device) including a touch sensor section 33, and includes a switching processing step and a pen input processing step. In the switching processing step, in addition to the switching processing to the handwriting recognition mode described above, the switching processing unit 113 performs a touch sensor unit 33 in the handwriting recognition mode to detect a touch caused by contact with an object other than a pen on the OSK area AR1. When input is detected, switch to OSK mode. In the pen input process, the pen input processing unit 112 converts the trajectory of the pen 30 input by handwriting into text information in the handwriting recognition mode, and outputs the text information as input information.

As a result, the control method according to the present embodiment has the same effect as the tablet terminal 1 described above, can seamlessly switch between the software keyboard and handwriting input (handwriting recognition), and improves the user experience of the pen. I can do it.

Note that the tablet terminal 1 (information processing device) according to this embodiment may have the following form. The tablet terminal 1 (information processing device) is arranged on the display unit 14 and the screen DF of the display unit 14, and is capable of detecting touch input caused by contact with an object on the screen DF and pen input caused by contact with the pen 30. It includes a touch sensor section 33, a main memory 12 (memory) that temporarily stores programs, and a CPU 11 (processor) that executes the programs stored in the main memory 12. The CPU 11 (processor) executes a mode switching process by executing a program stored in the main memory 12. In the mode switching process, the CPU 11 (processor) selects the OSK area of the software keyboard in an OSK mode (first mode) in which at least part of the OSK area AR1 (area) of the screen DF functions as a software keyboard based on the touch input. When the pen 30 approaches AR1 within a distance of a threshold value, the OSK area AR1 is switched to a handwriting recognition mode (second mode) in which it functions as a handwriting input device based on the pen input.

As a result, the tablet terminal 1 according to the present embodiment has the same effect as the control method described above, can seamlessly switch from the software keyboard to handwriting input (handwriting recognition), and can improve the user experience of the pen.

[Second embodiment]
Next, a tablet terminal 1a according to a second embodiment will be described with reference to the drawings.
FIG. 8 is a block diagram showing an example of the functional configuration of the tablet terminal 1a according to the second embodiment.

In this embodiment, a modification example in which the processing of the handwriting recognition mode is changed will be described.
Note that the appearance and hardware configuration of the tablet terminal 1a according to the second embodiment are the same as those of the first embodiment shown in FIGS. 1 and 2 described above, and therefore the description thereof will be omitted here.

As shown in FIG. 8, the tablet terminal 1a according to the present embodiment includes a main control section 10a, a touch screen 20, a pen 30, an embedded controller 31, and a storage section 40.

Note that in FIG. 8, the same components as those in FIG. 3 described above are given the same reference numerals, and the description thereof will be omitted. Further, the embedded controller 31 in this embodiment includes a coordinate transformation processing section 311.

The main control unit 10a is a functional unit realized by the CPU 11 and the chipset 21 executing programs stored in the main memory 12, and executes various processes based on the OS. The main control unit 10a includes, for example, a display driver 101, an EC driver 102, an application 103, and a virtual input processing unit 110a.

The virtual input processing unit 110a is a functional unit realized by the CPU 11 and the chipset 21 executing a program stored in the main memory 12, and performs virtual input processing using a virtual input device (handwritten input using an OSK and a pen 30). Execute. Similar to the virtual input processing unit 110 of the first embodiment, the virtual input processing unit 110a has an OSK mode (first mode) in which OSK input processing is executed, and a handwriting recognition mode in which handwriting input processing using the pen 30 is executed. It can be executed by switching the mode (second mode).

Note that in the OSK mode, the virtual input processing unit 110a switches to the handwriting recognition mode when the pen 30 approaches the OSK area AR1 of the software keyboard to a distance within a threshold value.

Furthermore, in the handwriting recognition mode, the virtual input processing unit 110a has the touch sensor unit 33 move the pen 30 to a specific area (for example, the OSK mode switching button BT1 shown in FIG. 9) of the OSK area AR1. When a touch input due to contact with an included object is detected, the mode is switched to OSK mode.
Further, the virtual input processing section 110a includes an OSK processing section 111, a pen input processing section 112a, and a switching processing section 113a.

The pen input processing unit 112a executes virtual input processing in handwriting recognition mode. For example, as shown in state ST2a in FIG. 9, the pen input processing unit 112a enlarges the area around the character input cursor CS1 specified in a display area AR2 (second area) different from the OSK area AR1, and displays the area in the OSK area AR1. At the same time, the trajectory PS1 of the pen 30 input by handwriting is displayed in the OSK area AR1 based on the pen input. The pen input processing unit 112a displays the trajectory of the pen 30 in the OSK area AR1 via the display driver 101.

Further, the pen input processing unit 112a further converts the size of the trajectory PS1 of the pen 30 and displays it at a corresponding position in the display area AR2, and uses the converted trajectory PS1 of the pen 30 as input information in the application. 103. Note that the pen input processing unit 112a acquires the detected coordinates of the pen input converted for the display area AR2 of the application 103 from the embedded controller 31, and determines the trajectory PS1 of the pen 30 based on the converted detected coordinates of the pen input. The size of is converted so that it can be displayed in the display area AR2.

The switching processing unit 113a executes switching processing between the OSK mode and the handwriting recognition mode. The switching processing unit 113a performs the switching processing from the OSK mode to the handwriting recognition mode in the same manner as the switching processing unit 113 of the first embodiment described above.

Further, in the handwriting recognition mode, when the touch sensor unit 33 detects a touch input to a specific area (switching button BT1 shown in FIG. 9) of the OSK area AR1, the switching processing unit 113a controls the OSK mode, and causes the OSK processing unit 111 to execute virtual input processing in the OSK mode.

The coordinate conversion processing unit 311 converts the touch sensor based on the position and size of the character input cursor CS1 in the display area AR2 acquired in advance from the virtual input processing unit 110a and the position and enlarged size of the character input cursor CS1 in the OSK area AR1. The detected coordinates of the pen input detected by the unit 33 are converted into the detected coordinates of the pen input for the display area AR2. The coordinate conversion processing unit 311 outputs the converted pen input detection coordinates for the display area AR2 to the pen input processing unit 112a.

Next, with reference to FIG. 9, switching processing between the OSK mode and the handwriting recognition mode in this embodiment will be described.
FIG. 9 is an image diagram showing an example of mode switching of the tablet terminal 1a according to the present embodiment.

In FIG. 9, a state ST1 indicates an example of the OSK mode state of the tablet terminal 1a. In the OSK mode state ST1, the OSK processing unit 111 of the tablet terminal 1a displays an image of a soft keyboard in the OSK area AR1, which is a part of the screen DF of the display unit 14. Furthermore, the display area AR2 (an example of a second area) is an image area of the application 103 to which the virtual input processing unit 110a is linked. The application 103 displays, for example, text information and a character input cursor CS1 as shown in the display area AR2 on the display unit 14.

Note that the processing of the OSK processing unit 111 in this embodiment is the same as that in the first embodiment, so the description thereof will be omitted here.

Moreover, state ST2a shows an example of the state of the handwriting recognition mode of the tablet terminal 1a. In the handwriting recognition mode state ST2a, the pen input processing unit 112a of the tablet terminal 1a displays an enlarged image of the periphery of the character input cursor CS1 in the display area AR2 in the OSK area AR1 of the screen DF of the display unit 14. The pen input processing unit 112a also supplies the position and size of the character input cursor CS1 in the display area AR2 and the position and enlarged size of the character input cursor CS1 in the OSK area AR1 to the embedded controller 31.

Furthermore, in state ST2a of the handwriting recognition mode, the pen input processing unit 112a acquires the detection coordinates of the pen input detected by the touch sensor unit 33 and the detection coordinates of the pen input converted by the coordinate conversion processing unit 311 of the embedded controller 31. Then, the trajectory PS1 of the handwritten input using the pen 30 is displayed in the OSK area AR1. Furthermore, the pen input processing unit 112a displays a handwritten input trajectory PS2 corresponding to the handwritten input trajectory PS1 in the display area AR2.

Note that the pen input processing unit 112a outputs the handwritten input trajectory PS2 to the application 103 as input information. The application 103 displays, for example, characters corresponding to the input information in the display area AR2.

In the handwriting recognition mode state ST2a, the switching processing unit 113a switches the OSK mode to to make the transition to the OSK mode state ST1.

Next, the operation of the tablet terminal 1a according to this embodiment will be described with reference to the drawings.
FIG. 10 is a flowchart showing an example of the mode switching process of the tablet terminal 1a according to this embodiment.

As shown in FIG. 10, the switching processing unit 113a of the tablet terminal 1a first determines whether the tablet terminal 1a is in the handwriting recognition mode (step S201).
Note that in FIG. 10, the processing from step S201 to step S204 is the same as the processing from step S101 to step S104 shown in FIG. 5 described above, so the description thereof will be omitted here. When the switching processing unit 113a is in the handwriting recognition mode (step S201: YES), the switching processing unit 113a advances the process to step S205.

In step S205, the switching processing unit 113a determines whether pressing of the OSK switching button BT1 is detected. When the switching processing unit 113a detects that the switching button BT1 is pressed (step S205: YES), the switching processing unit 113a advances the process to step S206. Further, if the switching processing unit 113a does not detect that the switching button BT1 has been pressed (step S205: NO), the switching processing unit 113a returns the process to step S201.

In step S206, the switching processing unit 113a switches to OSK mode. The switching processing unit 113a causes the OSK processing unit 111 to execute virtual input processing in OSK mode. After the processing in step S206, the switching processing unit 113a returns the processing to step S201.

Next, the details of the OSK process of the tablet terminal 1a according to the present embodiment are the same as the process shown in FIG. 6 described above, so the description thereof will be omitted here.

Next, details of handwriting input processing (processing in handwriting recognition mode) of the tablet terminal 1a according to the present embodiment will be described with reference to FIG. 11.
FIG. 11 is a flowchart showing an example of handwriting input processing of the tablet terminal 1a according to the present embodiment.

As shown in FIG. 11, the pen input processing unit 112a of the tablet terminal 1a first obtains the position and size of the cursor (step S210). The pen input processing unit 112a acquires the current position and size of the character input cursor CS1 on the display area AR2 from the application 103.

Next, the pen input processing unit 112a displays an enlarged image around the cursor in the OSK area AR1 (step S211). The pen input processing unit 112a acquires image data around the character input cursor CS1 from the application 103, generates an image enlarged to the size of the OSK area AR1 based on the position and size of the cursor described above, and generates an image around the cursor. The enlarged image is displayed on the display unit 14 via the display driver 101.

Next, the pen input processing unit 112a determines whether or not pen input is detected (step S212). When the pen input processing unit 112a detects a pen input (step S212: YES), the pen input processing unit 112a advances the process to step S213. Further, if pen input processing section 112a does not detect pen input (step S212: NO), the process returns to step S210.

In step S213, the pen input processing unit 112a determines whether the detected coordinates of the pen input are in the OSK area AR1. If the detected coordinates of the pen input are in the OSK area AR1 (step S213: YES), the pen input processing unit 112a advances the process to step S214. Further, if the detected coordinates of the pen input are not in the OSK area AR1 (step S213: NO), the pen input processing unit 112a returns the process to step S214.

In step S214, the pen input processing unit 112a displays an enlarged image around the cursor including the trajectory of the pen input in the OSK area AR1. The pen input processing unit 112a displays an enlarged image (see OSK area AR1 in state ST2a in FIG. It is displayed on the display section 14 via.

Next, the pen input processing unit 112a converts the detected coordinates of the pen input based on the position and size of the cursor (step S215). The pen input processing unit 112a transmits, for example, the position and size of the character input cursor CS1 in the display area AR2 and the position and enlarged size of the character input cursor CS1 in the OSK area AR1 to the coordinate conversion processing unit 311 of the embedded controller 31. Then, the detected coordinates of the pen input converted for the display area AR2 are acquired from the coordinate conversion processing unit 311. In this way, the pen input processing section 112a uses the coordinate conversion processing section 311 to convert the detected coordinates of the pen input for the display area AR2.

Next, the pen input processing unit 112a outputs the detected coordinates of the converted pen input to the application 103 (step S215). After the process in step S215, the pen input processing unit 112a returns the process to step S210.

As described above, the tablet terminal 1a (information processing device) according to the present embodiment includes the display section 14, the touch sensor section 33, and the virtual input processing section 110a. The virtual input processing section 110a includes a switching processing section 113a and a pen input processing section 112a. The touch sensor unit 33 is disposed on the screen DF of the display unit 14 and can detect touch input caused by contact with an object on the screen DF and pen input caused by contact with the pen 30. The switching processing unit 113a switches to the handwriting recognition mode (second mode) when the pen 30 approaches the OSK area AR1 of the software keyboard within a threshold value in the OSK mode (first mode). Here, the OSK mode (first mode) causes at least part of the OSK area AR1 (first area) of the screen DF to function as a software keyboard based on a touch input. The handwriting recognition mode (second mode) causes the OSK area AR1 to function as a handwriting input device based on pen input. Furthermore, when the touch sensor unit 33 detects a touch input to a specific area (for example, the switching button BT1) of the OSK area AR1 in the handwriting recognition mode, the switching processing unit 113a switches the mode to the OSK mode. Switch. In the handwriting recognition mode, the pen input processing unit 112a enlarges the area around the character input cursor specified in the display area AR2 (second area) different from the OSK area AR1 and displays it in the OSK area AR1, and also performs pen input. Based on this, the handwritten trajectory of the pen 30 is displayed in the OSK area AR1. The pen input processing unit 112a further converts the size of the trajectory of the pen 30 and displays it at a corresponding position in the display area AR2, and sends the converted trajectory of the pen 30 to, for example, the application 103 as input information. Output.

Thereby, the tablet terminal 1a according to the present embodiment can seamlessly switch between the software keyboard and handwriting input (handwriting recognition), and can further improve usability. In the tablet terminal 1a according to the present embodiment, for example, in OSK mode, when inputting characters or symbols that are not on the software keyboard, switching to handwriting recognition mode, which is seamless handwriting input (handwriting recognition), allows quick and easy operation. You can enter characters or symbols that are not on the software keyboard. Therefore, the tablet terminal 1a according to the present embodiment can improve the user experience pen in virtual input processing.

Note that the present invention is not limited to the above-described embodiments, and can be modified without departing from the spirit of the present invention.
For example, in each of the above embodiments, an example has been described in which the information processing device is the tablet terminal 1 (1a), but the information processing device is not limited to this. It may also be a processing device.

For example, in a notebook PC, the main screen, which is part of the B cover, is used while standing up, and if you try to input with a pen on the B cover screen, there is a problem that it is difficult to input because the screen is not horizontal. . On the other hand, when the present invention is applied to a notebook PC, it is possible to easily switch from OSK to pen input using the screen of the C cover part, so it is possible to improve the user experience of the pen. can.

Further, in each of the above embodiments, an example has been described in which the tablet terminal 1 (1a) is a notebook PC-based information processing device equipped with the embedded controller 31. However, the invention is not limited to this, and the tablet terminal 1 (1a) The information processing device may not be equipped with the above information. Further, the OS of the information processing device is not limited to Windows (registered trademark), and may be another OS such as Android (registered trademark).

Further, in each of the above embodiments, the embedded controller 31 executes a part of the processing executed by the main control unit 10 (virtual input processing unit 110) or the main control unit 10a (virtual input processing unit 110a). Good too. For example, in the first embodiment, the embedded controller 31 may execute the process of converting the trajectory of the pen 30 into text information.

Further, in each of the above embodiments, a part of the processing executed by the embedded controller 31 is executed by the main control unit 10 (virtual input processing unit 110) or the main control unit 10a (virtual input processing unit 110a). Good too. For example, in the second embodiment, the processing executed by the coordinate transformation processing section 311 may be executed by the main control section 10a (virtual input processing section 110a).

Note that each component included in the tablet terminal 1 (1a) described above includes a computer system therein. Then, a program for realizing the functions of each component of the tablet terminal 1 (1a) described above is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into a computer system and executed. By doing so, processing in each configuration included in the tablet terminal 1 (1a) described above may be performed. Here, "reading a program recorded on a recording medium into a computer system and executing it" includes installing the program on the computer system. The "computer system" here includes hardware such as an OS and peripheral devices.
Further, a "computer system" may include a plurality of computer devices connected via a network including the Internet, a WAN, a LAN, a communication line such as a dedicated line, etc. Furthermore, the term "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, and storage devices such as hard disks built into computer systems. In this way, the recording medium storing the program may be a non-transitory recording medium such as a CD-ROM.

The recording medium also includes a recording medium provided internally or externally that can be accessed from the distribution server for distributing the program. Note that the program may be divided into a plurality of parts, each downloaded at different timings, and then combined with each component provided in the tablet terminal 1 (1a), or the distribution servers that distribute each of the divided programs may be different. . Furthermore, a ``computer-readable recording medium'' refers to a storage medium that retains a program for a certain period of time, such as volatile memory (RAM) inside a computer system that serves as a server or client when a program is transmitted via a network. This shall also include things. Moreover, the above-mentioned program may be for realizing a part of the above-mentioned functions. Furthermore, it may be a so-called difference file (difference program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

Further, some or all of the functions described above may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each of the above-mentioned functions may be implemented as an individual processor, or some or all of them may be integrated into a processor. Moreover, the method of circuit integration is not limited to LSI, but may be implemented using a dedicated circuit or a general-purpose processor. Further, if an integrated circuit technology that replaces LSI emerges due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

1, 1a Tablet terminal 10, 10a Main control unit 11 CPU
12 Main memory 13 Video subsystem 14 Display unit 20 Touch screen 21 Chipset 22 BIOS memory 23 SSD
24 USB connector 25 audio system 26 WLAN card 30 pen 31 embedded controller 32 key input section 33 touch sensor section 34 power supply circuit 35 sensor section 40 storage section 101 display driver 102 EC driver 103 application 110 virtual input processing section 111 OSK processing section 112, 112a Pen input processing unit 113, 113a Switching processing unit 311 Coordinate conversion processing unit 331 Contact detection unit 332 Pen detection unit AR1 OSK area AR2 Display area CS Housing DF Screen LC1 Resonance circuit

Claims (3)

A display section;
a touch sensor unit disposed on the screen of the display unit and capable of detecting touch input caused by contact with an object on the screen and pen input caused by contact with a pen;
In a first mode in which at least a partial area of the screen functions as a software keyboard based on the touch input, when the pen approaches the area of the software keyboard to a distance within a threshold value, the area and a virtual input processing unit that switches the input device to a second mode in which the input device functions as a handwriting input device based on the pen input. A display section;
a touch sensor unit disposed on the screen of the display unit and capable of detecting touch input caused by contact with an object on the screen and pen input caused by contact with a pen;
In a first mode in which at least a part of the screen functions as a software keyboard based on the touch input, when the pen approaches the area of the software keyboard to a distance within a threshold value, The area is switched to a second mode in which the area functions as a handwriting input device based on the pen input; a switching processing unit that switches to the first mode when a touch input is detected;
An information processing device comprising: a pen input processing unit that converts a handwritten trajectory of the pen into text information and outputs the text information as input information in the second mode. A display section;
a touch sensor unit disposed on the screen of the display unit and capable of detecting touch input caused by contact with an object on the screen and pen input caused by contact with a pen;
In a first mode in which a first area of at least a portion of the screen functions as a software keyboard based on the touch input, the pen approaches the first area of the software keyboard at a distance within a threshold value. In this case, the first area is switched to a second mode in which the first area functions as a handwriting input device based on the pen input, and in the second mode, the touch sensor section is configured to switch to a second mode in which the first area functions as a handwriting input device based on the pen input; a switching processing unit that switches to the first mode when the touch input is detected;
In the second mode, an area around a character input cursor specified in a second area different from the first area is enlarged and displayed in the first area, and a handwritten input is performed based on the pen input. Displaying the trajectory of the pen in the first area, converting the size of the trajectory of the pen and displaying it in a corresponding position in the second area, and inputting the converted trajectory of the pen into input information. An information processing device comprising: a pen input processing unit that outputs as .

Images