JP7463914B2 - Input reception program and input reception system - Google Patents

Description

The present invention relates to an input reception program, etc.

Electronic whiteboard systems are used when multiple people are sharing information or discussing something during a meeting or class. In an electronic whiteboard system, users input annotations and other information by handwriting while scrolling a logically large screen. The following conventional technologies exist for switching between scrolling and handwriting input in an electronic whiteboard system.

For example, in prior art 1, a grip area is provided on part of the screen of an electronic whiteboard, and when an operation is received in the grip area, the screen is scrolled, and when input is received in an area other than the grip area, the input trajectory is displayed as handwritten input.

There is also prior art 2, which automatically scrolls the screen when handwritten input of a specific shape, such as a punctuation mark, is accepted.

International Publication No. 2012/108350 JP 2010-176190 A JP 2012-168620 A JP 2016-162032 A Japanese Patent Application Laid-Open No. 3-278286 JP 2018-152129 A

However, the above-mentioned conventional technology has the problem that the user cannot selectively use both handwriting input and scrolling without giving a special instruction.

For example, in conventional technology 1, scrolling cannot be performed outside the grip area, which is inconvenient. In addition, in conventional technology 2, scrolling does not occur automatically unless a specific shape is handwritten.

When using an electronic whiteboard system, it is desirable to be able to input handwritten text without hassle while still maintaining the convenience of being able to scroll anywhere on the electronic whiteboard. Furthermore, this issue is not limited to electronic whiteboard systems, but can also occur with tablets and PCs (Personal Computers) that have touch panel functionality that allows handwritten input and scrolling operations.

In one aspect, the present invention aims to provide an input reception program and an input reception system that allow a user to selectively use both handwritten input and scrolling operations without giving any special instructions.

In the first proposal, the computer executes the following process. When the computer receives a drag operation on a coordinate detection device that detects the coordinates at which a touch operation is received, the computer scrolls the screen displayed on the display device according to the trajectory of the drag operation. The computer stores information about the trajectory received during scrolling in a storage device. When the computer determines that a predetermined number or more of end points, start points, or bend points have occurred on the trajectory, the computer returns the display to the screen at the time the scrolling began, and displays the trajectory using the trajectory information stored in the storage device.

The user can use both handwriting input and scrolling without having to give any special instructions.

FIG. 1 is a diagram for explaining the process of an input reception system according to the present embodiment. FIG. 2 is a functional block diagram showing the configuration of an information processing apparatus according to the present embodiment. FIG. 3 is a diagram illustrating an example of a data structure of the trajectory information table. FIG. 4 is a flowchart showing the counting process executed by the determination unit. FIG. 5 is a flowchart (1) showing the processing procedure of the input reception system. FIG. 6 is a flowchart (2) showing the processing procedure of the input reception system. FIG. 7 is a flowchart (3) showing the processing procedure of the input reception system. FIG. 8 is a flowchart (4) showing the processing procedure of the input reception system. FIG. 9 is a flowchart (5) showing the processing procedure of the input reception system. FIG. 10 is a flowchart (6) showing the processing procedure of the input reception system. FIG. 11 is a flowchart showing the procedure of other counting processes. FIG. 12 is a diagram illustrating an example of a hardware configuration of a computer that realizes the same functions as the information processing apparatus of the embodiment.

Below, examples of the input reception program and input reception system disclosed in the present application are described in detail with reference to the drawings. Note that the present invention is not limited to these examples.

FIG. 1 is a diagram for explaining the processing of the input reception system according to the present embodiment. In this embodiment, the explanation will be given using a touch panel 110 and a stylus pen 50 as an example of a pointing device. The touch panel 110 is connected to an information processing device and displays screen information output from the information processing device. For example, the screen information includes a plurality of objects 10a and 10b. When the touch panel 110 receives a touch operation by the stylus pen 50, it outputs information related to the touch operation to the information processing device. The information related to the touch operation includes information on the input coordinates by the touch operation, etc. The user operates the stylus pen 50 to perform various touch operations.

When a drag operation (touch operation) is performed on the touch panel 110, the information processing device scrolls the screen of the touch panel 110 according to the trajectory of the drag operation. The information processing device stores in a storage unit trajectory information received during scrolling and screen movement information from the start of scrolling.

When the information processing device determines that a predetermined number or more of end points, start points, or bend points have occurred in the trajectory, the information processing device returns the display of the screen of the touch panel 110 to the screen at the time when scrolling started. In addition, the information processing device uses the trajectory information stored in the memory unit to display the trajectory (a trajectory equivalent to handwritten characters) on the touch panel 110.

On the other hand, if the trajectory does not have a predetermined number of end points, start points, or bends, the information processing device leaves the screen displayed on the touch panel 110 as is and does not display the trajectory on the touch panel 110.

Step S10 in FIG. 1 will be described. When the information processing device accepts a drag operation on the touch panel 110, it scrolls the screen of the touch panel 110 according to the trajectory 50a. As a result, the objects 10a and 10b displayed on the touch panel 110 move downward. The information processing device stores the coordinates (x _a1 , y _a1 ) in the storage unit as the movement information of the screen from before the start of scrolling. This means that the coordinates of the reference point before the start of scrolling have moved to the coordinates (x _a1 , y _a1 ) by the scroll operation. For example, the information processing device sets the reference point before the start of scrolling to the upper left corner (origin) of the screen. Since the predetermined number or more of start points, end points, or bend points have not occurred for the trajectory 50a, the information processing device leaves the screen of the touch panel 110 as it is.

When the information processing apparatus does not receive a touch operation (drag operation) for a predetermined time, it erases the coordinates (x _a1 , y _a1 ) that are information on the amount of movement stored in the storage unit.

Step S11 in FIG. 1 will now be described. When the information processing device receives a drag operation on the touch panel 110, it scrolls the screen of the touch panel 110 in small increments according to the trajectory 50b. This causes the objects 10a and 10b displayed on the touch panel 110 to move in small increments.

The information processing device stores information about the movement of the screen from the start of scrolling in a storage unit. Because the trajectory 50b does not have a predetermined number of start points, end points, or bend points, the information processing device leaves the screen of the touch panel 110 as it is. The information processing device stores information about the trajectory 50b in a storage unit.

Step S12 in FIG. 1 will now be described. When the information processing device accepts a drag operation on the touch panel 110, it scrolls the screen of the touch panel 110 in small increments according to the trajectory 50c. This causes the objects 10a and 10b displayed on the touch panel 110 to move in small increments. Because the trajectory 50c does not have a predetermined number of starting points, ending points, or bending points, the information processing device leaves the screen of the touch panel 110 as it is. The information processing device stores information about the trajectory 50c in a storage unit. The information processing device updates the information about the movement of the screen from the start of scrolling that is stored in the storage unit.

Step S13 in FIG. 1 will be described. When the information processing device accepts a drag operation on the touch panel 110, the information processing device scrolls the screen of the touch panel 110 finely according to the trajectory 50d. As a result, the objects 10a and 10b displayed on the touch panel 110 move finely. The information processing device stores information on the trajectory 50d in a storage unit. In addition, when the coordinates corresponding to the reference point before the start of scrolling have moved to coordinates (x _a2 , y _a2 ) by the scroll operation in steps S11 to S13, the coordinates (x _a2 , y _a2 ) are stored in the storage unit as movement information. Here, when a predetermined number or more of start points, end points, or bend points are generated for the trajectory 50d, the information processing device executes the process shown in step S14.

Step S14 in Fig. 1 will be described. When the information processing device determines that a predetermined number or more of end points, start points, or bend points have occurred in the trajectory 50d, the information processing device returns the display to the screen at the time when the scrolling started, and displays the trajectory 50d on the touch panel 110 using the information of the trajectory 50d stored in the storage unit. For example, when the coordinates (x _a2 , y _a2 ) are stored in the storage unit as the movement information, the information processing device moves each coordinate of the image by (-x _a2 , -y _a2 ), respectively, to return the display to the screen at the time when the scrolling started.

As described in FIG. 1, the information processing device according to this embodiment scrolls the screen in response to a drag operation. When the information processing device determines that the input is handwritten input based on the end point, start point, or bend point of the drag operation trajectory, the information processing device returns the scroll position to the start position and displays the characters of the input trajectory. This allows the user to selectively use both handwritten input and scrolling without having to give any special instructions.

Next, an example of the configuration of an information processing device according to this embodiment will be described. The information processing device is an example of an "input reception system". FIG. 2 is a functional block diagram showing the configuration of an information processing device according to this embodiment. As shown in FIG. 2, this information processing device 100 is connected to a touch panel 110. The information processing device 100 has a communication unit 120, a storage unit 140, and a control unit 150. Here, a case will be described in which the information processing device 100 and the touch panel 110 are externally connected, but the information processing device 100 may also have a touch panel 110.

The touch panel 110 is connected to the information processing device 100 and displays screen information output from the information processing device 100. When the touch panel 110 receives a touch operation by a stylus pen 50 or the like, it outputs information about the touch operation (drag operation) to the information processing device 100. The information about the touch operation includes information about the input coordinates where the touch operation is performed. The touch panel 110 corresponds to a coordinate detection device and a display device.

The communication unit 120 is connected to an external device or the like via a wired or wireless connection, and transmits and receives information to and from the external device or the like. For example, the communication unit 120 is realized by a NIC (Network Interface Card) or the like. The communication unit 120 may be connected to a network (not shown).

The timer 130 outputs the current date and time information to the control unit 150.

The storage unit 140 has screen information 141, a trajectory information table 142, and movement information 143. The storage unit 140 is realized, for example, by a semiconductor memory element such as a random access memory (RAM) or a flash memory, or a storage device such as a hard disk or an optical disk.

Screen information 141 is information about the screen to be displayed on touch panel 110. Screen information 141 includes images of objects 10a and 10b described in FIG. 1. For example, the logical size of screen information 141 is larger than the size of the display screen of touch panel 110.

The trajectory information table 142 is a table that holds information on trajectories created by touch operations (drag operations). For example, the trajectory information table 142 holds information on input coordinates created by touch operations at regular intervals. FIG. 3 is a diagram showing an example of the data structure of the trajectory information table. As shown in FIG. 3, this trajectory information table 142 associates input times with input coordinates. The input time indicates the time when an input created by a touch operation was accepted. The input coordinates indicate the coordinates on the touch panel 110 where an input created by a touch operation was accepted.

The movement information 143 includes information about the movement of the screen from the start of the scroll operation to the current time. The movement information is indicated by the coordinates of a reference point. For example, if the coordinates ( _xd , _yd ) of the reference point are registered as the movement information, it means that the reference point in the upper left corner at the start of the scroll operation has moved to the coordinates ( _xd , _yd ).

Returning to the explanation of FIG. 2, the control unit 150 has an input reception unit 151, a trajectory registration processing unit 152, a movement information registration unit 153, a determination unit 154, a state reset unit 155, and a screen control unit 156. The control unit 150 is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). The control unit 150 may also be executed by an integrated circuit, for example, an ASIC (Application specific Integrated Circuit) or an FPGA (Field Programmable gate Array).

The input reception unit 151 is a processing unit that receives information on input coordinates by a touch operation from the touch panel 110. The input reception unit 151 outputs the received information on the input coordinates to the trajectory registration processing unit 152, the state reset unit 155, and the screen control unit 156.

The trajectory registration processing unit 152 is a processing unit that acquires information on a trajectory created by a touch operation from the input receiving unit 151, and registers the trajectory information in the trajectory information table 142. For example, the trajectory registration processing unit 152 acquires input coordinates from the input receiving unit 151, associates the acquired time (input time) with the input coordinates, and registers them in the trajectory information table 142. The trajectory registration processing unit 152 acquires the input time from the timer 130.

When a scroll operation is started, the movement information registration unit 153 sets a reference point at a position corresponding to the upper left corner of the display screen of the touch panel 110. When a scroll operation is started, the coordinates of the reference point are (0, 0). When the coordinates of the reference point are moved to (x _B , y _B ) by an operation of scrolling the screen by the screen control unit 156 described later, the movement information registration unit 153 registers the coordinates (x _B , y _B ) of the reference point of the destination in the movement information 143. By moving the coordinates of the screen information of the touch panel 110 by (-x _B , -y _B ), the display screen at the start of the scroll operation is returned to.

In this embodiment, the reference point at the time when the scroll operation is started is set to the upper left corner of the display screen of the touch panel 110, but this is not limited to this. For example, the movement information registration unit 153 may set the center position of the object at the time when the scroll operation is started as the reference point.

The determination unit 154 is a processing unit that determines whether or not the current touch operation (drag operation) is a handwriting input based on the trajectory information registered in the trajectory information table 142. The determination unit 154 outputs the determination result to the screen control unit 156. As described below, the determination unit 154 executes a counting process and a determination process. The determination unit 154 also executes a process for determining a tap input and a process for determining the end of handwriting input.

The counting process executed by the determination unit 154 will be described. The determination unit 154 counts the number of bend points in the shape of the trajectory through the counting process. In the counting process, the start point and end point of the trajectory are ignored, and the bend points are counted as one connected trajectory. For example, if a straight trajectory is input in parallel in the shape of "=" based on information about the touch operation from the touch panel 110, the determination unit 154 determines the number of bend points to be "2". The breakdown of this bend point is one at the end point of the first drag operation and another at the start point of the second drag operation.

Figure 4 is a flowchart showing the counting process executed by the determination unit. As shown in Figure 4, the determination unit 154 refers to the trajectory information table 142 and calculates the distance and angle between adjacent points in time among the points (input coordinates) that make up the trajectory (step S50).

The determination unit 154 tracks the points by tracing the trajectory from the beginning and adding up the distance between the points until the distance reaches a certain level (step S51). When the distance reaches a certain level, the determination unit 154 divides the trajectory and calculates the angle between the first point and the last point of the division (step S52).

The determination unit 154 tracks the points by adding the distance between the last point of the segment and the next point until the distance reaches a certain level (step S53). The determination unit 154 compares the angle of the previous segment with the angle of the current segment (step S54). If the angle difference is equal to or greater than a certain angle, the determination unit 154 counts up the number of bends (step S55).

If there are remaining points in the trajectory information table 142 (step S56, Yes), the determination unit 154 proceeds to step S53. On the other hand, if there are no remaining points in the trajectory information table 142 (step S56, No), the determination unit 154 ends the process.

Here, the process by which the determination unit 154 counts the bending points has been described with reference to Figure 4, but any conventional technology may be used to count the bending points of the trajectory.

The following describes the determination process executed by the determination unit 154. The determination unit 154 identifies the number of bends and the number of start points or end points for the trajectory information stored in the trajectory information table 142, and calculates an evaluation value by dividing the number by the time required to input the trajectory. For example, the determination unit 154 calculates the evaluation value R based on formula (1).

R = (α × Nc + β × Nd) / T ... (1)

In formula (1), Nc corresponds to the number of bending points counted by the above-mentioned counting process. Nd corresponds to the number of starting points. α and β are constants that are set in advance. Here, Nd is the number of starting points, but is not limited to this, and may be the number of end points, or the total number of starting points and end points. Here, the starting point is the point where the stylus pen 50 is pressed, and the end point is the point where the stylus pen 50 is released.

The determination unit 154 refers to the trajectory information table 142 and counts the first input coordinate in the trajectory information table 142 as the starting point. The determination unit 154 calculates the difference between the previous and next input times, and determines that the stylus pen 50 has been removed when the difference is equal to or greater than a predetermined threshold. For example, when the difference between time t _n and time t _n+1 is equal to or greater than a threshold, the determination unit 154 determines that the stylus pen 50 has been removed at time t _n and determines the input coordinate at time t _n to be the input coordinate of the end point. The determination unit 154 determines the input coordinate at time t _n+1 to be the input coordinate of the starting point. The determination unit 154 may determine the starting point and the end point by other well-known techniques, and count the number of starting points and the number of end points.

In formula (1), T corresponds to the input time of the trajectory. For example, the determination unit 154 calculates the input time T of the trajectory by subtracting the first input time from the last input time registered in the trajectory information table 142.

If the evaluation value R is greater than a preset threshold value Rt, the determination unit 154 determines that the current drag operation is handwriting input. The determination unit 154 outputs the determination result to the screen control unit 156.

The process by which the determination unit 154 determines whether a tap input has occurred is described below. A tap input is an operation in which the user presses and releases the button with almost no dragging.

The determination unit 154 starts measuring the first time based on the trajectory information table 142 when the input point by the touch operation is the end point, and continues measuring the first time until the input of the start point by the next touch operation is received. The determination unit 154 determines whether or not a tap input has occurred before the first time reaches T1 seconds, and if a tap input has occurred before the first time reaches T1 seconds, determines to switch to handwriting input mode. If the determination unit 154 determines that the trajectory is a tap input, it outputs information to the effect that a tap input has been received to the screen control unit 156.

The process by which the determination unit 154 determines the end of handwriting input will be described. For example, after determining that the current drag operation is handwriting input, the determination unit 154 starts measuring the second time when the input point by the touch operation is the end point. The determination unit 154 continues measuring the second time from the touch operation corresponding to the end point until the input of the start point by the next touch operation is received. The determination unit 154 determines that the handwriting input has ended if the second time is equal to or greater than the predetermined time T2. If the determination unit 154 determines that the handwriting input has ended, it outputs information to the effect that the handwriting input has ended to the screen control unit 156.

Returning to the explanation of FIG. 2, the state reset unit 155 is a processing unit that refers to the trajectory information table 142, and erases the information in the trajectory information table 142 if no next touch operation is performed for a certain period of time after the last touch operation on the touch panel 110. For example, the state reset unit 155 erases the information in the trajectory information table 142 if no next touch operation is performed for one second. Here, the time for determining whether to erase is one second, but it may be adjusted as appropriate between 0.5 and 1 second.

The state reset unit 155 also erases the information in the trajectory information table 142 when the screen control unit 156 (described later) reads out the trajectory information stored in the trajectory information table 142 and displays the trajectory on the touch panel 110.

The screen control unit 156 is a processing unit that displays the screen information 141 on the touch panel 110, and scrolls the screen and displays the trajectory of handwritten input in response to a drag operation. The processing of the screen control unit 156 corresponds to the processing described in steps S10 to S14 of FIG. 1. An example of the processing of the screen display unit 156 is described below.

The screen control unit 156 outputs the screen information 141 to the touch panel 110, and causes the screen information 141 to be displayed on the touch panel 110. The screen control unit 156 receives information on input coordinates by a touch operation from the input receiving unit 151, and outputs a control command to the touch panel 110 to scroll the screen information 141 according to the trajectory of the input coordinates. This processing is a drag scroll processing by the screen control unit 156.

When the screen control unit 156 acquires a determination result from the determination unit 154 indicating that the input is handwritten, the screen control unit 156 executes a process of returning the screen of the touch panel 110 to the screen at the start of scrolling. The screen control unit 156 acquires the coordinates of the reference point by referring to the movement information 143, and returns the display position of the screen information to the original by the amount by which the reference point has moved. For example, when the coordinates of the reference point registered in the movement information 143 are (x _B1 , y _B1 ), the screen control unit 156 outputs to the touch panel 110 a control signal for moving the coordinates of each pixel of the screen information by (-x _B1 , -y _B1 ).

Next, the screen control unit 156 refers to the trajectory information table 142, and outputs and displays information on the trajectory connecting the input coordinates (information corresponding to handwritten characters) on the touch panel 110. The process of outputting and displaying the trajectory information stored in the trajectory information table 142 on the touch panel 110 is a handwritten input process performed by the screen control unit 156.

When the screen control unit 156 acquires information from the determination unit 154 that a tap input has been received, the screen control unit 156 executes a process of returning the screen of the touch panel 110 to the screen at the start of scrolling. The screen control unit 156 acquires the coordinates of the reference point by referring to the movement information 143, and returns the display position of the screen information to the original position by the amount of the movement of the reference point. The screen control unit 156 then causes the trajectory information stored in the trajectory information table 142 to be displayed on the touch panel 110. For example, by performing a tap input within a certain period of time after drawing an underline, the scrolling can be restored and the trajectory information stored in the trajectory information table 142 can be redisplayed as handwritten input. In this case, the screen control unit 156 does not recognize the tap input used for switching as handwritten input and does not display it on the touch panel 110.

The screen control unit 156 may also transition to handwriting input processing after a certain time has elapsed since accepting a tap input. If the screen control unit 156 accepts a touch operation within a certain time since accepting a tap input, the above-mentioned trajectory registration processing unit 152 registers trajectory information, including the tap input, in the trajectory information table 142. By executing such processing, it is possible to prevent the input of punctuation marks that appear in characters from being unintentionally determined as a tap input for switching to handwriting input.

If the screen control unit 156 does not receive a touch operation within a certain period of time before and after a tap input, the screen control unit 156 transitions to handwriting input processing. If the screen control unit 156 transitions to handwriting input processing, the screen control unit 156 does not execute a scroll operation. After transitioning to handwriting input mode, if the screen control unit 156 receives information on input coordinates by a touch operation from the input reception unit 151, the screen control unit 156 causes the touch panel 110 to display information on the path connecting the input coordinates.

When the screen control unit 156 receives information from the determination unit 154 that the handwriting input has ended, it transitions to the drag scroll processing described above.

Next, an example of the processing procedure of the input reception system according to this embodiment will be described. Figs. 5 to 10 are flowcharts showing the processing procedure of the input reception system. Fig. 5 will be described. The information processing device 100 receives input coordinates by a touch operation from the touch panel 110 (step S101). The screen control unit 156 of the information processing device 100 determines whether or not the mode is handwriting input mode (step S102).

If the current mode is the handwriting input mode (step S102, Yes), the screen control unit 156 executes the handwriting input process (step S103) and proceeds to the process of step S120 in FIG. 8.

If the current mode is not the handwriting input mode (step S102, No), the screen control unit 156 proceeds to step S104. The trajectory registration processing unit 152 of the information processing device 100 registers the input coordinates in the trajectory information table 142 (step S104).

The determination unit 154 of the information processing device 100 counts the number of bending points based on the trajectory information table 142 (step S105). The process of step S105 in FIG. 5 corresponds to the process procedure described in FIG. 4.

The determination unit 154 counts the number of starting points Nd based on the trajectory information table 142 (step S106). The determination unit 154 calculates the evaluation value R based on formula (1) (step S107) and proceeds to step S108 in FIG. 6.

Now, we move on to the explanation of FIG. 6. The determination unit 154 determines whether the evaluation value R is equal to or greater than the threshold value Rt (step S108). If the evaluation value R is equal to or greater than the threshold value Rt (step S108, Yes), the determination unit 154 determines that the drag operation is handwriting input (step S109).

The screen control unit 156 returns the display position to the position at the start of scrolling based on the movement information 143 (step S110). The screen control unit 156 changes the current mode to handwriting input mode (step S111). The screen control unit 156 reads out the trajectory information stored in the trajectory information table 142 and displays it on the touch panel 110 (step S112). The state reset unit 155 of the information processing device 100 erases the trajectory information stored in the trajectory information table 142 (step S113).

However, if the evaluation value R is not equal to or greater than the threshold value Rt (step S108, No), the determination unit 154 determines that the drag operation is not handwriting input (step S114). The screen control unit 156 executes drag scroll processing (step S115) and proceeds to step S116 in FIG. 7.

Now, we move on to the explanation of FIG. 7. The determination unit 154 determines whether the input point by the touch operation is the start point (step S116). If the input point by the touch operation is the start point (step S116, Yes), the determination unit 154 stops measuring the first time (step S117).

On the other hand, if the input point by the touch operation is not the start point (step S16, No), the determination unit 154 determines whether the input point by the touch operation is the end point (step S118). If the input point by the touch operation is the end point (step S118, Yes), the determination unit 154 starts measuring the first time (step S119).

On the other hand, if the input point by the touch operation is not the end point (step S118, No), the determination unit 154 ends the process.

Now, we move on to the explanation of FIG. 8. The determination unit 154 determines whether the input point by the touch operation is the start point (step S120). If the input point by the touch operation is the start point (step S120, Yes), the determination unit 154 stops measuring the second time (step S121).

On the other hand, if the input point by the touch operation is not the start point (step S120, No), the determination unit 154 determines whether the input point by the touch operation is the end point (step S122). If the input point by the touch operation is the end point (step S122, Yes), the determination unit 154 starts measuring the first time (step S123).

On the other hand, if the input point by the touch operation is not the end point (step S122, No), the determination unit 154 ends the process.

Now moving on to the explanation of FIG. 9. The determination unit 154 of the information processing device 100 determines whether or not a tap input has been received before the first time reaches T1 seconds (step S201). If the determination unit 154 determines that a tap input has not been received before the first time reaches T1 (step S201, No), the movement information registration unit 153 of the information processing device 100 initializes the position of the reference point (step S202).

The state reset unit 155 of the information processing device 100 erases the trajectory information from the trajectory information table 142 (step S203).

On the other hand, if the determination unit 154 receives a tap input before the first time reaches T1 seconds (step S201, Yes), it determines that a tap input has been received and switches to handwriting input mode (step S204).

Now, we move on to the explanation of FIG. 10. The determination unit 154 of the information processing device 100 determines whether the second time is equal to or greater than T2 (step S301). If the determination unit 154 determines that the second time is equal to or greater than T2 (step S301, Yes), the screen control unit 156 proceeds to drag scroll processing (step S302).

On the other hand, if the determination unit 154 determines that the second time is not equal to or greater than T2 (step S301, No), the screen control unit 156 continues processing the handwritten input (step S303).

Next, the effects of the input reception system according to this embodiment will be described. In the input reception system, the information processing device 100 scrolls the screen in response to a drag operation. When the information processing device 100 determines that the input is handwritten input based on the end point, start point, or bend point of the drag operation trajectory, it returns the scroll position to the start position and displays the characters of the input trajectory. This allows the user to selectively use both handwritten input and scrolling without having to give any special instructions.

The information processing device 100 erases the information in the trajectory information table 142 if no next touch operation is performed for a certain period of time after the last touch operation on the touch panel 110. This makes it possible to erase trajectories that are not handwritten inputs associated with the scroll operation when the user's drag operation is a scroll.

The information processing device 100 counts the number of end points, start points, or bend points that occur in the trajectory, and if the value obtained by dividing the counted number by the time of the drag operation is equal to or greater than a threshold value, returns the display to the screen at the time when the scrolling started, and displays the trajectory using the trajectory information stored in the storage device. This makes it possible to accurately determine handwritten input and transition from scrolling processing to handwritten input processing.

However, the above-described embodiment is merely an example, and the information processing device 100 may execute other processes. The following describes other processes executed by the information processing device 100.

An example of the counting process executed by the determination unit 154 of the information processing device 100 has been described using FIG. 4, but the counting process may be executed, for example, by the processing procedure shown in FIG. 11. FIG. 11 is a flowchart showing the processing procedure of other counting processes. As shown in FIG. 11, the determination unit 154 of the information processing device 100 refers to the trajectory information table 142 and calculates the distance from the straight line L connecting the first point and the last point to each point among the points (input coordinates) that constitute the trajectory (step S60).

The determination unit 154 extracts the point P that is located farthest from the line L (step S61). If the distance between the line L and the point P exceeds the threshold D, the determination unit 154 divides the coordinate sequence into two before and after the point P (step S62). The determination unit 154 determines whether the line L can be further divided (step S63).

If the line L can be further divided (step S63, Yes), the determination unit 154 calculates the distance from the line L to each point for each divided line L (step S64) and proceeds to step S61.

If the straight line L cannot be further divided (step S63, No), the determination unit 154 calculates the angle between the first point and the last point in each divided section (step S65). The determination unit 154 calculates the angle difference between the previous and next divided sections, determines the parts where the angle difference is equal to or greater than a predetermined angle as bending points, and counts the number of bending points (step S66).

The determination unit 154 may count the number of bending points using the processing procedure shown in FIG. 11.

The screen control unit 156 of the information processing device 100 may display the trajectory input by the drag operation on the touch panel 110 at a lighter density than normal for a certain period of time, even while the drag scroll process is being performed. The normal density corresponds to the density of the trajectory when the screen control unit 156 processes handwritten input. If the user performs a drag operation intending to perform handwritten input, there is a time lag until it is determined to be handwritten input. Drag scrolling occurs for a while, but the screen control unit 156 displays the trajectory on the touch panel 110 for a certain period of time, allowing the trajectory of the handwritten input to be confirmed. The screen control unit 156 may control the display of the trajectory so that the trajectory displayed on the touch panel 110 gradually becomes lighter over time, and ultimately no trajectory is displayed.

When switching from drag scrolling to handwriting input, the screen control unit 156 executes a process to return the screen to the screen at the start of scrolling, but before returning the screen, the screen to be returned to may be displayed as a faint superimposed display on the screen before being returned. If the user is performing a drag operation intending to perform handwriting input, there is a time lag until it is determined to be handwriting input. Drag scrolling occurs for a while, but if the user is inputting annotations to the content displayed on the screen, the positional relationship between the screen content and the handwriting becomes important. By displaying a superimposed display of the screen to which the screen will be returned if it is determined to be handwriting input, the user can correctly grasp the positional relationship between the screen content and the handwriting even before the screen returns to the original state.

In this embodiment, the touch panel 110 has been used as an example of a coordinate detection device and a display device, but the present invention is not limited to this. The information processing device 100 may be connected to a coordinate detection device that detects the coordinates of a touch operation, and a display device that displays an image from the control unit 150. The display device may be an electronic whiteboard, etc.

In addition to lightly superimposing the trajectory, the screen control unit 156 may also lightly superimpose the screen at the start of scrolling. With this configuration, for example, when trying to draw a circle in a document displayed below, the screen before scrolling is also lightly displayed, making it easy to see where the circle is being drawn while writing.

Next, an example of the hardware configuration of a computer that realizes the same functions as the information processing device 100 shown in the above embodiment will be described. FIG. 12 is a diagram showing an example of the hardware configuration of a computer that realizes the same functions as the information processing device of the embodiment.

As shown in FIG. 12, computer 200 has a CPU 201 that executes various types of arithmetic processing, and a touch panel 202 that receives touch operations from a user and displays images. Computer 200 also has a communication device 203 that transmits and receives data to and from external devices, etc., via a wired or wireless network, and an interface device 204. Computer 200 also has a RAM 205 that temporarily stores various types of information, and a hard disk device 206. Each of devices 201 to 206 is connected to a bus 207.

The hard disk device 206 has an input reception program 206a, a trajectory registration processing program 206b, a movement information registration program 206c, a judgment program 206d, a state reset program 206e, and a screen control program 206f. The CPU 201 also reads out each of the programs 206a to 206f and expands them in the RAM 205.

The input reception program 206a functions as the input reception process 205a. The trajectory registration processing program 206b functions as the trajectory registration processing process 205b. The movement information registration program 206c functions as the movement information registration process 205c. The judgment program 206d functions as the judgment process 205d. The state reset program 206e functions as the state reset process 205e. The image control program 206f functions as the image control process 205f.

The processing of the input reception process 205a corresponds to the processing of the input reception unit 151. The processing of the trajectory registration processing process 205b corresponds to the processing of the trajectory registration processing unit 152. The processing of the movement information registration process 205 corresponds to the processing of the movement information registration unit 153. The processing of the judgment process 205d corresponds to the processing of the judgment unit 154. The processing of the state reset process 205e corresponds to the processing of the state reset unit 155. The processing of the screen control process 205f corresponds to the processing of the screen control unit 156f.

Note that each of the programs 206a to 206f does not necessarily have to be stored in the hard disk device 307 from the beginning. For example, each program may be stored in a "portable physical medium" such as a flexible disk (FD), CD-ROM, DVD, magneto-optical disk, or IC card that is inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 206a to 206f.

The following notes are further provided with respect to the embodiments including the above examples.

(Appendix 1) To the computer,
when a drag operation is received on a coordinate detection device which detects the coordinates at which a touch operation is received, scrolling a screen displayed on a display device in accordance with a path of the drag operation;
storing the information of the trajectory received during the scrolling in a storage device;
when it is determined that a predetermined number or more of end points, start points, or bend points have occurred on the trajectory, the display is returned to the screen at the time when the scrolling was started;
An input receiving program that causes a process of displaying the trajectory to be executed by using the trajectory information stored in the storage device.

(Appendix 2) The input acceptance program described in Appendix 1 is characterized in that the process of displaying the trajectory starts displaying the trajectory from the time the drag operation is accepted.

(Appendix 3) The input acceptance program described in appendix 1 or 2 further executes a process to erase the trajectory information stored in the storage device if the next drag operation is not resumed even after a predetermined time has elapsed since the acceptance of the drag operation was interrupted.

(Appendix 4) An input reception program as described in appendix 1, 2 or 3, characterized in that it counts the number of end points, start points or bend points that occur in the trajectory, and if the counted number divided by the time of the drag operation is equal to or greater than a threshold value, returns the display to the screen at the time when the scrolling started, and displays the trajectory using the information of the trajectory stored in the storage device.

(Appendix 5) The input reception program described in any one of appendices 1 to 4, further comprising a process for lightly superimposing the trajectory received during the scrolling on the screen.

(Appendix 6) The input acceptance program described in Appendix 5 further executes a process of superimposing a light image of the screen at the time the scrolling begins on the screen.

(Supplementary Note 7) An input reception system having a coordinate detection device that detects coordinates at which a touch operation is received, a display device that displays screen information output from an information processing device, and the information processing device,
The information processing device includes:
a screen control unit that, when a drag operation on the coordinate detection device is received, scrolls a screen displayed on a display device in accordance with a path of the drag operation;
a path registration processing unit that stores information on the path received during the scrolling in a storage unit;
The input reception system is characterized in that, when the screen control unit determines that a predetermined number or more of end points, start points, or bend points have occurred in the trajectory, it returns the display to the screen at the time the scrolling began, and displays the trajectory using the information of the trajectory stored in the memory unit.

(Appendix 8) The input reception system described in Appendix 7 is characterized in that the screen control unit starts displaying the trajectory from the point in time when the drag operation is received.

(Appendix 9) The input reception system described in appendix 7 or 8 further comprises a state reset unit 155 that erases the trajectory information stored in the storage device if the next drag operation is not resumed even after a predetermined time has elapsed since the point at which the acceptance of the drag operation was interrupted.

(Appendix 10) The input reception system described in appendix 7, 8 or 9 is characterized in that the screen control unit counts the number of end points, start points or bend points that occur in the trajectory, and if the value obtained by dividing the counted number by the time of the drag operation is equal to or greater than a threshold value, returns the display to the screen at the time when the scrolling started, and displays the trajectory using the information of the trajectory stored in the storage device.

(Appendix 11) The input reception system described in any one of appendices 7 to 10, characterized in that the screen control unit further executes a process of superimposing the trajectory received during the scrolling on the screen in a light manner.

(Appendix 12) The input reception system described in Appendix 11, characterized in that the screen control unit further executes a process of superimposing a light image of the screen at the time the scrolling begins on the screen.

REFERENCE SIGNS LIST 100 Information processing device 110 Touch panel 120 Communication unit 130 Timer 140 Storage unit 141 Screen information 142 Trajectory information table 143 Movement information 150 Control unit 151 Input reception unit 152 Trajectory registration processing unit 153 Movement information registration unit 154 Determination unit 155 Status reset unit 156 Screen control unit

Claims (7)

On the computer,
when a drag operation is received on a coordinate detection device which detects the coordinates at which a touch operation is received, scrolling a screen displayed on a display device in accordance with a path of the drag operation;
storing the information of the trajectory received during the scrolling in a storage device;
Counting at least one of the number of start points and the number of end points of the locus and the number of bending points of the locus based on the information of the locus;
determining whether the input is handwritten or not based on a value calculated based on at least one of the counted number of the start points or the counted number of the end points and the number of the bend points;
If it is determined that the input is handwritten input, the display is returned to the screen at the time when the scrolling started,
An input receiving program that causes a process of displaying the trajectory to be executed by using the trajectory information stored in the storage device. The input acceptance program according to claim 1, characterized in that the process of displaying the trajectory starts displaying the trajectory from the time the drag operation is accepted. The input acceptance program according to claim 1 or 2, further comprising a process for erasing the information on the trajectory stored in the storage device if the next drag operation is not resumed even after a predetermined time has elapsed since the acceptance of the drag operation was interrupted. An input reception program according to claim 1, 2 or 3, characterized in that it counts the number of end points, start points or bend points that occur in the trajectory, and if the value obtained by dividing the counted number by the time of the drag operation is equal to or greater than a threshold value, it returns the display to the screen at the time when the scrolling started, and displays the trajectory using the information of the trajectory stored in the storage device. The input reception program according to any one of claims 1 to 4, further comprising a process for displaying the trajectory received during the scrolling in a lightly superimposed manner on the screen. The input reception program according to claim 5, further comprising a process for displaying a light version of the screen at the time the scrolling begins superimposed on the screen. An input reception system having a coordinate detection device that detects coordinates at which a touch operation is received, a display device that displays screen information output from an information processing device, and the information processing device,
The information processing device includes:
a screen control unit that, when a drag operation on the coordinate detection device is received, scrolls a screen displayed on a display device in accordance with a path of the drag operation;
a path registration processing unit that stores information on the path received during the scrolling in a storage unit;
the screen control unit counts the number of at least one of the starting points or the ending points of the trajectory and the number of bending points of the trajectory based on the information about the trajectory, determines whether or not the input is handwritten based on a value calculated based on the counted number of at least one of the starting points or the ending points and the number of bending points, and if it determines that the input is handwritten, returns the display to the screen at the time the scrolling began, and displays the trajectory using the information about the trajectory stored in the memory unit.

Images