JP2016126476A - Handwriting system and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a handwriting system in which coordinate detection accuracy of an indication tool has been improved, even in the case where shielding occurs between the indication tool and a light receiving sensor.SOLUTION: When a shielding determination unit 156 determines that shielding has occurred between an indication tool and a light receiving sensor, a coordinate prediction unit 157 determines whether or not one stroke has terminated in a shielded state. Then, when it determines that the one stroke has terminated in a shielded state, the coordinate prediction unit 157 predicts a coordinate of the indication tool on the basis of a time change amount of writing pressure calculated by the writing pressure measured by a writing pressure measurement unit, a time change amount of acceleration calculated by the acceleration measured by an acceleration measurement unit and a calculation result of the coordinate by a coordinate calculation unit 154. A coordinate correction unit 158 corrects the coordinate calculated by the coordinate calculation unit 154 on the basis of the prediction result by the coordinate prediction unit 157. A display control unit 161 displays characters and the like on a display on the basis of the coordinate corrected by the coordinate correction unit 158.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a handwriting system such as an electronic blackboard provided with optical coordinate detection means and a program used in the system.

In recent years, electronic blackboards in which whiteboards used in conference rooms and the like are digitized have become widespread.
In this electronic blackboard, the light emitted (radiated) from the pointing device is detected by a light receiving sensor installed around the electronic blackboard (for example, two corners of the electronic blackboard), and the coordinates of the pointing tool are detected by triangulation. The technology to do is known.

  In an electronic blackboard equipped with such a technology for detecting the coordinates of the pointing tool, when the shielding occurs between the pointing tool and the light receiving sensor (for example, when the electronic blackboard is touched), It is also known that there is a problem that the accuracy of detecting coordinates is lowered.

As a countermeasure for solving this problem, for example, Patent Document 1 discloses a case where a plurality of pieces of image information are acquired in a target region, a prediction process based on the image information is executed, and the pointing tool is shielded. In addition, an input system for tracking the coordinate position of the pointing tool is described.
However, in the input system described in Patent Document 1, since the prediction process is executed based only on the image information, there is a problem that the accuracy of detecting the coordinates of the pointing tool is lowered.

  The present invention has been made in view of the above-described conventional problems, and an object of the present invention is to create a shield between an indicator and a light receiving sensor in a handwritten system such as an electronic blackboard provided with an optical coordinate detection means. Even in this case, it is to improve the coordinate detection accuracy of the pointing tool.

  The present invention relates to an electronic blackboard, an indicator that is a light-emitting input device for inputting characters or figures on the electronic blackboard, two or more light receiving sensors that receive light emitted from the indicator, and the light receiving device. Coordinate calculation means for calculating the coordinates of the pointing tool based on light reception information detected by the sensor, shielding determination means for determining whether or not shielding has occurred between the pointing tool and the light receiving sensor, and the coordinate calculation means And a display control means for controlling the character or figure input by the pointing tool to be displayed on the electronic blackboard based on the coordinates calculated in step 1, wherein the writing pressure of the pointing tool is measured. Measured by writing pressure measuring means, acceleration measuring means for measuring the acceleration of the pointing tool, writing pressure change amount calculating means for calculating a temporal change amount of writing pressure measured by the writing pressure measuring means, and the acceleration measuring means An acceleration change amount calculating means for calculating a time change amount of the acceleration, and a time change amount of the writing pressure calculated by the writing pressure change amount calculating means when it is determined that the shielding is occurring by the shielding determining means, and Based on the time change amount of acceleration calculated by the acceleration change amount calculation means, the coordinate prediction means for predicting the coordinate position of the pointing tool, and the coordinate position of the pointing tool is corrected based on the coordinates predicted by the coordinate prediction means. And a coordinate correction means.

  According to the present invention, in a handwriting system such as an electronic blackboard provided with an optical coordinate detection means, even when a shielding occurs between the pointing tool and the light receiving sensor, the coordinate detection accuracy of the pointing tool is improved. be able to.

It is a schematic block diagram of the handwriting system which concerns on embodiment of this invention. It is a hardware block diagram of the computer of the handwriting system which concerns on embodiment of this invention. It is a functional block diagram of the pointing tool of the handwriting system which concerns on embodiment of this invention. It is a functional block diagram of the computer of the handwriting system which concerns on embodiment of this invention. It is a flowchart regarding the display process of the handwriting system which concerns on embodiment of this invention. It is a flowchart regarding the determination process of the frame of the handwriting system which concerns on embodiment of this invention. It is a flowchart regarding the shielding determination process of the handwriting system which concerns on embodiment of this invention. It is a flowchart regarding the coordinate prediction process and coordinate correction process of the handwriting system which concerns on embodiment of this invention. It is a figure explaining predicting and correcting the coordinates of the pointing tool from the images before and after the shielding. It is a figure explaining predicting and correcting the coordinates of the pointing tool from the image before shielding. It is a figure which shows the time variation | change_quantity of the pen pressure calculated by the pen pressure change amount calculation part. It is a figure which shows the time change amount of the acceleration calculated by the acceleration change amount calculation part. It is a figure explaining the determination of the last writing and correction | amendment of a coordinate based on the time change amount of writing pressure and acceleration. It is a figure explaining the prediction method of the characteristic of "Splash" and the direction of "Splash".

Hereinafter, the handwriting system 1 which is the embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic block diagram of a handwriting system 1 according to an embodiment of the present invention.
As shown in the figure, the handwriting system 1 in FIG. 1 mainly includes a computer 10, a display 20, an indicator 30, and a light receiving sensor 40 (40a to 40d) installed at each corner of the display 20.
Further, a PC (Personal Computer) 60 is connected to the computer 10 as an additional element.
In addition, 50 (50a-50d) of FIG. 1 is a peripheral light emission part.

An application corresponding to coordinate detection or the like is installed in the computer 10, and when the user draws with the pointing tool 30, the position coordinates of the pointing tool 30 are calculated based on a signal from the light receiving sensor 40.
For example, when the user selects “draw line” from the menu screen displayed on the display 20 and draws a figure on the display 20 with the pointing tool 30 such as an electronic pen having a light emitting unit, the computer 10 displays the pointing tool 30. Based on the light radiated from the light emitting unit (that is, the signal from the light receiving sensor 40), the position where the indicator 30 is in contact with the display 20 is analyzed in real time, and the coordinate position is determined in time series from the analysis result. calculate.
Thereafter, the computer 10 controls the coordinate positions generated in time series to be connected by line segments and displayed on the display 20.

FIG. 1 shows an example in which the user draws the pointing tool 30 in the shape of a triangle on the display 20, and in this case, the computer 10 generates the coordinate positions constituting the triangle in time series and stores them as one stroke. To do.
Here, the stroke is a set of coordinates of the pointing tool 30 corresponding to a continuous frame from when the coordinates of the pointing tool 30 are detected until the detection ends.

Thereafter, the computer 10 controls to display the drawn triangle on the display 20 based on the calculated position coordinates.
In FIG. 1, since video data (that is, video data targeting the sun and mountains) is output from the PC 60, the computer 10 superimposes and displays the video data on the display 20. So that it is controlled.
The display 20 is a display device for displaying characters, figures, and the like input by handwriting by the user and video data output from the PC 60.

The indicator 30 is a light emitting input device.
Although details will be described later with reference to FIG. 3, the pointing tool 30 of the handwriting system 1 includes a writing pressure measurement unit, an acceleration measurement unit, and a measurement information transmission unit, and information on the measured writing pressure and acceleration is measured by the measurement information transmission unit. To the computer 10.
When the shielding occurs between the pointing tool 30 and the light receiving sensor 40, the computer 10 predicts and corrects the coordinates of characters, figures, and the like based on the measurement information transmitted from the measurement information transmitting unit of the pointing tool 30. .
For example, as shown in FIG. 1, the light receiving sensor 40 is disposed at four corners surrounding the rectangular display 20 and detects light emitted (radiated) from the pointing tool 30.

FIG. 2 is a hardware configuration diagram of the computer 10 of the handwriting system according to the embodiment of the present invention.
The computer 10 is an information processing apparatus, and as shown in FIG. 2, a CPU (Central Processing Unit) 101 and a ROM (Read Only Memory) are electrically connected via bus lines such as an address bus and a data bus. 102, RAM (Random Access Memory) 103, SSD (Solid State Drive) 104, network controller 105, external storage controller 106, capture device 107, GPU (Graphics Processing Unit) 108, display controller 109, sensor controller 110, indicator controller 111 and an application 112.

The CPU 101 is a control unit that controls the entire computer 10 and executes, for example, the application 112 and the like.
The ROM 102 stores a program (IPL (Initial Program Loader) or the like) executed by the CPU 101 at the time of startup.
The RAM 103 is used as a work area when the CPU 101 executes the application 112.

The SSD 104 is a nonvolatile memory that stores the application 112 and various data.
The network controller 105 executes processing based on a communication protocol when communicating with a server or the like via a network.
Here, the network is, for example, a local area network (LAN) or a wide area network (WAN) to which a plurality of LANs are connected.

The external storage controller 106 writes data to the removable external memory 113 and reads data from the external memory 113.
For example, a USB (Universal Serial Bus) memory, an SD card, or the like can be used as the removable external memory 113.

The capture device 107 captures (captures) video displayed on the display unit by the PC 60 connected to the computer 10.
The GPU 108 is a graphic processor that calculates the pixel value of each pixel of the display 20.
The display controller 109 outputs the image generated by the GPU 108 to the display 20.
The sensor controller 110 is connected to the light receiving sensor 40 and executes coordinate calculation by a triangulation method based on light emitted from the light emitting unit of the pointing tool 30.

The pointing tool controller 111 is a communication unit that receives measurement information from the pointing tool 30, and is connected to the pointing tool 30 according to a communication standard such as Bluetooth (registered trademark).
The CPU 101 can be executed by reading the application 112 from the external memory 113 or by downloading it from a predetermined server by the network controller 105. The application 112 may be stored in a compressed state in an external memory or the like.

FIG. 3 is a functional block diagram of the pointing tool 30 of the handwriting system according to the embodiment of the present invention.
As shown in FIG. 3, the pointing tool 30 includes a writing pressure measurement unit 301, an acceleration measurement unit 302, and a measurement information transmission unit 303 that transmits information about the measured writing pressure and acceleration to the computer 10.

The pen pressure measurement unit 301 is mainly composed of a pen pressure sensor or the like, and measures the pen pressure at the tip of the pointing tool 30.
The acceleration measuring unit 302 is mainly composed of an acceleration sensor or the like, and measures the acceleration at the distal end portion of the pointing tool 30.
As described above, the measurement information transmission unit 303 transmits information on the pen pressure measured by the pen pressure measurement unit 301 and the acceleration measured by the acceleration measurement unit 302 (that is, measurement information), for example, Bluetooth (registered trademark). The measurement information is received by the measurement information receiving unit 151 (FIG. 4) of the computer 10 in accordance with a communication standard such as the above.

FIG. 4 is a functional block diagram of the computer 10 of the handwriting system according to the embodiment of the present invention.
As shown in FIG. 4, the computer 10 includes a measurement information receiving unit 151, a writing pressure change amount calculating unit 152, an acceleration change amount calculating unit 153, a coordinate calculating unit 154, a frame determining unit 155, a shielding determining unit 156, and a coordinate predicting unit. 157, a coordinate correction unit 158, a shielding distance calculation unit 159, a shielding time measurement unit 160, and a display control unit 161.

The measurement information receiving unit 151 includes a writing pressure information receiving unit that receives information related to writing pressure and an acceleration information receiving unit that receives information related to acceleration, which are transmitted from the measurement information transmitting unit 303 (FIG. 3) of the pointing tool 30. Is done.
The writing pressure change amount calculation unit 152 calculates the amount of change in writing pressure over time based on the writing pressure information received by the measurement information receiving unit 151 (writing pressure information receiving unit).
Further, the acceleration change amount calculation unit 153 calculates the time change amount of acceleration based on the acceleration information received by the measurement information reception unit 151 (acceleration information reception unit).

The coordinate calculation unit 154 calculates the coordinates of the pointing tool 30 based on the image acquired by the light receiving sensor 40.
The frame determination unit 155 determines a frame that is a target of the shielding determination process.
The shielding determination unit 156 determines whether or not shielding has occurred between the pointing tool 30 and the light receiving sensor 40.

The coordinate predicting unit 157 includes the time variation of the pen pressure calculated by the pen pressure change calculating unit 152, the time variation of the acceleration calculated by the acceleration change calculating unit 153, and the coordinate calculation result by the coordinate calculating unit 154. Based on the above, the coordinates of the pointing tool 30 are predicted.
In addition, the coordinate predicting unit 157 also functions as a drawing direction specifying unit that specifies the drawing direction from the time change of the coordinates calculated by the coordinate calculating unit 154.

The coordinate correction unit 158 corrects the coordinates of the pointing tool 30 based on the prediction result by the coordinate prediction unit 157.
The shielding distance calculation unit 159 calculates the inter-coordinate distance (length) of the portion determined to be shielded by the shielding determination unit 156.
The shielding time measuring unit 160 measures the time from the occurrence of shielding until the end of one stroke.
The display control unit 161 generates data related to characters, graphics, and the like based on the coordinates corrected by the coordinate correction unit 158 and performs control to display the data on the display 20.

  Of the above-described units, the pen pressure measurement unit 301, the acceleration measurement unit 302, the pen pressure change amount calculation unit 152, the acceleration change amount calculation unit 153, the coordinate calculation unit 154, the shielding determination unit 156, the coordinate prediction unit 157, the coordinate The correction unit 158 and the display control unit 161 are function realizing means generated by causing the computer 10 to read a program.

FIG. 5 is a flowchart regarding display processing of the handwriting system 1 according to the embodiment of the present invention.
When a user draws a character, a figure or the like on the display 20, the light receiving sensor 40a is an image _{O L,} also the light receiving sensor 40b acquires the image _{O R} (S101).
The image acquired by the light receiving sensor 40 may be one-dimensional or two-dimensional.

Coordinate calculation unit 154, the feature amount _{F L} of the pointing device 30 from the image _{O L} obtained by the light receiving sensor 40, also calculates a characteristic amount _{F R} of the pointing tool 30 from the image _{O R} (S102).
Specifically, the coordinate calculation unit 154 cuts out the light emitting area from the image by the labeling process, and uses the position of the center of gravity of the light emitting area (that is, the position of the center of gravity of the high-luminance location) and the number of the centers of gravity as the feature amount. calculate.
Here, the position of the center of gravity of the light emitting area is determined in the angle calculation process in the coordinate calculation unit 154 as described later in step S103, and the number of the center of gravity of the light emitting area is determined in the shielding determination unit 156 as described later in step S106. Used in determination processing.
In the handwriting system according to the present embodiment, the light emitting type indicator 30 is used. However, when the light blocking type indicator is used, a blocking area is cut out from the image by the labeling process, and the blocking area is displayed. The position of the center of gravity (that is, the position of the center of gravity of the low-luminance portion) and the number of the centers of gravity are calculated as the feature amount.

Coordinate calculation unit 154, the angle theta _L from the feature _{F L} (angle line connecting the light receiving sensor 40a and the pointing device 30 makes with the horizontal direction of the display 20), also the angle theta _R (received from the feature _{F R} An angle formed by a line segment connecting the sensor 40b and the pointing tool 30 with respect to the horizontal direction of the display 20 is calculated (S103).
Specifically, the coordinate calculation unit 154 creates a reference table that indicates the position and angle of the center of gravity within the entire angle of view of the image in advance, and refers to the reference table, whereby the angle θ _L and the angle θ _{R are obtained.} Is calculated.

Coordinate calculation unit 154 calculates the coordinates of the pointing tool 30 from the calculated angle theta _L and the angle θ _R (S104).
A known triangulation may be used for calculating the coordinates.
Frame determination unit 155 determines whether the image _{O L} and image _{O R} obtained by the light receiving sensor 40 is an image (frame) to be determined shield (S105).
The frame determination process (S105) in the frame determination unit 155 will be described later with reference to FIG.

The shielding determination unit 156 determines the presence / absence of shielding from the number of centroids of the light emitting areas calculated by the coordinate calculation unit 154 (S106).
When the shielding determination unit 156 first determines that there is shielding, the shielding determination unit 156 stores the coordinates calculated so far in the RAM 103.
The shielding determination process (S106) in the shielding determination unit 156 will be described later with reference to FIG.

Further, the measurement information receiving unit 151 receives the pen pressure measured by the pen pressure measuring unit 301 at the same time as the time when the image is acquired by the light receiving sensor 40 as the pen pressure information (S107).
The writing pressure change amount calculation unit 152 calculates the time change amount of the writing pressure based on the writing pressure information received (acquired) by the measurement information receiving unit 151 (S108).

The measurement information receiving unit 151 further receives the acceleration measured by the acceleration measuring unit 302 at the same time as the time when the image is acquired by the light receiving sensor 40 as the acceleration information (S109).
The acceleration change amount calculation unit 153 calculates the time change amount of acceleration based on the acceleration information received (acquired) by the measurement information reception unit 151 (S110).
In addition, the process (S107, S108) from acquiring the pen pressure information to calculating the temporal change amount of the pen pressure and the process (S109, S108) from acquiring the acceleration information to calculating the temporal change amount of the acceleration. With regard to S110), the order of execution may be determined at any time, or may be executed simultaneously.

  The coordinate predicting unit 157, when the shielding determining unit 156 determines that there is shielding, the coordinates of the pointing tool 30 stored in the RAM 103, and the time variation and acceleration variation of the writing pressure calculated by the writing pressure variation calculating unit 152 The coordinates of the pointing tool 30 are predicted from the time variation of acceleration calculated by the calculation unit 153 (S111).

The coordinate correction unit 158 corrects the coordinates calculated by the coordinate calculation unit 154 based on the prediction result by the coordinate prediction unit 157 (S112).
The display control unit 161 displays characters, graphics, and the like on the display 20 based on the coordinates corrected by the coordinate correction unit 158 (S113).
In addition, the process of step S111 in the coordinate estimation part 157 and the process of step S112 in the coordinate correction | amendment part 158 are later mentioned using FIGS.

FIG. 6 is a flowchart regarding the frame determination process of the handwriting system 1 according to the embodiment of the present invention.
The frame determination unit 155 determines whether or not the frame number N of each frame output from the light receiving sensor 40 is greater than 1 (S201).

Here, the frame number N is a variable indicating what frame the image is after the light receiving sensor 40 is activated.
If the frame number N is greater than 1, the frame determination unit 155 shifts to the occlusion determination process (FIG. 7), and if not (ie, the frame number N = 1), the frame determination process ends.

FIG. 7 is a flowchart regarding the shielding determination process of the handwriting system 1 according to the embodiment of the present invention.
Shielding determination unit 156 compares the number of the center of gravity of the image _{O LN} frame number N obtained by the light receiving sensor 40a, a number of the center of gravity of the image _{O RN} frame number N obtained by the light receiving sensor 40b (S301).
Here, when the number of centroids of the image _OLN and the number of centroids of the image _ORN are not equal, the shielding determination unit 156 shields between the light receiving sensor 40 and the pointing tool 30 of the image having a small number of centroids. It is determined that Conversely, when the number of centroids of the image _OLN is equal to the number of centroids of the image _ORN , the shielding determination unit determines that there is no shielding between the light receiving sensor 40 and the indicator 30.

As a result of comparing the number of centroids, the shielding determination unit 156 determines that the number of centroids is not equal (that is, determines that there is shielding) (S301 Yes (with shielding)), whether the number of shielding is 0 or not. Is determined (S302).
Further, as a result of comparing the number of centroids, the shielding determination unit 156 determines that the number of centroids is equal (that is, determines that there is no shielding) (S301 No (no shielding)), the coordinate prediction and correction processing ( The processing is shifted to FIG.
Here, even if there is no occlusion, the reason for shifting to the coordinate prediction and correction process is that occlusion may have occurred at the time of acquisition of the image until immediately before the image determined to be occluded. This is because it is necessary to predict and correct the coordinates of the pointing tool 30 from the images before and after the shielding.

If the occlusion determination unit 156 determines that the occlusion frequency is 0 in step S302 (S302 Yes), the occlusion determination unit 156 stores coordinates corresponding to one stroke up to the frame number N-1 (S303).
The shielding determination unit 156 increments the shielding count after storing the coordinates (S304).
If the shielding determination unit 156 determines that the number of times of shielding is not 0 in step S302 (No in S302), the shielding number is incremented (S305), and the process proceeds to coordinate prediction and correction processing (FIG. 8). .

FIG. 8 is a flowchart showing a procedure of processing related to prediction of coordinates and correction of coordinates in the handwriting system 1 according to the embodiment of the present invention.
The coordinate predicting unit 157 determines whether or not the number of occlusions is “0” (S401). If the number of occlusions is not “0” (S401 Yes), the coordinate prediction unit 157 starts processing related to coordinate prediction and correction.
The coordinate predicting unit 157 shifts the process to the display process of the display control unit 161 without executing the process of predicting the coordinates of the pointing tool 30 when the number of times of occlusion is “0”.

The coordinate predicting unit 157 determines whether or not one stroke is completed from the writing pressure information received by the measurement information receiving unit 151 (S402).
The coordinate predicting unit 157 monitors the writing pressure information until one stroke is completed (S402 Yes).
Note that the coordinate predicting unit 157 determines that one stroke is completed when the writing pressure information changes from a predetermined value to “0”.

Next, the coordinate predicting unit 157 determines whether or not to end the process with one stroke blocked (S403).
If the coordinate predicting unit 157 determines that one stroke has been completed without being shielded (ie, if it is determined that shielding has occurred in the middle of one stroke) (No in S403), the pointing tool is determined from the images before and after the shielding. 30 coordinate positions are predicted (S404).

The coordinate correction unit 158 corrects the coordinates of the pointing tool 30 based on the predicted coordinate position (S405).
Based on the corrected coordinates of the pointing tool 30, the display control unit 161 marks and displays a part of characters, figures, etc. at the time of occurrence of shielding (S406).

The display control unit 161 determines whether or not the inter-coordinate distance (inter-coordinate distance before and after the shielding) of the shielding part calculated by the shielding distance calculation unit 159 is greater than a predetermined threshold (S407).
When the display control unit 161 determines that the inter-coordinate distance of the shielding portion is larger than the predetermined threshold (S407 Yes), the display control unit 161 displays a confirmation screen to the user regarding whether or not there is a problem with the marking content (S408). .

When the user selects “OK” on the displayed confirmation screen (S409 “OK”), the display control unit 161 continuously displays the marking content, and the shielding determination unit 156 resets the number of times of shielding to “0” ( S411).
When the user selects “NG” on the confirmation screen displayed (S409 “NG”), the display control unit 161 deletes the marking content and instructs the user to write again (S410).
When the display control unit 161 deletes the marking content (S410), the shielding determination unit 156 resets the shielding number to “0” (S411).

  In step S403, if the coordinate predicting unit 157 determines that the stroke has been completed in a blocked state (Yes in S403), the coordinate position of the pointing tool 30 is determined from the amount of change related to the writing pressure and acceleration of the pointing tool 30. Prediction is performed (S412).

The coordinate correction unit 158 corrects the coordinates of the pointing tool 30 based on the predicted coordinate position (S413).
Based on the corrected coordinates of the pointing tool 30, the display control unit 161 marks and displays a part of characters, figures, etc. at the time of occurrence of shielding (S414).

The display control unit 161 determines whether or not the shielding time measured by the shielding time measuring unit 160 (that is, the time from the occurrence of shielding to the end of the stroke) is longer than a predetermined threshold (S415). ).
If the display control unit 161 determines that the shielding time is longer than the predetermined threshold (S415 Yes), the display control unit 161 displays a confirmation screen to the user regarding whether or not there is a problem with the marking content (S416).

When the user selects “OK” on the displayed confirmation screen (S417 “OK”), the display control unit 161 continues to display the marking content, and the shielding determination unit 156 resets the number of shieldings to “0” ( S419).
When the user selects “NG” on the confirmation screen displayed (S417 “NG”), the display control unit 161 erases the marking content and instructs the user to write again (S418).
When the display control unit 161 deletes the marking content (S418), the shielding determination unit 156 resets the shielding number to “0” (S419).

FIG. 9 illustrates a case where the coordinates of the pointing tool 30 are predicted and corrected from images before and after the shielding.
That is, the case where the coordinates of the pointing tool 30 are predicted and corrected when shielding occurs in the middle of one stroke corresponds to the processing from step S404 to step S411 in FIG.

FIG. 9 illustrates a case where the Chinese numeral “three” is written and the image acquired by the light receiving sensor 40 is shielded.
In FIG. 9, the position where the shielding is generated is indicated by a dotted line portion, and the coordinate position of the pointing tool 30 that is temporarily stored when the shielding occurs is surrounded by a rectangle.

In FIG. 9, since shielding occurs in the middle of the third stroke and the shielding disappears thereafter, the handwriting system 1 predicts and corrects the coordinates of the pointing tool 30 from the images before and after the shielding.
Specifically, the coordinates of the pointing tool 30 at the time of the occurrence of the shielding, the coordinates of the pointing tool 30 stored when the shielding is started, and the coordinates of the pointing tool 30 calculated when the shielding is lost (that is, before and after the shielding). The coordinates of the pointing tool 30) are predicted and corrected by the least square method using an nth order polynomial.

Next, a process for predicting and correcting the coordinates of the pointing tool 30 from the image before shielding will be described with reference to FIGS. 10 to 14 when the process is finished in a state where one stroke is shielded.
This process corresponds to the process from step S412 to step S419 in FIG.

FIG. 10 shows the final writing of characters (that is, “Stop”, “Splash”, “Harai”) as an example of predicting and correcting the coordinates of the pointing tool 30 from the image before shielding.
In FIG. 10, FIG. 10A shows before the prediction and correction processing of the coordinates of the pointing tool 30, and FIG. 10B shows after the prediction and correction processing of the coordinates of the pointing tool 30.

In addition, in FIG. 10, although the final writing of the character is illustrated, it is applicable also to figures other than a character. For example, if a stroke occurs during the writing of a line segment and the stroke ends, the line segment can be displayed by predicting and correcting the coordinates of the pointing tool 30 in the same manner as the “stop” at the end of the stroke. Can be made.
Also, in FIG. 10A, as in FIG. 9, the location where the shielding is generated is indicated by a dotted line portion, and the coordinate position of the pointing tool 30 that is temporarily stored due to the occurrence of the shielding is surrounded by a rectangle. Yes.

The coordinate predicting unit 157 predicts the coordinates of the pointing tool 30 at the time of shielding from the coordinates surrounded by the square in FIG. 10A and the temporal change amount of the writing pressure and acceleration of the pointing tool 30 before and after the shielding.
Here, the coordinate prediction unit 157 includes a temporal change amount of the pen pressure and acceleration held in advance in the SSD 104 and the temporal change amount and acceleration change amount calculation unit of the pen pressure calculated by the pen pressure change calculator 152. The coordinate position of the pointing tool 30 is predicted by collating with the time change amount of the acceleration calculated in 153.

The coordinate prediction and correction process will be described below. Before that, the general characteristics of writing related to the final writing (“Tome”, “Hane”, “Harai”) This will be explained from the viewpoint.
When writing “stop”, as shown in FIG. 10B, the pointing tool 30 is stopped at the end point, so that the writing pressure is increased or maintained toward the end point, and the acceleration is decreased or maintained toward the end point.
When writing “splash”, as shown in FIG. 10B, it is necessary to temporarily stop the indicator 30 at the midpoint (ie, just before the splash) and then bounce from there to the end point. It will be raised or maintained to a point and then lowered. Further, the acceleration is reduced or maintained to a midpoint, and then increases.
When writing “Harai”, as shown in FIG. 10B, it is necessary to apply it to the end point, so that the writing pressure decreases toward the end point and the acceleration increases toward the end point.
Next, with respect to the writing characteristics of the final writing described above, that is, “Tome”, “Hane”, and “Harai”, FIG. 11 is used to show the time variation of writing pressure, and FIG. The amount of change in time will be described as a graph.

FIG. 11 is a diagram illustrating a temporal change amount of the writing pressure calculated by the writing pressure change amount calculation unit.
In FIG. 11, the amount of change in writing pressure with respect to “Tome”, “Hane”, and “Harai” over time is shown.
In FIG. 11, the horizontal axis represents time, and the vertical axis represents writing pressure.

In the case of “stop”, as shown in FIG. 11A, the writing pressure is increased or maintained with time.
In the case of “splash”, as shown in FIG. 11B, the writing pressure is raised to a midpoint or maintained to some extent, and then lowered.
In the case of “Haraai”, as shown in FIG. 11C, the writing pressure decreases with the passage of time.

FIG. 12 is a diagram illustrating the amount of time change in acceleration calculated by the acceleration change amount calculation unit.
In FIG. 12, the amount of change in acceleration with respect to “Tome”, “Splash”, and “Harai” is shown.
In FIG. 12, time is plotted on the horizontal axis and acceleration is plotted on the vertical axis.

In the case of “stop”, as shown in FIG. 12A, the acceleration is decreased or maintained as time passes.
In the case of “splash”, as shown in FIG. 12B, the acceleration is reduced or maintained to a midpoint and then increased.
In the case of “Harai”, the acceleration increases as shown in FIG. 12C.

The coordinate predicting unit 157 determines one of “Stop”, “Splash”, and “Harai” from the time change amount of the writing pressure shown in FIG. 11 and the time change amount of the acceleration shown in FIG. 12, and predicts the coordinates. To do.
Note that the temporal changes in the pen pressure and acceleration have the same characteristics with respect to the final writings “Tome” and “Hane”, but since they are different thereafter, the coordinate predicting unit 157 has “Tome”, “ Can be discriminated (predicted).
In addition, the coordinate predicting unit 157 can use the slopes of the graphs shown in FIGS. 11 and 12 in predicting the coordinates of the pointing tool 30 from the amount of change in writing pressure and acceleration over time, and a predetermined value may be used as the slope. A threshold can also be set.
For example, the coordinate predicting unit 157 determines “Hara” when the slope of the amount of change in acceleration with time increases within a predetermined range (threshold).

The coordinate correction unit 158 corrects the coordinates based on the predicted coordinates.
FIG. 13 is a diagram for explaining final stroke determination (coordinate prediction) and coordinate correction based on the temporal change amount of the pen pressure and acceleration.

The coordinate predicting unit 157 determines “stop” when the writing pressure is increased or maintained and the acceleration is decreased or maintained.
In this case, the coordinate correction unit corrects the data thicker (with a certain increase in writing pressure) and shorter based on the writing characteristics of “Tome”.
The coordinate predicting unit 158 determines that the spring is “splash” when the writing pressure is raised or maintained up to a midpoint and then lowered, and the acceleration is lowered or kept up to a midpoint and then rises.
In this case, the sitting correction unit is thick to the midpoint (with a certain increase in writing pressure), short and thin to the end point (with a certain decrease in writing pressure), based on the writing characteristics of “Splash”. Correct for longer.
The coordinate prediction unit determines that the writing pressure is “Hara” when the writing pressure decreases toward the end point and the acceleration increases.
In this case, the coordinate correction unit corrects the length narrower (with a lower writing pressure) and longer based on the writing characteristics of “Harai”.

If none of “Tome”, “Hane”, and “Harai” is applicable, an intermediate value is set. The intermediate value is a value that is an intermediate value when, for example, thickening or thinning when correcting “stop”, “splash”, and “hari”.
Further, in the handwriting system according to the embodiment of the present invention, it is assumed that an inexpensive sensor that does not measure the direction of acceleration (that is, measures only a scalar amount) is mounted.
In this case, the coordinate predicting unit 158 can predict the coordinates of the pointing tool 30 without any problem with respect to “stop” and “harness” from the temporal change amount of the writing pressure and acceleration of the pointing tool 30. ”Cannot be predicted in which direction.
Therefore, a method for predicting which direction the “splash” is in will be described with reference to FIG.

FIG. 14 is a diagram for explaining a method of predicting the characteristics of “splash” and the direction of “splash”.
“Splash” has the following characteristics as shown in FIG.
FIG. 14A shows a case where the drawing is performed in the lower left direction and thereafter the “splash” operation is started, and the operation is performed in the upper left direction.
FIG. 14B shows a case in which the drawing is performed in a direction directly below and thereafter a “splash” operation is started, and the image is bounced in the upper left direction.
FIG. 14C shows a case where the drawing is performed in the lower right direction and thereafter the “splash” operation is started, and the image is splashed in the upper right direction.
FIG. 14D shows a case in which the image is drawn in the horizontal direction and then enters the “splash” operation, and is bounced in the upward direction.

The coordinate predicting unit 157 bounces based on the time variation of the writing pressure and acceleration of the pointing tool 30 and the characteristics shown in FIG. 14 (that is, the drawing direction (time variation of the coordinate position) immediately before the “splash” operation). Predict directions.
As a specific process of coordinate prediction, it is possible to select a light process that does not impose a load on the processing speed, such as a linear prediction filter, and the temporal change amount of the writing pressure and acceleration of the pointing tool 30 and the drawing direction are set. By inputting and referring to a previously-printed table or the like, the length, thickness, and direction of the character to be corrected are uniquely determined.

  Regarding “Tome”, “Hane”, and “Harai” exemplified above, the descriptions are based on general user characteristics and general character characteristics. By registering the description characteristics of the characters and figures in advance, it is possible to execute character and figure determination processing (coordinate position prediction processing and correction processing) according to the user.

  As described above, according to the handwriting system such as the electronic blackboard provided with the optical coordinate detection means according to the embodiment of the present invention, the shielding is generated between the pointing tool and the light receiving sensor. In addition, the coordinate detection accuracy of the pointing tool can be improved.

An object of the present invention is to supply a recording medium on which a program code of software that realizes the function of a handwriting system having an indicator and an electronic blackboard with a light receiving sensor is recorded to the system or apparatus. This can also be achieved by executing a program code stored in a recording medium by a computer (ie, CPU, MPU (Micro Processing Unit), DSP (Digital Signal Processor)) of the apparatus.
In this case, the program code read from the recording medium itself realizes the function of a handwriting system including an indicator and an electronic blackboard with a light receiving sensor. The program code or the recording medium storing the program is This constitutes the present invention.
Recording media for supplying the program code include FD (Floppy Disk), hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, magnetic tape, non-volatile memory, ROM and other optical recording media, magnetic A recording medium, a magneto-optical recording medium, or a semiconductor recording medium can be used.

Further, by executing the program code read by the computer, not only the function of the handwriting system is realized, but an OS (operating system) operating on the computer based on the instruction of the program code is actually used. It goes without saying that a case where the function of the handwriting system is realized by performing part or all of the processing and the processing is included.
In addition, after the program code read from the recording medium is written in a memory provided in a function expansion board inserted in the computer or a function expansion unit connected to the computer, the function expansion is performed based on the instruction of the program code. It goes without saying that the CPU or the like provided in the board or the function expansion unit performs part or all of the actual processing and the functions of the image display system are realized by the processing.

  DESCRIPTION OF SYMBOLS 1 ... Handwriting system, 10 ... Computer, 20 ... Display, 30 ... Indicator, 40 (40a-40d) ... Light receiving sensor, 50 (50a-50d) ... Peripheral light emission part.

Special table 2011-522332 gazette

Claims (7)

Detected by an electronic blackboard, an indicator that is a light-emitting input device for inputting characters or figures on the electronic blackboard, two or more light-receiving sensors that receive light emitted from the indicator, and the light-receiving sensor Coordinate calculation means for calculating the coordinates of the pointing tool based on the light reception information, shielding determination means for determining whether or not shielding has occurred between the pointing tool and the light receiving sensor, and coordinates calculated by the coordinate calculation means A display control means for controlling the characters or figures input with the pointing tool to be displayed on the electronic blackboard, and a handwriting system comprising:
Writing pressure measuring means for measuring the writing pressure of the indicator;
Acceleration measuring means for measuring the acceleration of the pointing tool;
A writing pressure change amount calculating means for calculating a time change amount of the writing pressure measured by the writing pressure measuring means;
Acceleration change amount calculating means for calculating a time change amount of acceleration measured by the acceleration measuring means;
When it is determined that shielding has occurred by the shielding determination means, from the temporal change amount of the pen pressure calculated by the writing pressure change amount calculation means and the temporal change amount of the acceleration calculated by the acceleration change amount calculation means, Coordinate prediction means for predicting the coordinate position of the pointing tool;
Coordinate correction means for correcting the coordinate position of the pointing tool based on the coordinates predicted by the coordinate prediction means;
A handwriting system. The handwriting system according to claim 1,
The electronic blackboard has storage means for storing, for each character or figure, the amount of change in writing pressure and acceleration over time when a predetermined character or figure is written with the pointing tool,
The coordinate prediction means stores the time variation of the pen pressure and acceleration stored in the storage means, the time variation of the pen pressure calculated by the pen pressure change calculation means, and the acceleration calculated by the acceleration change calculation means. A handwriting system that predicts the coordinate position of the pointing tool by collating with a time change amount. In the handwriting system according to claim 2,
The coordinate prediction means sets a predetermined range for the temporal change amount of the pen pressure and acceleration stored in the storage means, and calculates the temporal change amount and acceleration change amount of the pen pressure calculated by the pen pressure change amount calculator. The handwriting system which predicts the coordinate position of the said indicator by determining whether the time change amount of the acceleration calculated by the means exists in the said predetermined range. In the handwriting system according to claim 3,
The handwriting system which can set the time variation | change_quantity of the pen pressure and acceleration memorize | stored in the said memory | storage means for every user. The handwriting system according to any one of claims 1 to 4,
A drawing direction specifying means for specifying a drawing direction of the pointing tool from a time change of coordinates calculated by the coordinate calculating means;
The handwriting system in which the coordinate prediction means predicts the coordinate position of the pointing tool from the drawing direction of the pointing tool specified by the drawing direction specifying means. The handwriting system according to claim 5,
The handwriting system in which the drawing direction specifying means specifies the drawing direction of the pointing tool in at least two directions.   The program for functioning a computer as each means of the handwriting system described in Claim 1.

Images