JP6424506B2 - Data processing apparatus and data processing program - Google Patents

Description

  The present invention relates to a data processing apparatus and a data processing program for digitizing a movement trajectory of a writing instrument based on an operation of writing on a paper medium.

  DESCRIPTION OF RELATED ART Conventionally, the writing data processing apparatus which can digitize the locus | trajectory of a movement of a writing instrument as a handwriting based on operation | movement of the writing instrument at the time of writing with a writing instrument with respect to a base is known. Patent Document 1 rearranges a plurality of stroke data (point sequence data of input coordinates starting with pen down and ending with pen up) input in the input unit in order from the smallest X coordinate value in the stroke alignment unit Then, a handwriting data processing apparatus is disclosed that performs character recognition processing in the recognition unit.

JP 7-14007 A

  As a handwritten character recognition process that recognizes multiple handwritten characters, in order to increase the accuracy of character recognition results, structural analysis is performed in which identification is performed by focusing on the positional relationship and connection relationship of stroke data and the stroke data generation order (stroke order) The law is known. For example, in a scene where a writing data processing apparatus is used by a user, after a handwritten character string is once written, a line drawing such as a punctuation mark that was forgotten to be written is newly added in the middle of the handwritten character string already written. There is a thing. In this case, since the stroke order of the line drawing added to the handwritten character string later is to be recognized, the character recognition result intended by the user may not be obtained in the handwritten character recognition process. In the writing data processing device described in Patent Document 1, the stroke data is rearranged according to the position information regardless of the writing order, so it is difficult to apply the character recognition processing method for improving the character recognition rate with the writing order as the recognition target May be

  An object of the present invention is to provide a data processing device and a data processing program capable of executing character recognition processing with excellent accuracy even if a line drawing is added later in the middle of a handwritten character string once written.

A data processing apparatus according to a first aspect of the present invention is directed to a recognition target of information represented by coordinate data indicating a position of a writing instrument in proximity to a detection area on which a paper medium is placed in a detection apparatus along a first direction. A data processing apparatus for recognizing handwritten characters written by the user, the data corresponding to one line drawing written on the paper medium by the writing instrument close to the detection area, the data detected by the detection apparatus; A plurality of stroke data including a plurality of coordinate data are acquired in association with time data indicating a point in time when the coordinate data is detected, and stroke acquiring means is included, and one entire handwritten character written on the paper medium is included. A configuration acquisition unit configured to acquire a configuration length determined as the length in the first direction in the region; and the plurality of components acquired by the stroke acquisition unit The first stroke data associated with the time data indicating the first time point among the stroke data, and the time indicating the second time point that is later than the first time point and closest to the first time point The first position indicated by the coordinate data included in the first stroke data and the second position indicated by the coordinate data included in the second stroke data, based on second stroke data associated with data. Of the plurality of pieces of stroke data acquired by the stroke acquisition means, the change amount calculated by the calculation means is larger than the configuration length among the plurality of pieces of stroke data acquired by the stroke acquisition means, and Specific stroke which is the second stroke data in which the second position is in the second direction which is the opposite direction of the first direction than the first position When it is determined that the specific stroke data is present by the stroke determination means that determines whether or not there is a stroke data, and the specific stroke data is determined by the stroke determination means, the specific stroke data is calculated from the plurality of stroke data acquired by the stroke acquisition device. If it is determined that the specific stroke data does not exist while the stroke determination unit determines that there is no specific stroke data, the plurality of pieces of stroke data acquired by the stroke acquisition unit are determined as the recognition target. Character recognition means for recognizing a plurality of handwritten characters by at least referring to the determination means, the recognition object determined by the object determination means, and the time data associated with the recognition object, and the stroke determination Means for the specific stroke When it is determined that there is data, the specific stroke data is excluded from the position in the first direction indicated by the coordinate data included in the specific stroke data and the plurality of stroke data acquired by the stroke acquisition means A plurality of pieces of stroke data included in the comparison object based on comparison means for comparing the position in the first direction indicated by the coordinate data included in the comparison object and the comparison result obtained by the comparison means In order to insert the specific stroke data into the reference point determined by the reference point determination means, which determines the reference point for inserting the specific stroke data into the stroke data sequence arranged in time series order, The specific stroke data is arranged in chronological order in the stroke data sequence And changing means for changing the time data associated with the specific stroke data, and the specific stroke data for which the time data has been changed by the changing means to the base point determined by the base point determining means. and inserting means for inserting, line drawing corresponding to the specific stroke data, and a cross-determining means for determining whether intersects the respective line image corresponding to a plurality of the stroke data contained in the recognition target, the The object determining means determines the stroke data string into which the specific stroke data has been inserted by the inserting means as the recognition object, and the inserting means determines that the specific stroke data is included in the recognition objects by the intersection judging means. Determined not to intersect any of the line drawings corresponding to the Only if, characterized by inserting the specific stroke data to said base.

  A data processing apparatus according to a first aspect of the present invention is a first position indicated by coordinate data included in first stroke data, and a first point indicated by first time data associated with the first stroke data. An amount of change in a first direction with a second position indicated by coordinate data included in second stroke data associated with second time data indicating a second time is calculated. In the data processing device according to the present invention, the specific stroke data which is the second stroke data in which the calculated variation is larger than the configuration length and the second position is in the second direction than the first position is recognized from the recognition target Character recognition processing can be executed excluding it. Therefore, the data processing apparatus according to the present invention can execute character recognition processing with excellent accuracy even when a line drawing is added later in the middle of the handwritten character string once written.

A data processing program according to a second aspect of the present invention is directed to a recognition target of information represented by coordinate data indicating the position of a writing instrument in proximity to a detection area on which a paper medium is placed in a detection device, in a first direction. Data corresponding to one line drawing written on the paper medium by the writing instrument in proximity to the detection area in the computer of the data processing device for recognizing handwritten characters written on the paper, and detected by the detection device Obtaining a plurality of pieces of stroke data including a plurality of pieces of coordinate data in association with time data indicating a point in time when the coordinate data is detected; obtaining a stroke acquiring step; and one handwritten character written on the paper medium Acquiring the configuration length determined as the length in the first direction in the area including The first stroke data associated with the time data indicating the first time point among the plurality of stroke data acquired at the second time point, and the second time point later than the first time point and closest to the first time point On the basis of the second stroke data associated with the time data indicating two time points, the first position indicated by the coordinate data included in the first stroke data and the coordinate data included in the second stroke data Of the plurality of pieces of stroke data acquired in the stroke acquisition step, the change amount calculated in the calculation step is calculated based on the configuration length among the plurality of pieces of stroke data acquired in the stroke acquisition step. And the second position is in a second direction opposite to the first direction than the first position. A stroke determination step of determining whether or not there is a specific stroke data which is the second stroke data; and when it is determined that the specific stroke data is present in the stroke determination step, the plurality acquired in the stroke acquisition step It is determined that the specific stroke data is excluded from the stroke data of the above as the recognition target, and when it is determined that the specific stroke data is not present in the stroke determination step, the plurality of strokes acquired in the stroke acquisition step A plurality of handwritten characters are recognized with reference to at least an object determining step of determining data as the recognition object, the recognition object determined in the object determination step, and the time data associated with the recognition object. Do When it is determined that the specific stroke data is present in the character recognition step and the stroke determination step, the position in the first direction indicated by the coordinate data included in the specific stroke data and the stroke acquisition step are acquired Comparing the position in the first direction indicated by the coordinate data included in the comparison target excluding the specific stroke data from the plurality of stroke data; and based on the comparison result obtained in the comparing step. A step of determining a base point for inserting the specific stroke data into a stroke data string in which a plurality of pieces of stroke data included in the comparison object are arranged in chronological order; and the base point determined in the base point determination step Said specific stroke Changing the time data associated with the specific stroke data so that the specific stroke data is arranged in chronological order in the stroke data string in order to insert a data; and determining in the base point determination step Inserting the specific stroke data whose time data has been changed in the changing step at the changing point, and a line drawing corresponding to the specific stroke data is included in the plurality of stroke data included in the recognition target And an intersection determining step of determining whether each of the corresponding line drawings intersects , and the object determining step determines a stroke data string into which the specific stroke data is inserted in the inserting step as the recognition object. The inserting step is the crossing judgment Only if the specific stroke data in step is determined to not be intersect any line drawing corresponding to a plurality of the stroke data contained in the recognition target, characterized that you insert the specific stroke data to said base point I assume.

  In this case, when the computer of the data processing apparatus executes the data processing program of the second aspect, the same effect as that of the first aspect can be obtained.

FIG. 1 is a diagram showing an outline of a handwriting input system 1; It is a block diagram which shows the electric constitution of the reader 2 and the smart phone 19. As shown in FIG. FIG. 6 is a diagram showing a sheet 123. It is a figure which shows the example of the handwritten character string handwritten in the entry area 123A. It is a flowchart of a 1st main process. It is a flowchart of 2nd main processing. It is a flowchart of 2nd main processing. It is a flowchart of a line feed determination process. It is a flowchart of return additional recording processing.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. An outline of the handwriting input system 1 according to the present embodiment will be described with reference to FIG. In the following description, the upper left side, the lower right side, the upper side, the lower side, the upper right side, and the lower left side of FIG. 1 are respectively defined as the left side, the right side, the front side, the rear side, the upper side, and the lower side of the reading device 2 explain.

  As shown in FIG. 1, the handwriting input system 1 mainly includes a reading device 2, an electronic pen 3, and a smartphone 19. The reading device 2 is a thin and light handwriting input device that can be folded and carried. In the handwriting input system 1, the user uses the electronic pen 3 to write points and line drawings on the sheet 123 of the paper medium 100 mounted on the reading device 2. Below, the case where a line drawing is written is demonstrated. Line drawings include characters, numbers, symbols, figures, and the like. The reading device 2 acquires the position of the electronic pen 3 in the process of writing. The reader 2 creates a stroke data file based on the acquired position. The reading device 2 can transmit the created stroke data file to the smartphone 19. When the smartphone 19 receives the stroke data file, the smartphone 19 causes the display 192 to display an image corresponding to the received stroke data file. Thus, the user can visually recognize, through the display 192, a line drawing having the same shape as the line drawing written on the sheet 123 by the electronic pen 3.

  The reading device 2 has a pair of left and right reading devices 2L, a right reading device 2R, and a cover 4 as main components. The left reader 2L and the right reader 2R are in the form of rectangular thin plates. The left reading device 2 </ b> L and the right reading device 2 </ b> R are disposed on the front surface of the cover 4 so as to be spread in the left-right direction. The left reader 2L and the right reader 2R are electrically connected by a flat cable (not shown). The right reader 2R has three LEDs 5 at the top. The LED 5 can notify the user of the state of the reading device 2. The cover 4 is provided with a bag-like bag portion 4A on the left side. The left reader 2L is detachably attached to the cover 4 by being inserted into the bag 4A. The right reading device 2R is attached to the right front surface of the cover 4 by a double-sided tape, an adhesive resin film, or the like.

  The paper medium 100 is detachably mounted on the front of the reading device 2. The paper medium 100 is a booklet that can be spread in the left-right direction. In the paper medium 100, a pair of cover sheets (front cover 110L and back cover 110R) and a plurality of sheets 123 are bound at part of each edge. As an example, the paper medium 100 is an A5 size notebook. The format indicating the layout or the like of the pattern printed in advance on the sheet 123 differs depending on the type of the paper medium 100. The paper medium 100 is mounted on the reader 2 such that the front cover 110L is placed on the front of the left reader 2L and the back cover 110R is placed on the front of the right reader 2R. In the present embodiment, the paper medium 100 is mounted in a state where the paper medium 100 is positioned in the reading device 2 by a double-sided tape, a resin film having adhesiveness, or the like. That is, the left reading device 2L and the right reading device 2R move integrally with the front cover 110L and the back cover 110R, respectively. The user can write a line drawing on the paper 123 of the paper medium 100 using the electronic pen 3.

  The electronic pen 3 is a known electromagnetic induction type electronic pen, and mainly includes a cylinder 30, a core 31, a coil 32, capacitors 33 and 35, a substrate 34, and an ink storage portion 36. The cylindrical body 30 has a cylindrical shape, and accommodates a part of the core body 31, the coil 32, the capacitors 33 and 35, the substrate 34, and the ink storage portion 36 inside. The core 31 is provided at the tip of the electronic pen 3. The core body 31 is urged toward the tip end of the electronic pen 3 by an elastic member (not shown). The tip end of the core body 31 protrudes to the outside of the cylindrical body 30. The rear end side of the core body 31 is connected to the ink storage portion 36 in which the ink is stored. The ink storage unit 36 supplies ink to the core body 31. When the user writes on the sheet 123 using the electronic pen 3, a line drawing is formed on the sheet 123 by the ink.

  The coil 32 is held between the core body 31 and the capacitor 33 in a state of being wound around the ink storage portion 36. The capacitor 33 is fixed inside the electronic pen 3 by the substrate 34. A capacitor 35 is mounted on the substrate 34. Capacitors 33 and 35 are connected in parallel to coil 32 to form a well-known resonant (tuning) circuit.

  The smartphone 19 includes a touch panel 191 and a display 192. The touch panel 191 is used to input various instructions. The display 192 can display an image corresponding to the image file. Note that a general-purpose PC or tablet PC may be used instead of the smartphone 19.

  The electrical configuration of the handwriting input system 1 will be described with reference to FIG. First, the electrical configuration of the reading device 2 and the outline of the principle of detection of coordinate data by the reading device 2 will be described. The reader 2 includes sensor substrates 7L and 7R, a main substrate 20, sensor control substrates 28, 29, an input unit 25, and three LEDs 5. The sensor substrates 7L and 7R are provided in the left reading device 2L and the right reading device 2R, respectively. The input unit 25 and the three LEDs 5 are provided in the right reading device 2R.

  The main substrate 20 includes a CPU 21, a RAM 22, a flash ROM 23, and a wireless communication unit 24. The RAM 22, the flash ROM 23, and the wireless communication unit 24 are electrically connected to the CPU 21. The CPU 21 controls the reading device 2. The RAM 22 temporarily stores various data such as operation data. The flash ROM 23 stores a first main program that the CPU 21 executes to control the reading device 2. In addition, the flash ROM 23 stores a plurality of layout data for each format of the sheet 123. The wireless communication unit 24 is a controller for performing near field communication with an external electronic device. The input unit 25 and the three LEDs 5 are electrically connected to the CPU 21. The input unit 25 is a switch for inputting an instruction to the reading device 2. The color of each of the three LEDs 5 is yellow, green and red.

  On the sensor boards 7L and 7R, a large number of elongated loop coils are arranged in each of the left and right direction (X axis direction) and the up and down direction (Y axis direction). The sensor substrate 7L is electrically connected to the ASIC 28A of the sensor control substrate 28. The ASIC 28A detects coordinate data indicating the position of the electronic pen 3 when the writing operation by the electronic pen 3 is performed on the sensor substrate 7L. The sensor substrate 7R is electrically connected to the ASIC 29A of the sensor control substrate 29. The ASIC 29A detects coordinate data indicating the position of the electronic pen 3 when the writing operation by the electronic pen 3 is performed on the sensor substrate 7R. Among the ASICs 28A and 29A, the ASIC 28A on the master side is directly connected to the CPU 21, and the ASIC 29A on the slave side is connected to the CPU 21 via the ASIC 28A.

  The principle of detecting coordinate data when the writing operation by the electronic pen 3 is performed on the sensor substrates 7L and 7R will be schematically described. The CPU 21 controls the ASICs 28A and 29A to flow a current (excitation transmission current) of a specific frequency one by one to each of the loop coils of the sensor substrates 7L and 7R. Thereby, a magnetic field is generated from each loop coil of sensor boards 7L and 7R. In this state, for example, when the user performs an operation of writing a line drawing on the sheet 123 of the paper medium 100 mounted on the reading device 2 using the electronic pen 3, the electronic pen 3 approaches the sensor substrates 7L and 7R. Therefore, the resonance circuit of the electronic pen 3 resonates by electromagnetic induction to generate an induction magnetic field.

  Next, the CPU 21 controls the ASICs 28A and 29A to stop the generation of the magnetic field from the loop coil of each of the sensor boards 7L and 7R. The loop coil of each of the sensor substrates 7 </ b> L and 7 </ b> R receives an induced magnetic field emitted from the resonant circuit of the electronic pen 3. The CPU 21 controls the ASICs 28A and 29A to detect a signal current (reception current) flowing through each of the loop coils of the sensor boards 7L and 7R. The ASICs 28A and 29A execute this operation one by one for all loop coils to detect the reception current. The CPU 21 detects coordinate data indicating the position of the electronic pen 3 by detecting the position in the X-axis direction and the Y-axis direction of the loop coil having detected the reception current as coordinate data. In the present embodiment, the origin of the coordinate data is the uppermost left position of the left reading device 2L and the right reading device 2R. The coordinate data is represented by X-coordinate values or Y-coordinate values of larger values as the position is (X, Y) = (0, 0) and the position is farther to the right or downward from the origin. The X coordinate value and the Y coordinate value are expressed in units of the number of loop coils. When the paper medium 100 is loaded into the reading device 2, the uppermost left corner of the sheet 123 is positioned at the uppermost left position of the left reading device 2L and the right reading device 2R, which is the origin position of the coordinate data.

  In a state where a line drawing is written on the paper 123 using the electronic pen 3, a pen pressure is applied to the core body 31. The inductance of the coil 32 changes in accordance with the writing pressure applied to the core body 31. Thereby, the resonance frequency of the resonance circuit of the electronic pen 3 changes in accordance with the writing pressure applied to the core body 31. The CPU 21 detects a change in the resonance frequency (phase change) to specify the writing pressure applied to the electronic pen 3. The CPU 21 can determine whether or not the line drawing is being written on the paper 123 of the paper medium 100 based on the specified pen pressure.

  Next, the electrical configuration of the smartphone 19 will be described. The smartphone 19 mainly includes a CPU 41, a RAM 42, a flash ROM 43, a wireless communication unit 44, an input circuit 45, an output circuit 46, a touch panel 191, and a display 192. The CPU 41 controls the smartphone 19. The CPU 41 is electrically connected to the RAM 42, the flash ROM 43, the wireless communication unit 44, the input circuit 45, and the output circuit 46.

  The RAM 42 stores various temporary data. The wireless communication unit 44 is a controller for performing near field communication with an external electronic device. The input circuit 45 performs control to send an instruction from the touch panel 191 to the CPU 41. The output circuit 46 performs control to display an image on the display 192 in accordance with an instruction from the CPU 41. The flash ROM 43 stores programs executed by the CPU 41. The smartphone 19 includes a medium reader (for example, a memory card slot) not shown. The smartphone 19 can read a program stored in a storage medium (for example, a memory card) by the medium reader and install the program in the flash ROM 43. Alternatively, the program may be received from an external device (not shown) connected to the smartphone 19 or a network and installed in the flash ROM 43.

  The paper 123 of the paper medium 100 will be described with reference to FIG. The left side, the right side, the upper side, and the lower side of FIG. 3 will be described as the left side, the right side, the upper side, and the lower side of the sheet 123, respectively. FIG. 3 shows the sheet 123 of one page of the paper medium 100. As shown in FIG. 3, the sheet 123 is a note sheet for writing a note. The form 123 has an entry area 123A and a check box 123B. In the entry area 123A, a plurality of ruled lines 123K extending in the left-right direction are printed at equal intervals in the top-bottom direction. The arrangement interval of the plurality of ruled lines 123K is set to the configuration length W. Since a plurality of ruled lines 123K are provided in the writing area 123A, the user often writes a line drawing along the ruled lines 123K when writing a line drawing in the writing area 123A. Therefore, the whole of one handwritten character written in the entry area 123A is likely to be included in a square area R whose one side is the configuration length W, which is an interval between the plurality of ruled lines 123K. In the present embodiment, various determinations are made based on the size of the region R in the second main process (see FIGS. 6 and 7) described later.

  The flash ROM 23 is information corresponding to the format of the sheet 123, and can specify the positions of the entry area 123A and the check box 123B of the sheet 123, and the configuration length W which is the length of one side of the area R Layout layout data. The CPU 21 refers to the layout data stored in the flash ROM 23 to acquire the configuration length W, and transmits the acquired configuration length W to the smartphone 19. The region R in FIG. 3 is illustrated to explain the size occupied by the region R, and a square region indicating the region R is not provided on the actual sheet 123. The entry area 123A is, for example, an area for the user to write a memo, like a handwritten character string shown in a dotted line frame 201. The check box 123B is printed at the lower right of the entry area 123A. The shape of the check box 123B is a rectangular single line. The check box 123B is an area where the user writes a line drawing in order to confirm the line drawing written in the writing area 123A.

  With reference to FIG. 3, an outline of processing when the CPU 21 of the reading device 2 creates a stroke data file based on the line drawing written on the sheet 123 will be described. While the pen pressure is applied to the electronic pen 3, the CPU 21 of the reading device 2 repeatedly acquires coordinate data indicating the position of the electronic pen 3 through the ASICs 28 </ b> A and 29 </ b> A in a constant cycle. The CPU 21 stores time data indicating the time at which the coordinate data is acquired, in the first area of the RAM 22 in association with each of the plurality of acquired coordinate data. The plurality of coordinate data and the plurality of time data stored in the first area of the RAM 22 when application of the writing pressure to the electronic pen 3 is completed is one of the line drawings written on the paper 123 by the electronic pen 3. Indicates the position of the line drawing. Hereinafter, a plurality of coordinate data indicating the position of one line drawing and a plurality of time data associated with each of the plurality of coordinate data are collectively referred to as stroke data. In the present embodiment, the stroke data includes, as header information, time data and the like associated with coordinate data acquired first among a plurality of coordinate data included in each stroke data. The time data included in the header information is hereinafter referred to as a time stamp of each stroke data. Hereinafter, the sheet 123 will be described as an example.

  Based on the stroke data stored in the first area of the RAM 22, the CPU 21 determines which of the entry area 123A of the sheet 123 and the check box 123B the line drawing has been written. When determining that the line drawing has been written in the entry area 123A, the CPU 21 stores the stroke data stored in the first area of the RAM 22 in the second area of the RAM 22, and clears the first area of the RAM 22. Each time the user writes a line drawing in the writing area 123A one by one using the electronic pen 3, stroke data is sequentially stored in the second area of the RAM 22.

  When determining that the line drawing has been written in the check box 123B, the CPU 21 creates a stroke data file including at least one stroke data stored in the second area of the RAM 22. The stroke data file includes at least one stroke data acquired from the time when the line drawing is written in the check box 123B to the time when the line drawing is written in the next check box 123B. Thus, in the stroke data file, a plurality of stroke data are arranged in order of time stamp of each stroke data. The CPU 21 stores the created stroke data file in the flash ROM 23.

  An overview of processing when the CPU 41 of the smartphone 19 acquires a stroke data file from the reading device 2 and creates an image file will be described. The CPU 41 of the smartphone 19 transmits a data request command (described later) for requesting transmission of a stroke data file or the like to the reading device 2. When the CPU 21 of the reading device 2 receives a data request command from the smartphone 19, the CPU 21 transmits the stroke data file stored in the flash ROM 23 to the smartphone 19. The CPU 41 of the smartphone 19 receives the stroke data file transmitted from the reading device 2 and stores the stroke data file in the RAM 42.

  The CPU 41 extracts at least one stroke data included in the stroke data file stored in the RAM 42 one by one. The CPU 41 connects a plurality of positions indicated by a plurality of coordinate data corresponding to the extracted stroke data with a straight line in the order in which each of the plurality of coordinate data is acquired. The CPU 41 combines line drawings obtained one by one for each stroke data, and specifies them as a plurality of line drawings written on the sheet 123. The CPU 41 creates an image file of an image including only the vicinity of a plurality of specified line drawings. The image file is a data file showing a line drawing by a digital image. Examples of digital images include vector images and raster images. Image files include JPEG files, GIF files, PNG files, and BMP files. The CPU 41 causes the display 192 to display an image based on the created image file.

  Further, the CPU 41 executes handwritten character recognition processing that recognizes a plurality of handwritten characters, with the stroke data file stored in the RAM 42 as a recognition target. A well-known method can be adopted for the handwriting character recognition process. In this embodiment, stroke data is extracted from a stroke data file created for a handwritten character string written in horizontal writing from left to right in order from left to right, and the positional relationship and connection relationship of the stroke data, and each of the stroke data A structural analysis method is used in which identification is performed by paying attention to time data (line drawing stroke order) or the like associated with. In addition to this, in the handwriting character recognition process, a pattern matching method can be used which compares a stroke data file and a model image called a template. The handwriting character recognition process may adopt a part of methods including these examples, or may combine a plurality of methods including other methods. In addition, for multiple character candidates recognized by handwriting character recognition processing, context processing that optimizes character strings that are recognition results, taking into account the probability of being a language, such as the connection relationship (context) of character information. May be performed together. In this case, since the most probable character candidate is selected in consideration of the entire text of the handwritten character string, the accuracy of the recognition result can be enhanced.

  An example in which the user writes a handwritten character string (hereinafter referred to as a handwritten character string) using the reader 2 will be described with reference to FIGS. 3 and 4. In FIG. 4, the ruled line 123K is omitted. As shown in FIG. 3, it is assumed that a handwritten character string “I have a good weather today”, which is composed of a dotted line frame 201 and a dotted line frame 202, is written in an entry area 123A of a sheet 123 attached to the reading device 2. As for the handwritten character string, it is assumed that handwritten characters constituting the handwritten character string are written in order from left to right. However, line drawings such as punctuation marks that were forgotten to be written may be added to a handwritten character string (hereinafter referred to as an existing character string) already written to complete the handwritten character string.

  A specific example is given and demonstrated. As shown in FIG. 4 (A), it is assumed that the user first describes the handwritten character string “Today is fine weather” (see dotted line frame 201) from left to right. After that, as shown in FIG. 4 (B), the user reads the reading point “in the position between“ ha ”and“ good ”which is the position returned to the left from the end of the existing character string shown by the dotted line frame 201 (Refer to dotted line frame 202) may be added to complete a handwritten character string. When handwritten characters constituting a handwritten character string consisting of a dotted line frame 201 and a dotted line frame 202 are written in order from left to right, and with respect to the dotted line frame 201 which is an already existing character string, Since the time data associated with the stroke data particularly indicating the reading point “,” is different from the additionally written case, the created stroke data file is also different. Therefore, when a line drawing is added to an existing character string, a character recognition result not intended by the user can be obtained, unlike the character recognition result obtained when the handwritten character string is sequentially written from left to right. is there. For example, when handwritten character recognition processing is faithfully performed on time data associated with stroke data, a line drawing is added to an existing character string, and thus, a handwritten character string to which a line drawing is added is formed. Character recognition results for one line may be inaccurate.

  Also, as shown in FIG. 4 (C), even if there is not enough space between the handwritten characters as in the handwritten character string “Today is a good weather” (see dotted line frame 203), FIG. 4 (D) As shown, a line drawing indicating a reading point “,” (see the dotted line frame 204) may be additionally written. In FIG. 4D, since there is not a sufficient space between “ha” and “good” in the handwritten character string indicated by the dotted line frame 203, a line drawing (see the dotted line frame 204) indicating the added reading point “,” However, it shows a state where it intersects with a line drawing that constitutes a handwritten character of "ha" and "good".

  The CPU 41 excludes the stroke data corresponding to the line drawing returned and added to the existing character string from the stroke data file to be recognized in the handwriting character recognition process, thereby making the character recognition result as intended by the user. It can be approached. In addition, the CPU 41 corresponds to the added line drawing when the added line drawing is sequentially written together with another handwritten character string with reference to coordinate data included in a plurality of stroke data in the stroke data file. It is possible to determine the origin at which stroke data is considered to be inserted into the stroke data file. Further, the CPU 41 can change the time stamp of the stroke data to be inserted at the determined base point, and arrange the stroke data inserted at the base point in time-series order with other stroke data. Further, the CPU 41 can insert stroke data corresponding to the added line drawing into the stroke data file, when the line drawing constituting the existing character string does not intersect the line drawing added to the existing character string. . It may be difficult to make an accurate judgment only by comparing the position of the line drawing if the line drawing added so as to intersect the line drawing constituting the existing character string is intentionally added to any position of the existing character string . The CPU 41 does not insert stroke data corresponding to a line drawing that is difficult to accurately identify such a base point and time stamp into the stroke data file, but only stroke data for which the base point and time stamp can be accurately identified. Can be inserted into Thus, the CPU 41 can increase the accuracy of the character recognition result for the stroke data file. Details will be described below.

  The first main process executed by the CPU 21 of the reading device 2 will be described with reference to FIG. The CPU 21 starts the first main processing by operating based on the program stored in the flash ROM 23 when the power supply of the reading device 2 is turned on.

  First, the CPU 21 executes the following initialization process (S10). The CPU 21 clears the data stored in the RAM 22. The CPU 21 starts control of the ASICs 28A and 29A. As a result, the CPU 21 can determine whether a line drawing has been written on the sheet 123 of the paper medium 100 mounted on the reading device 2 using the electronic pen 3. Further, when it is determined that the line drawing is being written using the electronic pen 3, the CPU 21 can acquire coordinate data indicating the position of the electronic pen 3.

  The CPU 21 specifies the format of the paper 123 of the paper medium 100 mounted on the reading device 2 (S11). Specifically, the CPU 21 specifies the format as follows. The CPU 21 turns on the red LED 5 to notify the user that the format of the sheet 123 is not specified. The user writes a line drawing with the electronic pen 3 in the order corresponding to the format of the sheet 123 at the positions of a plurality of calibration marks (not shown) printed at the corners of the sheet 123. The CPU 21 sequentially acquires a plurality of coordinate data indicating the position at which the operation of writing the line drawing is performed, and specifies the position at which the line drawing is written and the order in which the line drawing is written. The CPU 21 specifies the format of the sheet 123 corresponding to the specified position and order. Hereinafter, the case where the sheet 123 (see FIG. 3) is mounted on the reading device 2 and the format of the sheet 123 is specified will be specifically described by way of example.

  The CPU 21 reads layout data corresponding to the specified format of the sheet 123 from the flash ROM 23 and stores it in the RAM 22 (S11). The CPU 21 specifies the configuration length W based on the layout data stored in the RAM 22 in S11. Further, the CPU 21 specifies an area corresponding to the check box 123B in the area where the paper 123 is loaded, that is, the front surface of the reading device 2, based on the layout data stored in the RAM 22 in S11. Hereinafter, the area on the front surface of the reading device 2 corresponding to the rectangular area constituting the check box 123B is referred to as a check area.

  The CPU 21 determines whether a line drawing is being written on the sheet 123 based on the pen pressure applied to the electronic pen 3 (S17). When it is determined that the line drawing is being written on the sheet 123 (S17: YES), the CPU 21 acquires coordinate data. The CPU 21 further acquires time data indicating a time when the acquired coordinate data is detected (S24). The CPU 21 associates the acquired coordinate data and time data, and stores it in the first area of the RAM 22 (S24). The CPU 21 repeats the process of storing the acquired coordinate data and time data in the first area of the RAM 22 until the writing of one line drawing by the electronic pen 3 is completed. When writing of the line drawing by the electronic pen 3 is completed, a plurality of coordinate data and a plurality of time data corresponding to one line drawing are stored in the first area of the RAM 22 as stroke data.

  The CPU 21 determines whether a line drawing has been written in the check box 123B based on at least one stroke data stored in the first area of the RAM 22 in S24 (S25). Specifically, when at least one of the plurality of coordinate data of the acquired at least one stroke data indicates a position within the check area, the CPU 21 determines that the line drawing has been written in the check box 123B (S25: YES). On the other hand, when all the plurality of coordinate data indicate the position outside the check area, the CPU 21 determines that the line drawing is not written in the check box 123B (S25: NO).

  When determining that the line drawing has not been written in the check box 123B (S25: NO), the CPU 21 determines that the line drawing has been written in the entry area 123A. The CPU 21 stores at least one stroke data stored in the first area of the RAM 22 in the process of S24 in the second area of the RAM 22 (S29). The CPU 21 deletes at least one stroke data stored in the second area of the RAM 22 from the first area of the RAM 22 (S30). The CPU 21 returns the process to S17.

  On the other hand, when determining that the line drawing has been written in the check box 123B (S25: YES), the CPU 21 creates a stroke data file including at least one stroke data stored in the second area of the RAM 22 (S27). The CPU 21 stores the created stroke data file in the flash ROM 23 (S27). The CPU 21 deletes at least one stroke data included in the created stroke data file from the second area of the RAM 22 (S28). The CPU 21 returns the process to S17.

  When the CPU 21 determines that the line drawing is not written on the sheet 123 in the process of S17 (S17: NO), the data request command wirelessly transmitted from the smartphone 19 is received via the wireless communication unit 24. It is determined whether it has been done (S19). If the CPU 21 determines that the data request command has not been received (S19: NO), it returns the process to S17. When the CPU 21 determines that the data request command has been received (S19: YES), the stroke data file stored in the flash ROM 23 by the process of S27 and the information indicating the configuration length W stored in the RAM 22 by the process of S11 are wirelessly transmitted It wirelessly transmits to the smart phone 19 via the communication unit 24 (S21). The CPU 21 deletes the stroke data file transmitted to the smartphone 19 from the flash ROM 23 (S22). The CPU 21 returns the process to S17.

  The CPU 21 develops the program stored in the flash ROM 23 on the RAM 22 to function as an example of a processor that executes the first main process. A microcomputer, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or the like may be used as a processor instead of the CPU 21. Further, the first main processing may be distributed and processed by a plurality of electronic devices (that is, a plurality of CPUs). For example, part of the first main process may be executed by the CPU 41 of the smartphone 19.

  The second main process executed by the CPU 41 of the smartphone 19 will be described with reference to FIGS. 6 and 7. The CPU 41 starts the second main process by operating based on the program stored in the flash ROM 43 when the start operation of the application for wireless connection with the reading device 2 is performed through the touch panel 191. .

  As shown in FIG. 6, first, the CPU 41 executes the following initialization process (S41). The CPU 41 clears the data stored in the RAM 42. The CPU 41 transmits and receives an ID and the like to and from the reading device 2 in order to start wireless communication with the reading device 2. Next, the CPU 41 causes the display 192 to display a button for requesting acquisition of the stroke data file stored in the reading device 2 used by the user via the touch panel 191. The CPU 41 monitors an input operation detected via the touch panel 191.

  When the CPU 41 determines that the operation of selecting the button is detected (S42: YES), the stroke data file is acquired from the reading device 2 as follows. The CPU 41 wirelessly transmits a data request command requesting acquisition of a stroke data file to the reading device 2 via the wireless communication unit 44 (S43). The CPU 41 receives the stroke data file wirelessly transmitted from the reading device 2 according to the transmitted data request command via the wireless communication unit 44, and stores the received stroke data file in the RAM 42 (S44). Further, the CPU 41 receives, via the wireless communication unit 44, the information indicating the configuration length W wirelessly transmitted from the reading device 2 according to the transmitted data request command, and transmits the information indicating the received configuration length W to the RAM 42. The process is stored (S45), and the process proceeds to S51 (see FIG. 7).

  As shown in FIG. 7, the CPU 41 refers to stroke data (stroke data having a time stamp indicating the earliest point in the stroke data file) to be first arranged in the stroke data file stored in the RAM 42 (S51). In the following description, among the stroke data included in the stroke data file, the stroke data referred to by the CPU 21 for various determinations in the second main processing is referred to as “stroke S”. Further, in the second main processing, the CPU 41 determines the positional relationship on the sheet 123 of the line drawing corresponding to each stroke data based on the bounding box (BBOX) of each stroke. BBOX is a coordinate data group indicating a rectangular area circumscribing a line drawing corresponding to one or more stroke data, and the rectangular area is surrounded by a side extending along the X-axis direction and a side extending along the Y-axis direction Area. The SBBOX is a BBOX indicated by coordinate data included in the stroke S. The row BBOX is a BBOX indicated by coordinate data included in one or more stroke data that the CPU 41 has already referred to in the second main processing. In the paper 123, a line drawing may be written over a plurality of lines, but the line BBOX is indicated by coordinate data of stroke data corresponding to a line drawing written on the same line as the line drawing corresponding to the stroke S. The CPU 41 initializes the row BBOX to the SBBOX and stores it in the RAM 42. (S51). At this point, SBBOX is a BBOX based on the stroke data to be first arranged in the stroke data file. Next, the CPU 41 executes a line feed determination process (S52).

  The line feed determination processing (S52, see FIG. 7) will be described with reference to FIG. The line feed determination process is a process for determining whether or not the stroke S corresponds to a line drawing written with a line break with respect to the handwritten character string written on the sheet 123. As shown in FIG. 8, when the line feed determination process is started, the CPU 41 refers to the coordinate data included in the stroke S, identifies the SBBOX, and stores the identified SBBOX in the RAM 42 (S71). The CPU 41 refers to the SBBOX stored in the RAM 42, and acquires the minimum X coordinate value indicated by the SBBOX (S72). Further, the CPU 41 refers to the SBBOX stored in the RAM 42 to acquire the minimum Y coordinate value indicated by the SBBOX (S72). In the following description, it is assumed that the minimum X coordinate value and the minimum Y coordinate value indicated by SBBOX acquired in the process of S72 are SX1 and SY1, respectively. Next, the CPU 41 refers to the row BBOX stored in the RAM 42 (S73). The CPU 41 acquires the minimum X coordinate value indicated by the row BBOX. (S74). Further, the CPU 41 acquires the maximum Y coordinate value indicated by the row BBOX (S74). In the following description, it is assumed that the minimum X coordinate value and the maximum Y coordinate value indicated by the row BBOX acquired in the process of S74 are LX1 and LY2, respectively.

  The CPU 41 determines whether the value indicated by SX1 is smaller than the sum of LX1 and the configuration length W (S75). In this process, the CPU 41 executes an X process on a line drawing constituting a handwritten character written on the leftmost side among line drawing groups corresponding to one or a plurality of stroke data included in the line BBOX. It can be determined whether or not the writing has been made at the same position in the axial direction or at a position shifted further to the left. If the value indicated by SX1 is smaller than the sum of LX1 and the configuration length W (S75: YES), the CPU 41 determines whether SY1 indicates a value larger than LY2 (S76). In this process, the CPU 41 can determine whether or not the line drawing corresponding to the stroke S has been written below the line drawing group corresponding to one or a plurality of stroke data included in the row BBOX. If the SY1 indicates a value larger than LY2 (S76: YES), the CPU 41 gives the stroke S an identifier for making the stroke S distinguishable from other stroke data in the stroke data file, and the process is performed as the second main It returns to the process (S77). That is, the identifier given to the stroke S in the process of S77 is an identifier for identifying that the stroke S is stroke data corresponding to a line drawing in which a line feed is performed. In the following description, stroke data corresponding to a line drawing subjected to a line feed is referred to as a “line feed stroke”. On the other hand, if the value indicated by SX1 is equal to or greater than the value obtained by adding LX1 and the configuration length W (S75: NO), and if SY1 indicates a value less than LY2 (S76: NO) Return to 2 main processing.

  It returns to the explanation of FIG. Next, the CPU 41 determines whether the stroke S is a line feed stroke based on whether the stroke S has an identifier (S53). If the stroke S is a line feed stroke (S53: YES), the CPU 41 initializes the row BBOX to the SBBOX, stores it in the RAM 42, and shifts the process to the determination of S61. (S58).

  On the other hand, if the stroke S is not a line feed stroke (S53: NO), the CPU 41 refers to the row BBOX stored in the RAM 42 and acquires the maximum X coordinate value indicated by the row BBOX (S54). Further, the CPU 41 refers to the SBBOX stored in the RAM 42, and acquires SX1, which is the minimum X coordinate value indicated by the SBBOX (S54). In the following description, the maximum X coordinate value indicated by the row BBOX acquired in the process of S54 is set to LX2. The CPU 41 determines whether the value indicated by SX1 is smaller than the value obtained by subtracting the configuration length W from LX2 (S55). In this process, the CPU 41 determines the length of one handwritten character (configuration length W) from the end of the rightmost end of the line drawing group corresponding to one or a plurality of stroke data included in the line BBOX. It can be judged whether or not it is a line drawing additionally written at the position returned to the left above. In the following description, the line drawing added at the position returned to the left more than the length of one handwritten character from the end on the rightmost side of the line drawing group corresponding to one or more stroke data included in line BBOX , "Return added additional line drawing". If the value indicated by SX1 is smaller than the value obtained by subtracting the configuration length W from LX2 (S55: YES), the CPU 41 executes the return additional recording process (S56), and shifts the process to the determination of S61. If the value indicated by SX1 is equal to or greater than the value obtained by subtracting the configuration length W from LX2 (S55: NO), the CPU 41 updates the row BBOX by including the coordinate data included in SBBOX in the row BBOX at the current point The row BBOX is stored in the RAM 42 (S57). Thereafter, the CPU 41 shifts the processing to the determination of S61.

  The return additional recording process (S56, see FIG. 7) will be described with reference to FIG. In the return additional recording process, when the stroke S returns and corresponds to the additionally drawn line drawing, the stroke S is excluded from the stroke data file or the stroke in the stroke data file according to the positional relationship between the stroke S and other stroke data. This is a process of determining a base point TS into which S is inserted. As shown in FIG. 9, when the return appending process is started, the CPU 41 excludes the stroke S from the stroke data file stored in the RAM 42 and stores the stroke S in the RAM 42 separately from the stroke data file ( S80). The CPU 41 specifies the SBBOX with reference to the coordinate data included in the stroke S, and stores the specified SBBOX in the RAM 42 (S81). The CPU 41 refers to the SBBOX stored in the RAM 42, and acquires SX1 which is the minimum X coordinate value indicated by SBBOX and SX2 which is the maximum X coordinate value indicated by SBBOX (S82).

  The CPU 41 refers to the stroke data file stored in the RAM 42, and acquires a list LS which is a list in which a plurality of stroke data included in the same line as the stroke S are arranged (S83). The list LS can be obtained by focusing on a line feed stroke among the stroke data arranged in the stroke data file. Specifically, among the plurality of stroke data arranged in the stroke data file, the stroke data arranged from the stroke data arranged at the beginning of the stroke data file to the previous one of the line feed stroke arranged next is the same row Are a plurality of stroke data included in. Further, among a plurality of stroke data arranged in the stroke data file, stroke data arranged from a certain line feed stroke to one before the next line feed stroke is a plurality of stroke data included in the same line. Further, among the line feed stroke data of the stroke data file, stroke data arranged at the end of the stroke data file from line feed stroke data having a time stamp indicating the latest time point is a plurality of stroke data included in the same line. The CPU 41 acquires a stroke data string including the stroke S among such stroke data strings as the list LS.

  The CPU 41 extracts, from the list LS acquired in the process of S83, a stroke SL which is stroke data to be arranged first in the list LS (S84). In the following description, among the stroke data included in the list LS, the stroke data referred to by the CPU 41 for various determinations in the return additional recording process will be referred to as a stroke SL. The CPU 41 sets a base point TS indicating a time point to be a base point for inserting the stroke S into the list LS, at a time point obtained by subtracting the minimum unit time from the time stamp of the stroke SL (S85). The minimum unit time is a time equal to one cycle of a cycle in which the CPU 21 of the reading device 2 repeatedly acquires coordinate data indicating the position of the electronic pen 3 through the ASICs 28A and 29A.

  The CPU 41 refers to the coordinate data included in the stroke SL, specifies SLBBOX which is a BBOX of the stroke SL, and acquires the minimum X coordinate value and the maximum X coordinate value indicated by SLBBOX (S86). In the following description, it is assumed that the minimum X coordinate value and the maximum X coordinate value indicated by SLBBOX acquired in the process of S86 are SLX1 and SLX2, respectively. The CPU 41 determines whether the value indicated by SX1 is larger than the value indicated by SLX2 (S87). In this process, the CPU 41 can determine whether the SBBOX of the line drawing corresponding to the stroke S is disposed on the right side of the SLBBOX of the line drawing corresponding to the stroke SL. When the value indicated by SX1 is larger than the value indicated by SLX2 (S87: YES), the CPU 41 updates the base point TS so as to indicate the point when the minimum unit time is added to the time stamp of the stroke SL (S90). The CPU 41 shifts the processing to the determination of S92.

  Next, the CPU 41 determines whether the stroke SL is the last stroke data in the list LS (S92). If the stroke SL is not the last stroke data in the list LS (S 92: NO), the CPU 41 updates the stroke SL to stroke data arranged in the following order in the list LS (S 93), and the stroke SL is the last stroke in the list LS The processes of S86 to S92 are repeatedly executed until data is obtained. When the stroke SL is the last stroke data in the list LS (S92: YES), the CPU 41 changes the time stamp of the stroke S to the time indicated by the base point TS set in the process of S90 (S94). In this process, the CPU 41 can use the time point indicated by the base point TS at which the stroke S is inserted into the list LS as the time stamp of the stroke S. The CPU 41 inserts the stroke S into the base point TS in the stroke data file list LS to update the stroke data file, and stores the stroke data file in the RAM 42 (S95). The CPU 41 returns the process to the second main process (see FIG. 7).

  On the other hand, when the value indicated by SX1 is less than or equal to the value indicated by SLX2 (S87: NO), the CPU 41 determines whether the value indicated by SX2 is smaller than the value indicated by SLX1 (S88). In this process, the CPU 41 determines whether the SBBOX of the line drawing corresponding to the stroke S is disposed on the left side of the SLBBOX of the line drawing corresponding to the stroke SL, that is, the line drawing corresponding to the stroke S corresponds to the stroke SL. It can be determined whether it is disposed on the left side of the line drawing and does not intersect the line drawing corresponding to the stroke SL. When the value indicated by SX2 is smaller than the value indicated by SLX1 (S88: YES), the CPU 41 shifts the process to the determination of S92. If the value indicated by SX2 is equal to or greater than the value indicated by SLX1 (S88: NO), the CPU 41 returns the process to the second main process (see FIG. 7).

  It returns to the explanation of FIG. The CPU 41 determines whether the stroke S is the last stroke data in the stroke data file (S61). If the stroke S is not the last stroke data in the stroke data file (S61: NO), the CPU 41 updates the stroke S to stroke data arranged in the following order in the stroke data file (S62), and the stroke S in the stroke data file The processes from S52 to S61 are repeatedly executed until the final stroke data is obtained.

  If the stroke S is the last stroke data in the stroke data file (S61: YES), the CPU 41 determines the stroke data file stored in the RAM 42 as a recognition target in the handwritten character recognition process (see S64) (S63) . The CPU 41 executes handwritten character recognition processing with the stroke data file determined in the processing of S63 as a recognition target, and derives a character recognition result for the recognition target (S64). Next, the CPU 41 specifies the handwritten character string written based on the stroke data file. The CPU 41 creates an image file of an image showing the specified handwritten character string (S65). The CPU 41 controls the output of the created image file (S66). The CPU 41 returns the process to the determination of S42. The CPU 41 appropriately deletes from the RAM 42 the strokes S not inserted into the stroke data file in the process of S95 (see FIG. 9) after being excluded from the stroke data file in the process of S80 (see FIG. 9). Good.

  The CPU 41 develops the program stored in the flash ROM 43 on the RAM 42 to function as an example of a processor that executes the second main process. Note that, instead of the CPU 41, a microcomputer, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like may be used as a processor. Also, the second main processing may be distributed and processed by a plurality of electronic devices (that is, a plurality of CPUs). For example, part of the second main process may be executed by the CPU 21 of the reading device 2.

  As described above, in a scene where the reading device 2 is used by the user, as shown in FIGS. 4B and 4D, the handwritten character strings shown in the dotted line frames 201 and 203 are displayed on the paper 123 of the paper medium 100. On the other hand, after being written once, the line drawing (dotted line frames 202 and 204) constituting the reading point “,” which has been forgotten may be newly added. In the case where the handwritten character string is written without forgetting to write the reading point “,” and in the case where there is a writing failure and the reading point “,” is added to the existing character string, the arrangement of the line drawing constituting the handwritten character string is Even if they are the same, the time data associated with the corresponding stroke data is different for each line drawing. The CPU 41 of the smartphone 19 performs handwriting character recognition processing with reference to at least coordinate data included in a plurality of stroke data arrayed in the stroke data and time data associated with the plurality of stroke data arrayed in the stroke data file. Execute (S64). The result of the character recognition process performed by the data processing apparatus differs between when the line drawing is not added to the existing character string and when the line drawing is added. In particular, when a line drawing is added, there are cases where the user's intended character recognition result can not be obtained. The CPU 41 acquires LX2 which is the maximum X coordinate value indicated by the row BBOX and SX1 which is the minimum X coordinate value indicated by the SBBOX (S54). The CPU 41 determines whether the value indicated by SX1 is smaller than the value obtained by subtracting the configuration length W from LX2 (S55). That is, the CPU 41 calculates the length of one handwritten character (corresponding to the configuration length W from the end on the rightmost side of the line drawing group corresponding to one or more stroke data included in the line BBOX. ) It can be determined whether or not the line drawing has been additionally written at the position returned to the left. The CPU 41 excludes such a stroke S from the stroke data file stored in the RAM 42 (S80), and executes handwritten character recognition processing on the stroke data file from which the stroke S has been excluded (S63 and S64). Therefore, the CPU 41 can execute character recognition processing with excellent accuracy even when a line drawing is added later in the middle of the handwritten character string once written.

  The CPU 41 determines whether the value indicated by SX1 is larger than the value indicated by SLX2 (S87). In this process, the CPU 41 determines whether the SBBOX of the line drawing corresponding to the stroke S is arranged on the right side of the SLBBOX of the line drawing corresponding to the stroke SL (S87). When the value indicated by SX1 is larger than the value indicated by SLX2 (S87: YES), the CPU 41 indicates the time point when the minimum unit time is added to the time stamp of the stroke SL for the base point TS for inserting the stroke S in the list LS. To (S90). The CPU 41 repeatedly executes the processing of S86 to S92 until the stroke SL becomes the last stroke data in the list LS. When the stroke SL is the last stroke data in the list LS (S92: YES), the CPU 41 inserts the stroke S into the base point TS in the list LS (S95). For this reason, the base point TS in the list LS is determined at a position where it is assumed that the stroke S has been arranged in the list LS if handwritten characters are written without forgetting to write a line drawing. Further, the CPU 41 changes the time stamp of the stroke S to the time indicated by the base point TS (S90 and S94). For this reason, the CPU 41 can insert the stroke S into the list LS so as to arrange in time series with other stroke data in the list LS. Therefore, even when the line drawing is added to the existing character string, the CPU 41 can determine the stroke data file corresponding to the case where the handwritten character string is written without forgetting to write the line drawing as the recognition target (S63).

  The stroke data corresponding to the line drawing that intersects the line drawing forming the existing character string and is additionally written may cause erroneous recognition in the handwritten character recognition process (S64). Therefore, the CPU 41 changes the time stamp of the stroke S to the time indicated by the base point TS set in the process of S90 (S94), and inserts the stroke S into the base point TS in the list LS (S95). As a result, when a line drawing is added to the existing character string, the CPU 41 can select stroke data corresponding to the line drawing with a low possibility of erroneous recognition and add it as a recognition target in the handwritten character recognition process.

  When a plurality of ruled lines 123K are provided in the entry area 123A of the sheet 123, handwritten characters are likely to be written along the interval in which the plurality of ruled lines 123K are vertically aligned. In the sheet 123, the arrangement interval of the plurality of ruled lines 123K is set to the configuration length W. For this reason, the whole of one handwritten character written in the entry area 123A is likely to be included in a square area R whose one side is the configuration length W. Assuming that the area R is an area including the whole of one handwritten character written in the entry area 123A, the CPU 41 determines the length in the left-right direction of the area R as the configuration length W, and obtains the configuration length W To do (S45). For this reason, the CPU 41 causes the line drawing corresponding to the stroke S to be left more than the length of one handwritten character from the end on the rightmost side of the line drawing group corresponding to one or a plurality of stroke data included in the line BBOX. It can be accurately determined whether it is a line drawing that has been added to the returned position.

  In the present embodiment, the reading device 2 corresponds to the “detection device” of the present invention. The smartphone 19 corresponds to the “data processing device” of the present invention. The CPU 41 performing the process of S44 functions as the "stroke acquisition means" of the present invention. The CPU 41 that performs the process of S45 functions as the "configuration acquisition means" of the present invention. The CPU 41 performing the process of S54 functions as the "calculation means" of the present invention. The CPU 41 that makes the determination of S55 functions as the "stroke determination means" of the present invention. The CPU 41 that performs the process of S63 functions as the "object determination means" of the present invention. The CPU 41 performing the process of S64 functions as the "character recognition means" of the present invention.

  The CPU 41 performing the process of S87 functions as the "comparing means" of the present invention. The CPU 41 that performs the process of S90 functions as the "base point determination means" of the present invention. The CPU 41 that performs the process of S94 functions as the "changer" of the present invention. The CPU 41 performing the process of S95 functions as the "inserting means" of the present invention. The CPU 41 that performs the process of S88 functions as the "crossing determination means" of the present invention.

  The process of S44 corresponds to the "stroke acquisition step" of the present invention. The process of S45 corresponds to the "configuration acquisition step" of the present invention. The process of S54 corresponds to the "calculation step" of the present invention. The process of S55 corresponds to the "stroke determination step" of the present invention. The process of S63 corresponds to the "object determination step" of the present invention. The process of S64 corresponds to the "character recognition step" of the present invention. The process of S87 corresponds to the "comparison step" of the present invention. The process of S90 corresponds to the "base point determination step" of the present invention. The process of S94 corresponds to the "change step" of the present invention. The process of S95 corresponds to the "insertion step" of the present invention. The process of S88 corresponds to the "crossing determination step" of the present invention.

  In addition, this invention is not limited to said embodiment, A various change is possible. The CPU 21 detects the position of the electronic pen 3 on the sensor substrates 7L and 7R as coordinate data by an electromagnetic induction method, but the coordinate data is detected using other methods such as a resistive film method (so-called pressure sensitive method) and an electrostatic capacity method. May be detected. The size, format, material, and the like of the paper medium 100 and the paper 123 are not limited to the above embodiment.

  The CPU 41 does not necessarily have to make the determination of S88 in the return additional recording process (see FIG. 9). By performing the processing of S87, the CPU 41 can determine whether at least the SBBOX of the line drawing corresponding to the stroke S is disposed on the right side of the SLBBOX of the line drawing corresponding to the stroke SL. Thus, the CPU 41 can specify the base point TS to which the stroke S is inserted in the list LS.

  In addition, the CPU 41 may not perform the processes of S81 to S95 in the return additional recording process (see FIG. 9). The CPU 41 performs at least the processing of S80 to exclude the stroke data corresponding to the line drawing that has been added back to the existing character string from the stroke data file to be recognized in the handwriting character recognition process, thereby performing the character recognition The accuracy of the results can be improved.

  As the case where the line drawing is written at the position returned to the left from the end of the existing character string, the case of writing of the line drawing by return addition and the case of writing of the line drawing by line feed can be considered. In the above embodiment, in order to distinguish the two cases, it is determined in the line feed determination process (see FIG. 8) whether or not the stroke S is a line feed stroke. Since the line feed determination processing is an example of a method for determining whether or not the stroke S is a line feed stroke, even if another method other than the above is adopted to determine whether or not the stroke S is a line feed stroke. Good.

  In the above embodiment, the CPU 41 determines that the line drawing corresponding to the stroke S is the left side of the line drawing corresponding to the stroke SL and is the line drawing corresponding to the stroke SL in the determination of S87 and S88 in the return appending process (see FIG. 9). It is determined whether it is arranged at a position where it does not intersect. Although the intersection of the line drawings is determined based on the BBOX corresponding to each line drawing, the method of determining the intersection of the line drawings is not limited to the determination based on the BBOX. For example, in determining whether or not line drawings intersect, another method may be employed such as calculating whether or not two line drawings to be determined intersect based on coordinate data.

  In the above embodiment, the CPU 41 reads the layout data from the flash ROM 23 and acquires the configuration length W by specifying the format of the sheet 123 of the paper medium 100 loaded in the reading device 2 in the process of S11. The acquisition of the configuration length W is not limited to the above method. For example, after performing the initialization processing of S41 in FIG. 6, the CPU 41 may monitor an input operation detected via the touch panel 191, and specify the format of the sheet 123 according to the input operation. The smartphone 19 stores layout data for each format of the sheet 123 in the flash ROM 43 or the like, and can acquire the configuration length W by referring to the layout data. Further, the CPU 41 wirelessly connects with the CPU 21 so that the information indicating the format of the sheet 123 can be shared by the CPU 41 and the CPU 21.

  In the above embodiment, the CPU 41 performs handwritten character recognition processing on a stroke data file created for a handwritten character string written in horizontal writing from left to right. In addition to this, the present invention can be applied to the case where the handwritten character recognition process is performed with the stroke data file created for the handwritten character string written in the vertical writing order from the top to the bottom as a recognition target.

1 handwriting input system 2 reader 19 smart phone 21 CPU
22 RAM
23 Flash ROM
41 CPU
42 RAM
43 ROM
100 paper media 123 paper 123K ruled lines

Claims (4)

A data processing apparatus that recognizes handwritten characters written along a first direction, with information represented by coordinate data indicating the position of a writing instrument adjacent to a detection area on which a paper medium is placed in the detection apparatus. In
The coordinate data is data corresponding to one line drawing written on the paper medium by the writing instrument in proximity to the detection area, the coordinate data being a plurality of stroke data including a plurality of the coordinate data detected by the detection device. Stroke acquisition means that can be acquired in association with time data indicating a detected time point;
A configuration acquisition unit configured to acquire a configuration length defined as a length in the first direction in a region including the whole of the handwritten character of one written on the paper medium;
Among the plurality of stroke data acquired by the stroke acquiring means, first stroke data associated with the time data indicating a first time, and after the first time, and at the first time The first position indicated by the coordinate data contained in the first stroke data and the second position data contained in the second stroke data, based on the second stroke data associated with the time data indicating the second time point closest to the Calculating means for calculating the amount of change in the first direction with the second position indicated by the coordinate data;
Among the plurality of pieces of stroke data acquired by the stroke acquisition means, the variation calculated by the calculation means is larger than the configuration length, and the second position is larger than the first position. Stroke determination means for determining whether or not there is specific stroke data which is the second stroke data in a second direction which is the direction opposite to the direction;
When it is determined that the specific stroke data is present by the stroke determination means, the stroke data excluding the specific stroke data from the plurality of stroke data acquired by the stroke acquisition means is determined as the recognition target, while the stroke is determined Object determining means for determining the plurality of pieces of stroke data acquired by the stroke acquiring means as the recognition object when it is determined that the specific stroke data is not present by the judging means;
Character recognition means for recognizing a plurality of handwritten characters by at least referring to the recognition object determined by the object determination means and the time data associated with the recognition object ;
When it is determined by the stroke determination unit that there is the specific stroke data, the position in the first direction indicated by the coordinate data included in the specific stroke data, and the plurality of stroke data acquired by the stroke acquisition unit Comparing means for comparing the position in the first direction indicated by the coordinate data included in the comparison object excluding the specific stroke data from the above;
Based on the comparison result obtained by the comparison means, base point determination for determining the base point for inserting the specific stroke data into a stroke data string in which a plurality of the stroke data included in the comparison object are arranged in chronological order Means,
The time associated with the specific stroke data such that the specific stroke data is arranged in chronological order in the stroke data string in order to insert the specific stroke data at the base point determined by the base point determination means Change means for changing data,
Inserting means for inserting the specific stroke data whose time data has been changed by the changing means at the base point determined by the base point determining means;
Intersection determination means for determining whether or not the line drawing corresponding to the specific stroke data intersects with each of the line drawings corresponding to the plurality of pieces of stroke data included in the recognition target ;
The object determining means determines a stroke data string into which the specific stroke data is inserted by the inserting means as the recognition object.
The insertion unit is configured to set the specific stroke to the base point only when it is determined by the intersection determination unit that the specific stroke data does not intersect any of the line drawings corresponding to the plurality of stroke data included in the recognition target. A data processing apparatus characterized by inserting data. The paper medium comprises a sheet provided with a plurality of ruled lines extending in the first direction,
The data processing apparatus according to claim 1, wherein the configuration length acquired by the configuration acquisition unit is determined based on an interval at which the ruled line is provided. A data processing apparatus that recognizes handwritten characters written along a first direction, with information represented by coordinate data indicating the position of a writing instrument adjacent to a detection area on which a paper medium is placed in the detection apparatus. On your computer,
The coordinate data is data corresponding to one line drawing written on the paper medium by the writing instrument in proximity to the detection area, the coordinate data being a plurality of stroke data including a plurality of the coordinate data detected by the detection device. A stroke acquisition step that can be acquired in association with time data indicating a detected time point;
A configuration acquisition step of acquiring a configuration length defined as a length in the first direction in a region including the whole of the one handwritten character written on the paper medium;
First stroke data associated with the time data indicating a first time point among the plurality of stroke data acquired in the stroke acquisition step, and after the first time point and at the first time point The first position indicated by the coordinate data contained in the first stroke data and the second position data contained in the second stroke data, based on the second stroke data associated with the time data indicating the second time point closest to the Calculating the amount of change in the first direction with the second position indicated by the coordinate data;
Among the plurality of stroke data acquired in the stroke acquisition step, the change amount calculated in the calculation step is larger than the configuration length, and the second position is the first position than the first position. A stroke determining step of determining whether or not there is specific stroke data which is the second stroke data in a second direction which is the opposite direction of the directions;
If it is determined in the stroke determination step that there is the specific stroke data, the stroke data obtained by excluding the specific stroke data from the plurality of stroke data obtained in the stroke acquisition step is determined as the recognition target, while the stroke is determined An object determining step of determining the plurality of pieces of stroke data acquired in the stroke acquiring step as the recognition object when it is determined in the determining step that the specific stroke data is not present;
A character recognition step of recognizing a plurality of handwritten characters at least with reference to the recognition object determined in the object determination step and the time data associated with the recognition object ;
When it is determined in the stroke determination step that there is the specific stroke data, the position in the first direction indicated by the coordinate data included in the specific stroke data, and the plurality of stroke data acquired in the stroke acquisition step Comparing the position in the first direction indicated by the coordinate data included in the comparison object excluding the specific stroke data from the above;
Based on the comparison result obtained in the comparison step, determination of a base point to determine the base point for inserting the specific stroke data into a stroke data string in which a plurality of the stroke data included in the comparison object are arranged in chronological order Step and
The time associated with the specific stroke data such that the specific stroke data is arranged in chronological order in the stroke data string in order to insert the specific stroke data at the base point determined in the base point determination step Change step to change data,
Inserting, at the base point determined in the base point determination step, the specific stroke data whose time data has been changed in the change step;
Performing an intersection determining step of determining whether the line drawing corresponding to the specific stroke data intersects with each of the line drawings corresponding to the plurality of pieces of stroke data included in the recognition target ;
The object determining step determines a stroke data string into which the specific stroke data is inserted in the inserting step as the recognition object.
In the insertion step, the specific stroke is made at the base point only when it is determined in the intersection determination step that the specific stroke data does not intersect any of the line drawings corresponding to the plurality of stroke data included in the recognition target A data processing program characterized by inserting data. The paper medium comprises a sheet provided with a plurality of ruled lines extending in the first direction,
The data processing program according to claim 3 , wherein the configuration length acquired in the configuration acquisition step is determined based on an interval at which the ruled line is provided.

Images