JP2016024684A - Data processor and data processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a data processor and a data processing program capable of executing highly excellent precise character recognition processing even when a line drawing is added to the middle of a temporarily written handwritten character string.SOLUTION: A CPU is configured to acquire a stroke data file constituted of stroke data including coordinate data indicating the position of an electronic pen which has approached a reading device and configuration length W as the length of one side of an area including one whole handwritten character. The CPU is configured to refer to the stroke data of the stroke data file in order, and to determine whether or not a value indicated by a minimum X coordinate value SX1 in a BBOX of the referred stroke data is smaller than a value calculated by subtracting the configuration length W from a maximum X coordinate value LX2 of the BBOX of one or more referred stroke data (S55). The CPU is configured to, when the value of the SX1 is smaller than a value calculated by subtracting the configuration length W from the LX2 (S55: YES), execute return postscript processing (S56).SELECTED DRAWING: Figure 7

Description

  The present invention relates to a data processing apparatus and a data processing program for digitizing a locus of movement of a writing instrument based on an operation of writing on a paper medium.

  2. Description of the Related Art Conventionally, there is known a writing data processing apparatus that can digitize a movement locus of a writing tool as a handwriting based on the operation of the writing tool when writing with a writing tool on a pedestal. Patent Document 1 rearranges a plurality of stroke data (point sequence data of input coordinates starting with pen-down and ending with pen-up) input in the input unit in order from the smallest X coordinate value in the stroke alignment unit. Then, a writing data processing device that performs character recognition processing in the recognition unit is disclosed.

JP-A-7-14007

  As a handwritten character recognition process that recognizes multiple handwritten characters, in order to increase the accuracy of character recognition results, structural analysis is performed by focusing on the positional relationship and connection relationship of stroke data, the order of stroke data generation (stroke order), etc. The law is known. For example, in a scene where the handwritten data processing device is used by a user, after a handwritten character string is once written, a line drawing such as a punctuation mark that has been forgotten to be written is newly added at a position in the middle of the handwritten character string already written. Sometimes. In this case, since the stroke order of the line drawing added to the handwritten character string later is a recognition target, the character recognition result intended by the user may not be obtained in the handwritten character recognition process. In the handwritten data processing apparatus described in Patent Document 1, stroke data is rearranged according to position information regardless of the stroke order, so it is difficult to apply a character recognition processing method that improves the character recognition rate with the stroke order as a recognition target. There is a case.

  An object of the present invention is to provide a data processing apparatus and a data processing program capable of executing a character recognition process with excellent accuracy even if a line drawing is added later in the middle of a handwritten character string once written.

  The data processing device according to the first aspect of the present invention is directed along the first direction with information represented by coordinate data indicating the position of the writing instrument close to the detection area on which the paper medium is placed in the detection device as a recognition target. In the data processing device for recognizing handwritten characters written in the above, the data corresponding to one line drawing written on the paper medium by the writing tool close to the detection area, the data detected by the detection device A stroke acquisition unit capable of acquiring a plurality of stroke data including a plurality of coordinate data in association with time data indicating a point in time when the coordinate data is detected, and the whole of the one handwritten character written on the paper medium A configuration acquisition means for acquiring a configuration length defined as a length in the first direction in the region; and the composite acquisition acquired by the stroke acquisition means. First stroke data associated with the time data indicating the first time point, and the time indicating the second time point that is later than the first time point and closest to the first time point. Based on the second stroke data associated with the data, the first position indicated by the coordinate data included in the first stroke data and the second position indicated by the coordinate data included in the second stroke data. Of the plurality of stroke data acquired by the calculation means for calculating the change amount in the first direction and the stroke acquisition means, the change amount calculated by the calculation means is greater than the configuration length, and The specific stroke which is the second stroke data in which the second position is in a second direction which is opposite to the first direction than the first position. Stroke determination means for determining whether or not there is stroke data, and when the stroke determination means determines that the specific stroke data is present, the specific stroke data is obtained from the plurality of stroke data acquired by the stroke acquisition means. While determining the excluded object as the recognition object, when the stroke determination means determines that the specific stroke data is not present, the object to determine the plurality of stroke data acquired by the stroke acquisition means as the recognition object A determination unit; and a character recognition unit that recognizes a plurality of the handwritten characters with reference to at least the recognition target determined by the target determination unit and the time data associated with the recognition target.

  The data processing device according to the first aspect of the present invention includes a first position indicated by the coordinate data included in the first stroke data, and a first time indicated by the first time data associated with the first stroke data. A change amount in the first direction with respect to the second position indicated by the coordinate data included in the second stroke data associated with the second time data indicating the second time point is calculated. In the data processing device according to the present invention, the calculated change amount is larger than the configuration length, and the specific stroke data which is the second stroke data in which the second position is in the second direction from the first position is recognized from the recognition target. Character recognition processing can be executed with exclusion. Therefore, the data processing apparatus according to the present invention can execute a character recognition process with excellent accuracy even when a line drawing is added later in the middle of a handwritten character string once written.

  The data processing program according to the second aspect of the present invention is directed along the first direction with information represented by coordinate data indicating the position of the writing instrument close to the detection area on which the paper medium is placed in the detection device as a recognition target. Data corresponding to one line drawing written on the paper medium by the writing tool close to the detection area, and detected by the detection device. A stroke acquisition step capable of acquiring a plurality of stroke data including a plurality of the coordinate data in association with time data indicating a point in time when the coordinate data is detected, and the whole of the one handwritten character written on the paper medium A configuration acquisition step of acquiring a configuration length defined as the length in the first direction in the region including the stroke acquisition step; Among the plurality of stroke data acquired in step 1, the first stroke data associated with the time data indicating the first time point, and the first stroke data that is later than the first time point and closest to the first time point Based on the second stroke data associated with the time data indicating two time points, the first position indicated by the coordinate data included in the first stroke data and the coordinate data included in the second stroke data are indicated. Of the plurality of stroke data acquired in the stroke acquisition step, the calculation amount for calculating the amount of change in the first direction with respect to the second position, the change amount calculated in the calculation step is greater than the configuration length. And the second position is in a second direction that is opposite to the first direction than the first position. The stroke determination step for determining whether or not there is specific stroke data as second stroke data, and the plurality of pieces acquired in the stroke acquisition step when it is determined in the stroke determination step that the specific stroke data is present The stroke data excluding the specific stroke data is determined as the recognition target, and when it is determined that the specific stroke data does not exist in the stroke determination step, the plurality of strokes acquired in the stroke acquisition step Recognizing a plurality of handwritten characters by at least referring to a target determining step for determining data as the recognition target, the recognition target determined in the target determination step, and the time data associated with the recognition target Do And a character recognition step.

  In this case, the same effect as the first aspect can be obtained by the computer of the data processing apparatus executing the data processing program of the second aspect.

It is a figure which shows the outline | summary of the handwriting input system. 3 is a block diagram showing an electrical configuration of a reading device 2 and a smartphone 19. FIG. FIG. It is a figure which shows the example of the handwritten character string written in the entry area | region 123A. It is a flowchart of a 1st main process. It is a flowchart of the 2nd main process. It is a flowchart of the 2nd main process. It is a flowchart of a line feed determination process. It is a flowchart of a return postscript process.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. The outline of the handwriting input system 1 according to the present embodiment will be described with reference to FIG. In the following description, the upper left side, lower right side, upper side, lower side, upper right side, and lower left side in FIG. 1 are respectively defined as the left side, right side, front side, rear side, upper side, and lower side of the reader 2. explain.

  As shown in FIG. 1, the handwriting input system 1 mainly includes a reading device 2, an electronic pen 3, and a smartphone 19. The reading device 2 is a thin and light handwritten input device that can be folded and carried. In the handwriting input system 1, the user uses the electronic pen 3 to write a dot or a line drawing on the paper 123 of the paper medium 100 attached to the reading device 2. Below, the case where a line drawing is written is demonstrated. The line drawing includes characters, numerical values, symbols, figures, and the like. The reading device 2 acquires the position of the electronic pen 3 in the writing process. The reading device 2 creates a stroke data file based on the acquired position. The reading device 2 can transmit the created stroke data file to the smartphone 19. When the smartphone 19 receives the stroke data file, the smartphone 19 displays an image corresponding to the received stroke data file on the display 192. Thus, the user can visually recognize a line drawing having the same shape as the line drawing written on the paper 123 with the electronic pen 3 via the display 192.

  The reading device 2 mainly includes a pair of left and right left reading devices 2L, a right reading device 2R, and a cover 4. The left reading device 2L and the right reading device 2R have a rectangular thin plate shape. The left reading device 2L and the right reading device 2R are arranged on the front surface of the cover 4 so as to be spread in the left-right direction. The left reading device 2L and the right reading device 2R are electrically connected by a flat cable (not shown). The right reading device 2R includes three LEDs 5 at the upper end. The LED 5 can notify the user of the state of the reading device 2. The cover 4 includes a bag-like bag portion 4A on the left side. The left reading device 2L is detachably attached to the cover 4 by being inserted into the bag portion 4A. The right reading device 2R is attached to the right front surface of the cover 4 with a double-sided tape and an adhesive resin film.

  A paper medium 100 is detachably mounted on the front surface of the reading device 2. The paper medium 100 is a booklet that can be spread in the left-right direction. In the paper medium 100, a pair of front covers (a front cover 110L and a back cover 110R) and a plurality of papers 123 are bound at a part of each edge. As an example, the paper medium 100 is an A5 size notebook. The format indicating the layout of symbols preprinted on the paper 123 differs for each type of paper medium 100. The paper medium 100 is mounted on the reading device 2 such that the front cover 110L is placed on the front surface of the left reading device 2L and the back cover 110R is placed on the front surface of the right reading device 2R. In the present embodiment, the paper medium 100 is mounted in a state where the paper medium 100 is positioned on the reading device 2 by a double-sided tape, an adhesive resin film, or the like. That is, the left reading device 2L and the right reading device 2R move integrally with the front cover 110L and the back cover 110R, respectively. The user can write a line drawing on the paper 123 of the paper medium 100 using the electronic pen 3.

  The electronic pen 3 is a known electromagnetic induction electronic pen, and mainly includes a cylindrical body 30, a core body 31, a coil 32, capacitors 33 and 35, a substrate 34, and an ink storage unit 36. The cylindrical body 30 has a cylindrical shape, and accommodates a part of the core body 31, the coil 32, the capacitors 33 and 35, the substrate 34, and the ink storage portion 36 therein. The core body 31 is provided at the tip of the electronic pen 3. The core body 31 is urged toward the distal end side of the electronic pen 3 by an elastic member (not shown). The distal end portion of the core body 31 protrudes outside the cylindrical body 30. The rear end side of the core 31 is connected to an ink storage portion 36 that stores ink. The ink storage unit 36 supplies ink to the core body 31. When the user writes on the paper 123 using the electronic pen 3, a line drawing is formed on the paper 123 by the ink.

  The coil 32 is held between the core body 31 and the capacitor 33 while being wound around the ink storage portion 36. The capacitor 33 is fixed inside the electronic pen 3 by the substrate 34. A capacitor 35 is mounted on the substrate 34. The capacitors 33 and 35 are connected in parallel to the coil 32 and constitute a known resonance (tuning) circuit.

  The smartphone 19 includes a touch panel 191 and a display 192. The touch panel 191 is used for inputting various instructions. The display 192 can display an image corresponding to the image file. A general-purpose PC or tablet PC may be used instead of the smartphone 19.

  The electrical configuration of the handwriting input system 1 will be described with reference to FIG. First, the electrical configuration of the reading device 2 and the outline of the principle by which the reading device 2 detects coordinate data will be described. The reading device 2 includes sensor boards 7L and 7R, a main board 20, sensor control boards 28 and 29, an input unit 25, and three LEDs 5. The sensor substrates 7L and 7R are provided in the left reading device 2L and the right reading device 2R, respectively. The input unit 25 and the three LEDs 5 are provided in the right reading device 2R.

  The main board 20 includes a CPU 21, a RAM 22, a flash ROM 23, and a wireless communication unit 24. The RAM 22, the flash ROM 23, and the wireless communication unit 24 are electrically connected to the CPU 21. The CPU 21 controls the reading device 2. The RAM 22 temporarily stores various data such as calculation data. The flash ROM 23 stores a first main program that the CPU 21 executes to control the reading device 2. The flash ROM 23 stores a plurality of layout data for each format of the paper 123. The wireless communication unit 24 is a controller for executing short-range wireless communication with an external electronic device. The input unit 25 and the three LEDs 5 are electrically connected to the CPU 21. The input unit 25 is a switch for inputting an instruction to the reading device 2. The colors of the three LEDs 5 are yellow, green, and red.

  A large number of elongated loop coils are arranged in each of the left and right direction (X-axis direction) and the vertical direction (Y-axis direction) on the sensor substrates 7L and 7R. The sensor board 7L is electrically connected to the ASIC 28A of the sensor control board 28. The ASIC 28A detects coordinate data indicating the position of the electronic pen 3 when a writing operation by the electronic pen 3 is performed on the sensor substrate 7L. The sensor board 7R is electrically connected to the ASIC 29A of the sensor control board 29. The ASIC 29A detects coordinate data indicating the position of the electronic pen 3 when a writing operation by the electronic pen 3 is performed on the sensor substrate 7R. Of the ASICs 28A and 29A, the ASIC 28A on the master side is directly connected to the CPU 21, and the ASIC 29A on the slave side is connected to the CPU 21 via the ASIC 28A.

  The principle by which coordinate data is detected when a writing operation with the electronic pen 3 is performed on the sensor substrates 7L and 7R will be schematically described. The CPU 21 controls the ASICs 28A and 29A to flow a current (excitation transmission current) having a specific frequency to each of the loop coils of the sensor substrates 7L and 7R. Thereby, a magnetic field is generated from each loop coil of the sensor substrates 7L and 7R. In this state, for example, when the user performs an operation of writing a line drawing on the paper 123 of the paper medium 100 attached to the reading device 2 using the electronic pen 3, the electronic pen 3 comes close to the sensor substrates 7L and 7R. Therefore, the resonance circuit of the electronic pen 3 resonates by electromagnetic induction and generates an induction magnetic field.

  Next, the CPU 21 controls the ASICs 28A and 29A to stop the generation of magnetic fields from the loop coils of the sensor substrates 7L and 7R. Each of the loop coils of the sensor boards 7L and 7R receives an induced magnetic field generated from the resonance circuit of the electronic pen 3. The CPU 21 controls the ASICs 28A and 29A to detect signal currents (reception currents) flowing through the loop coils of the sensor boards 7L and 7R. The ASICs 28A and 29A execute this operation one by one for all the loop coils, and detect the received current. The CPU 21 detects coordinate data indicating the position of the electronic pen 3 by detecting, as coordinate data, the position in the X-axis direction and the Y-axis direction of the loop coil that has detected the received current. In the present embodiment, the origin of the coordinate data is the upper left position of the left reading device 2L and the right reading device 2R. The coordinate data is indicated by an X coordinate value or a Y coordinate value that is larger as the position is further away from the origin in the right direction or downward direction with the origin as (X, Y) = (0, 0). The X coordinate value and the Y coordinate value are expressed with the number of loop coils as one unit. When the paper medium 100 is loaded into the reading device 2, the upper left corner of the paper 123 is positioned at the upper left position of the left reading device 2L and the right reading device 2R, which are the origin positions of the coordinate data.

  In a state where a line drawing is written on the paper 123 using the electronic pen 3, writing pressure is applied to the core 31. The inductance of the coil 32 changes according to the writing pressure applied to the core body 31. Accordingly, the resonance frequency of the resonance circuit of the electronic pen 3 changes according to the writing pressure applied to the core body 31. The CPU 21 detects a change in the resonance frequency (phase change) and specifies the writing pressure applied to the electronic pen 3. The CPU 21 can determine whether or not a line drawing is written on the paper 123 of the paper medium 100 based on the specified writing pressure.

  Next, the electrical configuration of the smartphone 19 will be described. The smartphone 19 mainly includes a CPU 41, a RAM 42, a flash ROM 43, a wireless communication unit 44, an input circuit 45, an output circuit 46, a touch panel 191, and a display 192. The CPU 41 controls the smartphone 19. The CPU 41 is electrically connected to the RAM 42, the flash ROM 43, the wireless communication unit 44, the input circuit 45, and the output circuit 46.

  The RAM 42 stores various temporary data. The wireless communication unit 44 is a controller for executing short-range wireless communication with an external electronic device. The input circuit 45 performs control to send an instruction from the touch panel 191 to the CPU 41. The output circuit 46 performs control to display an image on the display 192 in accordance with an instruction from the CPU 41. The flash ROM 43 stores a program executed by the CPU 41. The smartphone 19 includes a medium reading device (for example, a memory card slot) (not shown). The smartphone 19 can read a program stored in a storage medium (for example, a memory card) with a medium reader and install it in the flash ROM 43. Alternatively, the program may be received from an external device (not shown) connected to the smartphone 19 or a network and installed in the flash ROM 43.

  The paper 123 of the paper medium 100 will be described with reference to FIG. The left side, right side, upper side, and lower side in FIG. 3 are defined as the left side, right side, upper side, and lower side of the sheet 123, respectively. FIG. 3 shows a sheet of paper 123 for one page of the paper medium 100. As shown in FIG. 3, the paper 123 is a memo paper for writing a memo. The sheet 123 has an entry area 123A and a check box 123B. In the entry area 123A, a plurality of ruled lines 123K extending in the left-right direction are printed at equal intervals in the up-down direction. The interval between the plurality of ruled lines 123K is set to the configuration length W. Since a plurality of ruled lines 123K are provided in the entry area 123A, when a user draws a line drawing in the entry area 123A, the user often enters a line drawing along the ruled line 123K. For this reason, the whole of one handwritten character written in the entry area 123A is highly likely to be included in a square area R having a side length of the constituent length W that is an interval between the plurality of ruled lines 123K. In the present embodiment, various determinations are made based on the size of the region R in a second main process (see FIGS. 6 and 7) described later.

  The flash ROM 23 is information corresponding to the format of the sheet 123, and can specify the position of each of the entry area 123A and the check box 123B of the sheet 123 and the configuration length W that is the length of one side of the area R. Memorize layout data. The CPU 21 refers to the layout data stored in the flash ROM 23 to acquire the configuration length W, and transmits the acquired configuration length W to the smartphone 19. Note that the region R in FIG. 3 is illustrated for explaining the size occupied by the region R, and the square region indicating the region R is not provided on the actual paper 123. The entry area 123A is an area for the user to write a memo, such as a handwritten character string shown in the dotted frame 201, for example. The check box 123B is printed at the lower right of the entry area 123A. The shape of the check box 123B is a rectangular single line. The check box 123B is an area where the user writes a line drawing in order to confirm the line drawing written in the entry area 123A.

  With reference to FIG. 3, an outline of processing when the CPU 21 of the reading device 2 creates a stroke data file based on the line drawing written on the paper 123 will be described. While the writing pressure is applied to the electronic pen 3, the CPU 21 of the reading device 2 repeatedly acquires coordinate data indicating the position of the electronic pen 3 through the ASICs 28A and 29A at a constant period. The CPU 21 associates time data indicating the time when the coordinate data is acquired with each of the plurality of acquired coordinate data, and stores the time data in the first area of the RAM 22. The plurality of coordinate data and the plurality of time data stored in the first area of the RAM 22 at the time when the application of the writing pressure to the electronic pen 3 is completed is one of the line drawings written on the paper 123 by the electronic pen 3. Indicates the position of the line drawing. Hereinafter, a plurality of coordinate data indicating the position of one line drawing and a plurality of time data associated with each of the plurality of coordinate data are collectively referred to as stroke data. In the present embodiment, the stroke data includes, as header information, time data associated with coordinate data acquired first among a plurality of coordinate data included in each stroke data. Hereinafter, the time data included in the header information is referred to as a time stamp of each stroke data. Hereinafter, the paper 123 will be described as an example.

  Based on the stroke data stored in the first area of the RAM 22, the CPU 21 determines which of the entry area 123 </ b> A and the check box 123 </ b> B of the sheet 123 has the line drawing written. When the CPU 21 determines that the line drawing has been written in the entry area 123A, the CPU 21 stores the stroke data stored in the first area of the RAM 22 in the second area of the RAM 22 and clears the first area of the RAM 22. Each time the user writes a line drawing one by one in the entry area 123 </ b> A using the electronic pen 3, the stroke data is sequentially stored in the second area of the RAM 22.

  When the CPU 21 determines that the line drawing is written in the check box 123B, the CPU 21 creates a stroke data file including at least one stroke data stored in the second area of the RAM 22. The stroke data file includes at least one stroke data acquired after a line drawing is written in the check box 123B until a line drawing is next written in the check box 123B. In this way, in the stroke data file, the plurality of stroke data are arranged in the order of the time stamp of each stroke data. The CPU 21 stores the created stroke data file in the flash ROM 23.

  An outline of processing when the CPU 41 of the smartphone 19 acquires a stroke data file from the reading device 2 and creates an image file will be described. The CPU 41 of the smartphone 19 transmits a data request command (described later) requesting transmission of a stroke data file or the like to the reading device 2. When the CPU 21 of the reading device 2 receives a data request command from the smartphone 19, the CPU 21 transmits the stroke data file stored in the flash ROM 23 to the smartphone 19. The CPU 41 of the smartphone 19 receives the stroke data file transmitted from the reading device 2 and stores it in the RAM 42.

  The CPU 41 extracts at least one stroke data included in the stroke data file stored in the RAM 42 one by one. The CPU 41 connects the plurality of positions indicated by the plurality of coordinate data corresponding to the extracted stroke data with straight lines in the order in which each of the plurality of coordinate data is acquired. The CPU 41 combines the line drawings obtained one by one for each stroke data, and specifies as a plurality of line drawings written on the paper 123. The CPU 41 creates an image file of an image including only the vicinity of the specified plurality of line drawings. The image file is a data file indicating a line drawing as a digital image. Examples of digital images include vector images and raster images. Examples of the image file include a JPEG file, a GIF file, a PNG file, and a BMP file. The CPU 41 displays an image on the display 192 based on the created image file.

  Further, the CPU 41 executes handwritten character recognition processing for recognizing a plurality of handwritten characters using the stroke data file stored in the RAM 42 as a recognition target. A well-known method can be adopted for the handwritten character recognition process. In the present embodiment, stroke data is extracted from a stroke data file created for a handwritten character string written in horizontal writing in order from left to right, and the mutual positional relationship and connection relationship of the stroke data, and each of the stroke data A structural analysis method is used in which identification is performed by paying attention to time data (stroke order of line drawings) associated with the. In addition to this, in the handwritten character recognition process, a pattern matching method for comparing a stroke data file with a model image called a template can be used. The handwritten character recognition process may employ a part of the methods including these examples, or may combine a plurality of methods including other methods. In addition, for a plurality of character candidates recognized by handwritten character recognition processing, context processing that optimizes a character string that is a recognition result in consideration of certainty as a language such as a connection relationship (context) between character information May also be performed. In this case, since the most probable character candidate is selected in consideration of the whole sentence of the handwritten character string, the accuracy of the recognition result can be improved.

  With reference to FIG. 3 and FIG. 4, the example at the time of a user writing the character string by handwriting (henceforth a handwritten character string) using the reader 2 is demonstrated. In FIG. 4, the ruled line 123K is omitted. As shown in FIG. 3, it is assumed that a handwritten character string “Today is a good weather today” composed of a dotted frame 201 and a dotted frame 202 is written in an entry area 123 </ b> A of a sheet 123 attached to the reading device 2. The handwritten character string is assumed to be completed by writing handwritten characters constituting the handwritten character string in order from left to right. However, line drawings such as punctuation marks that have been forgotten to be written may be added to a previously written handwritten character string (hereinafter referred to as an already-written character string) to complete a handwritten character string.

  A specific example will be described. As shown in FIG. 4A, it is assumed that the user first describes the handwritten character string “Today is a good weather” (see dotted line frame 201) in order from left to right. Thereafter, as shown in FIG. 4B, the user reads the punctuation mark “「 ”at the position between“ ha ”and“ good ”, which is the position returned from the end of the existing character string indicated by the dotted frame 201 to the left. , ”(See dotted line frame 202) may be added to complete the handwritten character string. When the handwritten characters constituting the handwritten character string composed of the dotted line frame 201 and the dotted line frame 202 are written in order from left to right, and the punctuation mark “,” indicated by the dotted line frame 202 with respect to the dotted line frame 201 that is an existing character string Since the time data associated with the stroke data indicating the punctuation mark “,” is particularly different from the case of the additional writing, the created stroke data file is also different. Therefore, when a line drawing is added to an existing character string, a character recognition result unintended by the user may be obtained, unlike a character recognition result obtained when handwritten character strings are written in order from left to right. is there. For example, when the handwritten character recognition process is performed faithfully with respect to the time data associated with the stroke data, the line drawing is added to the existing character string, so that the line drawing is added to the handwritten character string. May result in incorrect character recognition results for one line.

  Further, even when there is not enough space between handwritten characters as in the handwritten character string “Today is a good weather” (see dotted line frame 203) shown in FIG. 4C, FIG. As shown, a line drawing indicating a reading “,” (see dotted line frame 204) may be added. FIG. 4D shows a line drawing showing the added reading point “,” because there is not enough space between “ha” and “good” in the handwritten character string indicated by the dotted frame 203 (see the dotted frame 204). However, it shows a state where it intersects with the line drawings constituting the handwritten characters “ha” and “good”.

  The CPU 41 eliminates the stroke data corresponding to the line drawing that is added after returning from the existing character string from the stroke data file to be recognized by the handwritten character recognition process, thereby making the character recognition result intended by the user. You can get closer. In addition, the CPU 41 refers to the coordinate data included in the plurality of stroke data in the stroke data file, and corresponds to the added line drawing when the added line drawing is sequentially written along with other handwritten character strings. A base point at which the stroke data is considered to be inserted into the stroke data file can be determined. Further, the CPU 41 can change the time stamp of the stroke data to be inserted at the determined base point, and can arrange the stroke data inserted at the base point in time-series order together with other stroke data. Further, the CPU 41 can insert stroke data corresponding to the added line drawing into the stroke data file when the line drawing configuring the existing character string and the line drawing added to the existing character string do not intersect. . It may be difficult to accurately determine the line drawing added so as to intersect with the line drawing constituting the existing character string only by comparing the position of the line drawing with which position in the existing character string was intentionally added. . The CPU 41 does not insert stroke data corresponding to such a line drawing that is difficult to accurately specify the base point and the time stamp into the stroke data file, but only the stroke data that can accurately specify the base point and the time stamp. Can be inserted into. Therefore, the CPU 41 can improve the accuracy of the character recognition result for the stroke data file. Details will be described below.

  The first main process executed by the CPU 21 of the reading device 2 will be described with reference to FIG. The CPU 21 starts the first main process by operating based on a program stored in the flash ROM 23 when the power of the reading device 2 is turned on.

  First, the CPU 21 executes the following initialization process (S10). The CPU 21 clears the data stored in the RAM 22. The CPU 21 starts control of the ASICs 28A and 29A. As a result, the CPU 21 can determine whether or not a line drawing is written on the paper 123 of the paper medium 100 attached to the reading device 2 using the electronic pen 3. In addition, when the CPU 21 determines that the line drawing is being written using the electronic pen 3, the CPU 21 is in a state where the coordinate data indicating the position of the electronic pen 3 can be acquired.

  The CPU 21 specifies the format of the paper 123 of the paper medium 100 loaded in the reading device 2 (S11). Specifically, the CPU 21 specifies the format as follows. The CPU 21 turns on the red LED 5 to notify the user that the format of the paper 123 is not specified. The user writes line drawings with the electronic pen 3 at the positions of a plurality of calibration marks (not shown) printed at the corners of the paper 123 in the order corresponding to the format of the paper 123. The CPU 21 sequentially acquires a plurality of coordinate data indicating the position where the operation of writing the line drawing is performed, and specifies the position where the line drawing is written and the order in which the line drawing is written. The CPU 21 specifies the format of the paper 123 corresponding to the specified position and order. Hereinafter, the case where the sheet 123 (see FIG. 3) is loaded in the reading device 2 and the format of the sheet 123 is specified will be described as an example.

  The CPU 21 reads layout data corresponding to the specified format of the paper 123 from the flash ROM 23 and stores it in the RAM 22 (S11). The CPU 21 specifies the configuration length W based on the layout data stored in the RAM 22 in S11. Further, the CPU 21 specifies an area corresponding to the check box 123B in the area where the sheet 123 is mounted, that is, the front surface of the reading device 2, based on the layout data stored in the RAM 22 in S11. Hereinafter, the area on the front surface of the reading device 2 corresponding to the rectangular area constituting the check box 123B is referred to as a check area.

  The CPU 21 determines whether a line drawing is written on the paper 123 based on the writing pressure applied to the electronic pen 3 (S17). If the CPU 21 determines that the line drawing is written on the paper 123 (S17: YES), the CPU 21 acquires coordinate data. The CPU 21 further acquires time data indicating the time when the acquired coordinate data is detected (S24). The CPU 21 associates the acquired coordinate data and time data and stores them in the first area of the RAM 22 (S24). The CPU 21 repeats the process of storing the acquired coordinate data and time data in the first area of the RAM 22 until one line drawing with the electronic pen 3 is completed. When the writing of the line drawing by the electronic pen 3 is completed, a plurality of coordinate data and a plurality of time data corresponding to one line drawing are stored in the first area of the RAM 22 as stroke data.

  The CPU 21 determines whether or not a line drawing has been written in the check box 123B based on at least one stroke data stored in the first area of the RAM 22 in S24 (S25). Specifically, the CPU 21 determines that a line drawing has been written in the check box 123B when at least one of the plurality of coordinate data of the acquired at least one stroke data indicates a position in the check area (S25: YES) On the other hand, when all of the plurality of coordinate data indicate positions outside the check area, the CPU 21 determines that no line drawing is written in the check box 123B (S25: NO).

  If the CPU 21 determines that a line drawing is not written in the check box 123B (S25: NO), it determines that a line drawing is written in the entry area 123A. The CPU 21 stores at least one stroke data stored in the first area of the RAM 22 in the process of S24 in the second area of the RAM 22 (S29). The CPU 21 deletes at least one stroke data stored in the second area of the RAM 22 from the first area of the RAM 22 (S30). CPU21 returns a process to S17.

  On the other hand, if the CPU 21 determines that a line drawing has been written in the check box 123B (S25: YES), the CPU 21 creates a stroke data file including at least one stroke data stored in the second area of the RAM 22 (S27). The CPU 21 stores the created stroke data file in the flash ROM 23 (S27). The CPU 21 deletes at least one stroke data included in the created stroke data file from the second area of the RAM 22 (S28). CPU21 returns a process to S17.

  When the CPU 21 determines in the process of S17 that the line drawing is not written on the sheet 123 (S17: NO), the CPU 21 receives the data request command wirelessly transmitted from the smartphone 19 via the wireless communication unit 24. It is determined whether or not (S19). When determining that the data request command has not been received (S19: NO), the CPU 21 returns the process to S17. When the CPU 21 determines that the data request command has been received (S19: YES), the stroke data file stored in the flash ROM 23 by the process of S27 and the information indicating the configuration length W stored in the RAM 22 by the process of S11 are wirelessly stored. Wireless transmission is performed to the smartphone 19 via the communication unit 24 (S21). The CPU 21 deletes the stroke data file transmitted to the smartphone 19 from the flash ROM 23 (S22). CPU21 returns a process to S17.

  The CPU 21 functions as an example of a processor that executes the first main process by developing a program stored in the flash ROM 23 on the RAM 22. Instead of the CPU 21, a microcomputer, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like may be used as a processor. The first main process may be distributed by a plurality of electronic devices (that is, a plurality of CPUs). For example, a part of the first main process may be executed by the CPU 41 of the smartphone 19.

  With reference to FIG. 6 and FIG. 7, the 2nd main process performed by CPU41 of the smart phone 19 is demonstrated. The CPU 41 starts the second main process by operating based on a program stored in the flash ROM 43 when an application activation operation for wireless connection with the reading device 2 is performed via the touch panel 191. .

  As shown in FIG. 6, first, the CPU 41 executes the following initialization process (S41). The CPU 41 clears the data stored in the RAM 42. The CPU 41 transmits and receives an ID and the like to and from the reading device 2 in order to start wireless communication with the reading device 2. Next, the CPU 41 causes the display 192 to display a button for requesting acquisition of a stroke data file stored in the reading device 2 used by the user via the touch panel 191. The CPU 41 monitors input operations detected via the touch panel 191.

  When the CPU 41 determines that an operation for selecting the above button has been detected (S42: YES), the CPU 41 acquires a stroke data file from the reading device 2 as follows. The CPU 41 wirelessly transmits a data request command for requesting acquisition of a stroke data file to the reading device 2 via the wireless communication unit 44 (S43). The CPU 41 receives the stroke data file wirelessly transmitted from the reading device 2 in response to the transmitted data request command via the wireless communication unit 44, and stores the received stroke data file in the RAM 42 (S44). Further, the CPU 41 receives information indicating the configuration length W wirelessly transmitted from the reading device 2 in response to the transmitted data request command via the wireless communication unit 44, and stores the received information indicating the configuration length W in the RAM 42. Store (S45), and the process proceeds to S51 (see FIG. 7).

  As shown in FIG. 7, the CPU 41 refers to the stroke data arranged first in the stroke data file stored in the RAM 42 (stroke data having a time stamp indicating the earliest time point in the stroke data file) (S51). In the following description, the stroke data referred to by the CPU 21 for various determinations in the second main process among the stroke data included in the stroke data file is referred to as “stroke S”. In the second main process, the CPU 41 determines the positional relationship on the paper 123 of the line drawing corresponding to each stroke data based on the bounding box (BBOX) of each stroke. BBOX is a coordinate data group indicating a rectangular area circumscribing a line drawing corresponding to one or more stroke data, and the rectangular area is surrounded by a side extending along the X-axis direction and a side extending along the Y-axis direction. This area SBBOX is a BBOX indicated by coordinate data included in the stroke S. The row BBOX is a BBOX indicated by coordinate data included in one or more stroke data that the CPU 41 has already referred to in the second main process. On the sheet 123, the line drawing may be written over a plurality of lines, but the line BBOX is indicated by the coordinate data of the stroke data corresponding to the line drawing written in the same line as the line drawing corresponding to the stroke S. The CPU 41 initializes the row BBOX to SBBOX and stores it in the RAM 42. (S51). At this point, SBBOX is a BBOX based on stroke data arranged first in the stroke data file. Next, the CPU 41 executes a line feed determination process (S52).

  With reference to FIG. 8, the line feed determination process (S52, see FIG. 7) will be described. The line feed determination process is a process for determining whether or not the stroke S is stroke data corresponding to a line drawing written after a line break with respect to a handwritten character string written on the paper 123. As shown in FIG. 8, when the line feed determination process is started, the CPU 41 refers to the coordinate data included in the stroke S, identifies the SBBOX, and stores the identified SBBOX in the RAM 42 (S71). The CPU 41 refers to the SBBOX stored in the RAM 42 and acquires the minimum X coordinate value indicated by the SBBOX (S72). Further, the CPU 41 refers to the SBBOX stored in the RAM 42 and acquires the minimum Y coordinate value indicated by the SBBOX (S72). In the following description, the minimum X coordinate value and the minimum Y coordinate value indicated by SBBOX acquired in the process of S72 are SX1 and SY1, respectively. Next, the CPU 41 refers to the row BBOX stored in the RAM 42 (S73). The CPU 41 acquires the minimum X coordinate value indicated by the row BBOX. (S74). In addition, the CPU 41 acquires the maximum Y coordinate value indicated by the row BBOX (S74). In the following description, the minimum X coordinate value and the maximum Y coordinate value indicated by the row BBOX acquired in the process of S74 are LX1 and LY2, respectively.

  The CPU 41 determines whether or not the value indicated by SX1 is smaller than the value obtained by adding LX1 and the configuration length W (S75). In this process, the CPU 41 applies X to the line drawing that constitutes the handwritten character written on the left side of the line drawing group corresponding to one or more stroke data included in the row BBOX. It can be determined whether or not writing is performed at the same position in the axial direction or at a position shifted further to the left. When the value indicated by SX1 is smaller than the value obtained by adding LX1 and the configuration length W (S75: YES), the CPU 41 determines whether SY1 is greater than LY2 (S76). In this process, the CPU 41 can determine whether or not the line drawing corresponding to the stroke S has been written below the line drawing group corresponding to one or more stroke data included in the row BBOX. When SY1 shows a value larger than LY2 (S76: YES), the CPU 41 assigns an identifier for enabling the stroke S to be identified from other stroke data in the stroke data file, and performs the process in the second main process. Return to processing (S77). That is, the identifier given to the stroke S in the process of S77 is an identifier for identifying that the stroke S is stroke data corresponding to a line drawing with a line feed. In the following description, the stroke data corresponding to the line drawing after the line break is referred to as “line feed stroke”. On the other hand, when the value indicated by SX1 is equal to or greater than the value obtained by adding LX1 and the configuration length W (S75: NO), and when SY1 indicates a value equal to or less than LY2 (S76: NO), the CPU 41 Return to the two main processing.

  Returning to the description of FIG. Next, the CPU 41 determines whether or not the stroke S is a line feed stroke depending on whether or not an identifier is attached to the stroke S (S53). When the stroke S is a line feed stroke (S53: YES), the CPU 41 initializes the line BBOX to SBBOX, stores it in the RAM 42, and shifts the processing to the determination of S61. (S58).

  On the other hand, when the stroke S is not a line feed stroke (S53: NO), the CPU 41 refers to the row BBOX stored in the RAM 42 and acquires the maximum X coordinate value indicated by the row BBOX (S54). Further, the CPU 41 refers to the SBBOX stored in the RAM 42 and obtains SX1 that is the minimum X coordinate value indicated by the SBBOX (S54). In the following description, the maximum X coordinate value indicated by the row BBOX acquired in the process of S54 is LX2. The CPU 41 determines whether or not the value indicated by SX1 is smaller than the value obtained by subtracting the configuration length W from LX2 (S55). In this process, the CPU 41 determines that the line drawing corresponding to the stroke S is the length of one handwritten character from the rightmost end of the line drawing group corresponding to one or a plurality of stroke data included in the row BBOX (configuration length W It can be determined whether or not the line drawing has been additionally written at the position returned to the left. In the following description, a line drawing added at a position returning to the left more than the length of one handwritten character from the rightmost end of the line drawing group corresponding to one or more stroke data included in the line BBOX is shown. , “Returned line drawing”. When the value indicated by SX1 is smaller than the value obtained by subtracting the configuration length W from LX2 (S55: YES), the CPU 41 executes a return write process (S56) and shifts the process to the determination of S61. When the value indicated by SX1 is equal to or larger than the value obtained by subtracting the configuration length W from LX2 (S55: NO), the CPU 41 updates the row BBOX by including the coordinate data included in the SBBOX in the current row BBOX. The row BBOX is stored in the RAM 42 (S57). Thereafter, the CPU 41 proceeds to the determination of S61.

  With reference to FIG. 9, the return append process (S56, see FIG. 7) will be described. In the return appending process, when the stroke S corresponds to a line drawing that has been returned and appended, the stroke S is excluded from the stroke data file according to the positional relationship between the stroke S and other stroke data, or the stroke in the stroke data file This is a process of determining a base point TS into which S is inserted. As shown in FIG. 9, when the return appending process is started, the CPU 41 excludes the stroke S from the stroke data file stored in the RAM 42 and stores the stroke S in the RAM 42 separately from the stroke data file ( S80). The CPU 41 specifies SBBOX with reference to the coordinate data included in the stroke S, and stores the specified SBBOX in the RAM 42 (S81). The CPU 41 refers to the SBBOX stored in the RAM 42 and acquires SX1 that is the minimum X coordinate value indicated by SBBOX and SX2 that is the maximum X coordinate value indicated by SBBOX (S82).

  The CPU 41 refers to the stroke data file stored in the RAM 42, and acquires a list LS that is a list in which a plurality of stroke data included in the same row as the stroke S is arranged (S83). The list LS can be acquired by paying attention to a line feed stroke among stroke data arranged in the stroke data file. Specifically, among a plurality of stroke data arranged in the stroke data file, the stroke data arranged from the first stroke data arranged in the stroke data file to the previous line feed stroke arranged next is the same line. Is a plurality of stroke data included in. Of the plurality of stroke data arranged in the stroke data file, the stroke data arranged from one line feed stroke to the next one before the next line feed stroke is a plurality of stroke data included in the same line. In addition, among the line feed stroke data of the stroke data file, the stroke data arranged at the end of the stroke data file from the line feed stroke data having the time stamp indicating the latest time point is a plurality of stroke data included in the same line. The CPU 41 acquires a stroke data string including the stroke S among such stroke data strings as a list LS.

  The CPU 41 extracts a stroke SL which is stroke data arranged first in the list LS from the list LS acquired in the process of S83 (S84). In the following description, the stroke data referred to by the CPU 41 for various determinations in the return writing process among the stroke data included in the list LS is referred to as a stroke SL. The CPU 41 sets a base point TS indicating a time point serving as a base point at which the stroke S is inserted into the list LS at a time point when the minimum unit time is subtracted from the time stamp of the stroke SL (S85). The minimum unit time is a time equal to the time for one cycle of the cycle in which the CPU 21 of the reading device 2 repeatedly acquires coordinate data indicating the position of the electronic pen 3 via the ASICs 28A and 29A.

  The CPU 41 refers to the coordinate data included in the stroke SL, specifies the SLBBOX that is the BBOX of the stroke SL, and acquires the minimum X coordinate value and the maximum X coordinate value indicated by the SLBBOX (S86). In the following description, the minimum X coordinate value and the maximum X coordinate value indicated by SLBBOX acquired in the process of S86 are SLX1 and SLX2, respectively. The CPU 41 determines whether or not the value indicated by SX1 is larger than the value indicated by SLX2 (S87). In this process, the CPU 41 can determine whether or not the SBBOX of the line drawing corresponding to the stroke S is arranged on the right side of the SLBBOX of the line drawing corresponding to the stroke SL. When the value indicated by SX1 is larger than the value indicated by SLX2 (S87: YES), the CPU 41 updates the base point TS so as to indicate the time point when the minimum unit time is added to the time stamp of the stroke SL (S90). The CPU 41 proceeds to the determination of S92.

  Next, the CPU 41 determines whether or not the stroke SL is the last stroke data in the list LS (S92). If the stroke SL is not the last stroke data in the list LS (S92: NO), the CPU 41 updates the stroke SL to the stroke data arranged in the next order in the list LS (S93), and the stroke SL is the last stroke data in the list LS. The process from S86 to S92 is repeatedly executed until data is obtained. When the stroke SL is the last stroke data in the list LS (S92: YES), the CPU 41 changes the time stamp of the stroke S to the time indicated by the base point TS set in the process of S90 (S94). In this process, the CPU 41 can set the time point indicated by the base point TS at which the stroke S is inserted into the list LS as the time stamp of the stroke S. In the stroke data file list LS, the CPU 41 updates the stroke data file by inserting the stroke S at the base point TS, and stores it in the RAM 42 (S95). The CPU 41 returns the process to the second main process (see FIG. 7).

  On the other hand, when the value indicated by SX1 is equal to or less than the value indicated by SLX2 (S87: NO), the CPU 41 determines whether the value indicated by SX2 is smaller than the value indicated by SLX1 (S88). In this process, the CPU 41 determines whether or not the SBBOX of the line drawing corresponding to the stroke S is arranged on the left side of the SLBBOX of the line drawing corresponding to the stroke SL, that is, the line drawing corresponding to the stroke S corresponds to the stroke SL. It can be determined whether or not it is arranged on the left side of the line drawing at a position that does not intersect the line drawing corresponding to the stroke SL. When the value indicated by SX2 is smaller than the value indicated by SLX1 (S88: YES), the CPU 41 proceeds to the determination of S92. When the value indicated by SX2 is equal to or greater than the value indicated by SLX1 (S88: NO), the CPU 41 returns the process to the second main process (see FIG. 7).

  Returning to the description of FIG. The CPU 41 determines whether or not the stroke S is the last stroke data in the stroke data file (S61). When the stroke S is not the last stroke data in the stroke data file (S61: NO), the CPU 41 updates the stroke S to stroke data arranged in the following order in the stroke data file (S62), and the stroke S is updated in the stroke data file. The processes from S52 to S61 are repeatedly executed until the last stroke data is reached.

  When the stroke S is the last stroke data in the stroke data file (S61: YES), the CPU 41 determines the stroke data file stored in the RAM 42 as a recognition target in the handwritten character recognition process (see S64) (S63). . The CPU 41 executes a handwritten character recognition process using the stroke data file determined in the process of S63 as a recognition target, and derives a character recognition result for the recognition target (S64). Next, the CPU 41 specifies a written handwritten character string based on the stroke data file. The CPU 41 creates an image file of an image indicating the specified handwritten character string (S65). The CPU 41 controls output of the created image file (S66). CPU41 returns a process to judgment of S42. The CPU 41 appropriately deletes, from the RAM 42, the stroke S that was not inserted into the stroke data file in the process of S95 (see FIG. 9) after being excluded from the stroke data file in the process of S80 (see FIG. 9). Good.

  The CPU 41 functions as an example of a processor that executes the second main process by developing a program stored in the flash ROM 43 on the RAM 42. Instead of the CPU 41, a microcomputer, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or the like may be used as a processor. The second main process may be distributed by a plurality of electronic devices (that is, a plurality of CPUs). For example, a part of the second main process may be executed by the CPU 21 of the reading device 2.

  As described above, in the scene where the reading device 2 is used by the user, the handwritten character strings shown in the dotted frames 201 and 203 are applied to the paper 123 of the paper medium 100 as shown in FIGS. On the other hand, after writing once, the line drawing (dotted line frames 202 and 204) constituting the reading point “,” which has been forgotten to be written may be newly added. When the handwritten character string is written without forgetting to write the punctuation mark “,”, and when the punctuation mark “,” is added to the existing character string forgetting to write, the arrangement of the line drawing constituting the handwritten character string is Even if they are the same, the time data associated with the stroke data corresponding to each line drawing is different. The CPU 41 of the smartphone 19 performs handwritten character recognition processing with reference to at least coordinate data included in the plurality of stroke data arranged in the stroke data and time data associated with the plurality of stroke data arranged in the stroke data file. Execute (S64). The result of the character recognition processing performed by the data processing device differs depending on whether the line drawing is not added to the existing character string or not. In particular, when a line drawing is added, a character recognition result intended by the user may not be obtained. The CPU 41 obtains LX2 that is the maximum X coordinate value indicated by the row BBOX and SX1 that is the minimum X coordinate value indicated by the SBBOX (S54). The CPU 41 determines whether or not the value indicated by SX1 is smaller than the value obtained by subtracting the configuration length W from LX2 (S55). That is, the CPU 41 determines that the line drawing corresponding to the stroke S is the length of one handwritten character from the rightmost end of the line drawing group corresponding to one or more stroke data included in the row BBOX (corresponding to the constituent length W). ) It can be determined whether or not the line drawing is additionally written at the position returned to the left. The CPU 41 excludes the stroke S from the stroke data file stored in the RAM 42 (S80), and executes handwritten character recognition processing on the stroke data file from which the stroke S is excluded (S63 and S64). Therefore, the CPU 41 can execute a character recognition process with excellent accuracy even when a line drawing is added later in the middle of the handwritten character string once written.

  The CPU 41 determines whether or not the value indicated by SX1 is larger than the value indicated by SLX2 (S87). In this process, the CPU 41 determines whether or not the SBBOX of the line drawing corresponding to the stroke S is arranged on the right side of the SLBBOX of the line drawing corresponding to the stroke SL (S87). When the value indicated by SX1 is larger than the value indicated by SLX2 (S87: YES), the CPU 41 indicates the base point TS at which the stroke S is inserted in the list LS, indicating the time when the minimum unit time is added to the time stamp of the stroke SL. (S90). The CPU 41 repeatedly executes the processing from S86 to S92 until the stroke SL becomes the last stroke data in the list LS. When the stroke SL is the last stroke data in the list LS (S92: YES), the CPU 41 inserts the stroke S at the base point TS in the list LS (S95). Therefore, the base point TS in the list LS is determined at a position where the stroke S is assumed to be arranged in the list LS if handwritten characters are written without forgetting to draw a line drawing. Further, the CPU 41 changes the time stamp of the stroke S to the time indicated by the base point TS (S90 and S94). For this reason, the CPU 41 can insert the stroke S into the list LS so as to be arranged in time series with other stroke data in the list LS. Therefore, even when the line drawing is added to the existing character string, the CPU 41 can determine the stroke data file corresponding to the case where the handwritten character string is written without forgetting to write the line drawing as the recognition target (S63).

  Stroke data corresponding to a line drawing added by crossing the line drawing constituting the existing character string may cause erroneous recognition in the handwritten character recognition process (S64). For this reason, the CPU 41 changes the time stamp of the stroke S to the time indicated by the base point TS set in the process of S90 (S94), and inserts the stroke S at the base point TS in the list LS (S95). Thereby, when a line drawing is added to the existing character string, the CPU 41 can select stroke data corresponding to the line drawing with a low possibility of erroneous recognition and add it to the recognition target in the handwritten character recognition processing.

  When a plurality of ruled lines 123K are provided in the entry area 123A of the paper 123, handwritten characters are easily written along the intervals in which the plurality of ruled lines 123K are arranged in the vertical direction. In the paper 123, the interval between the plurality of ruled lines 123K is set to the configuration length W. For this reason, the whole of one handwritten character written in the entry area 123A is highly likely to be included in the square area R having the constituent length W as one side length. The CPU 41 assumes that the region R is a region including the whole of one handwritten character written in the entry region 123A, determines the length in the left-right direction of the region R as the constituent length W, and acquires the constituent length W. (S45). For this reason, the CPU 41 moves the line drawing corresponding to the stroke S to the left more than the length of one handwritten character from the rightmost end of the line drawing group corresponding to one or more stroke data included in the row BBOX. It is possible to accurately determine whether or not the line drawing is added to the returned position.

  In the present embodiment, the reading device 2 corresponds to the “detection device” of the present invention. The smartphone 19 corresponds to the “data processing apparatus” of the present invention. The CPU 41 that performs the process of S44 functions as the “stroke acquisition means” of the present invention. The CPU 41 that performs the process of S45 functions as the “configuration acquisition unit” of the present invention. The CPU 41 that performs the process of S54 functions as the “calculation unit” of the present invention. The CPU 41 that performs the determination in S55 functions as the “stroke determination unit” of the present invention. The CPU 41 that performs the process of S63 functions as the “target determination unit” of the present invention. The CPU 41 that performs the process of S64 functions as the “character recognition unit” of the present invention.

  The CPU 41 that performs the process of S87 functions as the “comparison means” of the present invention. The CPU 41 that performs the process of S90 functions as a “base point determination unit” of the present invention. The CPU 41 that performs the process of S94 functions as the “changing unit” of the present invention. The CPU 41 that performs the process of S95 functions as the “insertion unit” of the present invention. The CPU 41 that performs the process of S88 functions as the “intersection determination unit” of the present invention.

  The process of S44 corresponds to the “stroke acquisition step” of the present invention. The process of S45 corresponds to the “configuration acquisition step” of the present invention. The process of S54 corresponds to the “calculation step” of the present invention. The process of S55 corresponds to the “stroke determination step” of the present invention. The process of S63 corresponds to the “target determination step” of the present invention. The process of S64 corresponds to the “character recognition step” of the present invention. The process of S87 corresponds to the “comparison step” of the present invention. The process of S90 corresponds to the “base point determination step” of the present invention. The process of S94 corresponds to the “change step” of the present invention. The process of S95 corresponds to the “insertion step” of the present invention. The process of S88 corresponds to the “intersection determination step” of the present invention.

  In addition, this invention is not limited to said embodiment, A various change is possible. The CPU 21 detects the position of the electronic pen 3 on the sensor substrates 7L and 7R as coordinate data by an electromagnetic induction method. May be detected. The size, format, material, and the like of the paper medium 100 and the paper 123 are not limited to the above embodiment.

  The CPU 41 does not necessarily have to make the determination of S88 in the return write process (see FIG. 9). The CPU 41 can determine whether at least the SBBOX of the line drawing corresponding to the stroke S is arranged on the right side of the SLBBOX of the line drawing corresponding to the stroke SL by performing the process of S87. Thereby, the CPU 41 can identify the base point TS at which the stroke S is inserted in the list LS.

  Further, the CPU 41 does not have to perform the processes of S81 to S95 in the return appending process (see FIG. 9). The CPU 41 performs the process of at least S80 to exclude the stroke data corresponding to the line drawing added back to the existing character string from the stroke data file that is the recognition target of the handwritten character recognition process and perform the character recognition. The accuracy of the result can be improved.

  As a case where a line drawing is written at a position returning from the end of the existing character string to the left, there are a case of writing a line drawing by return addition and a case of writing a line drawing by line feed. In the above embodiment, in order to distinguish between these two cases, it is determined whether or not the stroke S is a line feed stroke in the line feed determination process (see FIG. 8). Since the line feed determination process is an example of a method for determining whether or not the stroke S is a line feed stroke, another method other than the above may be employed for determining whether or not the stroke S is a line feed stroke. Good.

  In the above embodiment, the CPU 41 determines that the line drawing corresponding to the stroke S is on the left side of the line drawing corresponding to the stroke SL and the line drawing corresponding to the stroke SL in the determination in S87 and S88 of the return appending process (see FIG. 9). It is determined whether or not they are arranged at positions that do not intersect. The intersection between the line drawings is determined based on the BBOX corresponding to each line drawing, but the method for determining the intersection between the line drawings is not limited to the determination based on the BBOX. For example, when determining whether or not the line drawings intersect with each other, another method may be employed such as calculating whether or not two line drawings to be determined intersect with each other based on the coordinate data.

  In the above embodiment, the CPU 41 specifies the format of the paper 123 of the paper medium 100 loaded in the reading device 2 in the process of S11, thereby reading the layout data from the flash ROM 23 and acquiring the configuration length W. The acquisition of the configuration length W is not limited to the above method. For example, the CPU 41 may monitor the input operation detected via the touch panel 191 after performing the initialization process of S41 in FIG. 6, and may specify the format of the paper 123 according to the input operation. The smartphone 19 can acquire the configuration length W with reference to the layout data by storing the layout data for each format of the paper 123 in the flash ROM 43 or the like. The CPU 41 can share information indicating the format of the paper 123 between the CPU 41 and the CPU 21 by wirelessly connecting to the CPU 21.

  In the above embodiment, the CPU 41 performs a handwritten character recognition process using a stroke data file created for a handwritten character string written in horizontal writing in order from left to right as a recognition target. In addition, the present invention can also be applied to a case where handwritten character recognition processing is performed on a stroke data file created for a handwritten character string written vertically in order from top to bottom.

1 Handwriting input system 2 Reading device 19 Smartphone 21 CPU
22 RAM
23 Flash ROM
41 CPU
42 RAM
43 ROM
100 Paper medium 123 Paper 123K Ruled line

Claims (8)

A data processing device for recognizing handwritten characters written along the first direction using information represented by coordinate data indicating the position of a writing instrument close to a detection area on which a paper medium is placed in a detection device as a recognition target In
Data corresponding to one line drawing written on the paper medium by the writing tool close to the detection area, the coordinate data being a plurality of stroke data including a plurality of the coordinate data detected by the detection device. Stroke acquisition means that can be acquired in association with time data indicating the detected time point;
Configuration acquisition means for acquiring a configuration length defined as a length in the first direction in an area including the whole of the one handwritten character written on the paper medium;
Of the plurality of stroke data acquired by the stroke acquisition means, the first stroke data associated with the time data indicating the first time point, and the first time point after the first time point. Based on the second stroke data associated with the time data indicating the second time point closest to the first position, the first position indicated by the coordinate data included in the first stroke data and the second stroke data included in the second stroke data Calculating means for calculating a change amount in the first direction with respect to the second position indicated by the coordinate data;
Of the plurality of stroke data acquired by the stroke acquisition means, the amount of change calculated by the calculation means is greater than the configuration length, and the second position is higher than the first position. Stroke determining means for determining whether or not there is specific stroke data that is the second stroke data in the second direction that is opposite to the direction;
When it is determined by the stroke determination means that the specific stroke data is present, the recognition target is determined by excluding the specific stroke data from the plurality of stroke data acquired by the stroke acquisition means. An object determining unit that determines the plurality of stroke data acquired by the stroke acquiring unit as the recognition target when the determining unit determines that the specific stroke data is not present;
Character recognition means for recognizing a plurality of the handwritten characters by at least referring to the recognition object determined by the object determination means and the time data associated with the recognition object. Data processing device. When the stroke determining means determines that the specific stroke data is present, the position in the first direction indicated by the coordinate data included in the specific stroke data and the plurality of stroke data acquired by the stroke acquiring means Comparing means for comparing the position in the first direction indicated by the coordinate data included in the comparison target excluding the specific stroke data from
Base point determination for determining a base point for inserting the specific stroke data into a stroke data sequence in which a plurality of stroke data included in the comparison target are arranged in time series based on the comparison result obtained by the comparison unit Means,
In order to insert the specific stroke data at the base point determined by the base point determination means, the time associated with the specific stroke data so that the specific stroke data is arranged in time series in the stroke data string Change means to change the data;
Inserting means for inserting the specific stroke data whose time data has been changed by the changing means into the base point determined by the base point determining means,
The data processing apparatus according to claim 1, wherein the target determining unit determines a stroke data string into which the specific stroke data is inserted by the inserting unit as the recognition target. An intersection determining means for determining whether or not a line drawing corresponding to the specific stroke data intersects each of the line drawings corresponding to the plurality of stroke data included in the recognition target;
The insertion means only applies the specific stroke to the base point when the intersection determination means determines that the specific stroke data does not intersect any of the line drawings corresponding to the plurality of stroke data included in the recognition target. The data processing apparatus according to claim 2, wherein data is inserted. The paper medium includes a paper provided with a plurality of ruled lines extending in the first direction,
4. The data processing apparatus according to claim 1, wherein the configuration length acquired by the configuration acquisition unit is determined based on an interval at which the ruled line is provided. A data processing device for recognizing handwritten characters written along the first direction using information represented by coordinate data indicating the position of a writing instrument close to a detection area on which a paper medium is placed in a detection device as a recognition target On the computer
Data corresponding to one line drawing written on the paper medium by the writing tool close to the detection area, the coordinate data being a plurality of stroke data including a plurality of the coordinate data detected by the detection device. A stroke acquisition step that can be acquired in association with time data indicating the detected time point;
A configuration acquisition step of acquiring a configuration length defined as a length in the first direction in an area including the whole of the one handwritten character written on the paper medium;
Of the plurality of stroke data acquired in the stroke acquisition step, the first stroke data associated with the time data indicating the first time point, and the first time point after the first time point Based on the second stroke data associated with the time data indicating the second time point closest to the first position, the first position indicated by the coordinate data included in the first stroke data and the second stroke data included in the second stroke data A calculation step of calculating a change amount in the first direction with respect to the second position indicated by the coordinate data;
Of the plurality of stroke data acquired in the stroke acquisition step, the amount of change calculated in the calculation step is greater than the configuration length, and the second position is higher than the first position. A stroke determination step of determining whether or not there is specific stroke data that is the second stroke data in the second direction that is opposite to the direction;
When it is determined that the specific stroke data is present in the stroke determination step, the stroke excluding the specific stroke data from the plurality of stroke data acquired in the stroke acquisition step is determined as the recognition target, while the stroke An object determining step for determining the plurality of stroke data acquired in the stroke acquiring step as the recognition object when it is determined in the determining step that the specific stroke data is not present;
A data processing program for executing a character recognition step of recognizing a plurality of the handwritten characters with reference to at least the recognition object determined in the object determination step and the time data associated with the recognition object . When it is determined that the specific stroke data is present in the stroke determination step, the position in the first direction indicated by the coordinate data included in the specific stroke data and the plurality of stroke data acquired in the stroke acquisition step A comparison step of comparing the position in the first direction indicated by the coordinate data included in the comparison target excluding the specific stroke data from
Base point determination for determining a base point for inserting the specific stroke data into a stroke data sequence in which the plurality of stroke data included in the comparison target are arranged in time series based on the comparison result obtained in the comparison step Steps,
In order to insert the specific stroke data at the base point determined in the base point determination step, the time associated with the specific stroke data so that the specific stroke data is arranged in time series in the stroke data string A change step to change the data;
Inserting the specific stroke data in which the time data is changed in the changing step into the base point determined in the base point determining step; and
6. The data processing program according to claim 5, wherein the target determining step determines a stroke data string into which the specific stroke data is inserted in the inserting step as the recognition target. An intersection determination step of determining whether or not the line drawing corresponding to the specific stroke data intersects each of the line drawings corresponding to the plurality of stroke data included in the recognition target;
The insertion step includes the specific stroke at the base point only when it is determined in the intersection determination step that the specific stroke data does not intersect any of the line drawings corresponding to the plurality of stroke data included in the recognition target. The data processing program according to claim 6, wherein data is inserted. The paper medium includes a paper provided with a plurality of ruled lines extending in the first direction,
The data processing program according to any one of claims 5 to 7, wherein the configuration length acquired in the configuration acquisition step is determined based on an interval at which the ruled line is provided.

Images