JP2013097393A - System converting handwritten characters into text - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a system for converting handwritten characters into a text in which handwritten characters written in a note are read with high accuracy, and unauthorized use of the note can be prevented.SOLUTION: Processing to convert handwritten characters written in a note into text data can be performed with high accuracy by transmitting character image data photographed by a digital camera loaded on a terminal to a server using a communication line network, and converting the character image data into text data by a text data conversion processing part. Deletion of the handwritten characters and an alteration of a use frequency of a text data conversion processing service caused by copying of the note etc. can be prevented because the note is authenticated before the service is used using a two dimensional code added to the note.

Description

  The present invention relates to a handwritten character text conversion system, and more particularly to a handwritten character text conversion system that converts handwritten characters entered in a notebook into text data using a text data conversion processing unit of a server.

In recent years, as a technique for converting handwritten contents into electronic data, for example, handwritten characters entered in a notebook are photographed with a digital camera of a smartphone, and character image data is displayed on a screen and stored in a storage by an application installed on the smartphone. A “shot note” manufactured by King Jim Co., Ltd. has been developed (Non-Patent Document 1).
In Non-Patent Document 1, the guide marks marked at the four corners of the paper are used to correct the distortion of the character image data by the distortion correction unit mounted on the application, and a simple OCR (Optical Character Reader) mounted on the application. ) Function to optically read only the header date and reference number into text data and store them in the application storage in association with the character image data, making it easy to search for handwritten content in the notebook .

[Search on September 25, 2011], Internet <URL: http: // www. kingjim. co. jp / sp / shotnote />

However, in the “shot note” of Non-Patent Document 1, handwritten characters are displayed on the screen as character image data and stored in the application storage. Therefore, for example, when the client wants to send the minutes stored as character image data in the storage to the related parties using e-mail as a document, the application is first started, and the date and arrangement of the header converted into text data are started. Using the number as an index, it was necessary to read the character image data of the minutes on the display of a smartphone, and input the contents into a word processing software on a personal computer using a keyboard to convert it into text data.
Therefore, in order to eliminate the annoyance of manually inputting such a client, the application is equipped with a high-accuracy OCR that optically reads handwritten characters, instead of the conventional simple OCR that only reads numbers. Can be considered. However, OCR that can read handwritten characters in this way cannot be said to have sufficient reading accuracy.

Therefore, as a result of diligent research, the present inventors transmit character image data captured by a digital camera mounted on a smartphone to a center server using a communication network, where the character image data is converted into text data. I thought of what I should do. That is, if a text data conversion processing unit for character image data that uses (partial) text data as a reading result transmitted from the operator's terminal is provided on the server, high-precision text data of handwritten characters (including printed characters) It came to know that it can be processed.
However, on the server side, before the client uses this text data conversion service, it is necessary to know the size of the notebook used by the client, the number of pages, and the like. The reason is to prevent illegal use of the text data processing unit such as erasing handwritten characters or copying notes. Therefore, the inventor prints a two-dimensional code such as a QR code (registered trademark) on a notebook and uses this to cause the client (user) to activate the notebook (authentication process) in advance. The present invention has been completed by discovering that all the above-mentioned problems can be solved if it is configured so as to obtain information on the number of times the notebook has been converted into text data.

  An object of the present invention is to provide a handwritten character text conversion system capable of reading a handwritten character written in a notebook with high accuracy and preventing unauthorized use of the notebook.

  The present invention has a handwritten space for characters, a handwritten note on which a two-dimensional code for note authentication is displayed, a digital camera for photographing a handwritten character entered in the handwritten space, and the two-dimensional code. A terminal equipped with a two-dimensional code reader for reading, a terminal-side communication unit using a communication line network, and the two-dimensional code read data by the two-dimensional code reader respectively transmitted from the terminal-side communication unit; , A server on which a server-side communication unit that receives character image data of the handwritten characters by the digital camera using the communication network is mounted, and the server is provided with the two-dimensional code read By receiving data, the use of the note is authenticated, and the number of times the text data conversion processing unit uses the note A note authentication unit that acquires over data, a handwriting-to-text system comprising a text data processing unit for converting the text data by scanning the character image data.

The “notebook” here is a notebook (book, booklet, etc.) in which a plurality of papers (page paper, paper) are bound or vertically bound. Further, only one sheet of paper may be used. Examples of the note size include A7, A5, and A1. The paper surface (handwritten surface) on which paper is handwritten is, for example, the surface of the paper.
“Handwritten characters” are characters (kanji, hiragana, katakana, numbers, alphabets, etc.) written by hand using writing instruments such as pencils, mechanical pencils, ballpoint pens, magic pens, and brushes. It is different from the type.
The “handwriting space” is usually all or only the center of the handwritten surface of the paper, but may be only the upper part, only the lower part or only the outer peripheral part of the handwritten surface, for example. You may enter a horizontal rule, a vertical rule, a grid, etc. in the handwriting space.

  Further, the distortion of the photographed image on the notebook paper may be corrected by, for example, the distortion correction function of the application of the terminal. As a correction method, a trapezoidal correction method (projective transformation) or the like can be employed. Specifically, a digital camera is used to photograph the paper surface, find the four corner guide marks in the photographed image, and use the found four corner guide marks to make the photographed image rectangular by the trapezoidal correction method. Convert to paper image. At the same time, the note type specifying function of the terminal application specifies the type of note to be read (note size, note grade) based on the shape and arrangement of the guide marks at the four corners.

Here, “note authentication” is processing that is performed by an application activation function and that grants usage permission to the corresponding note.
The “two-dimensional code (two-dimensional bar code)” here is a data in which data such as characters and numbers is imprinted and printed on a paper surface as a two-dimensional graphic pattern, for example, a matrix type two-dimensional code (QR code). And a stack type two-dimensional code.
The display position of the two-dimensional code on the notebook is arbitrary. For example, it may be the front surface or back surface of the cover of a notebook, the surface of each sheet (page paper), or the surface of an authentication-dedicated sheet disposed between the cover and the first reading sheet. In this case, a description of how to use the notebook including the activation method may be provided on the back side of the authentication-dedicated paper. In particular, if a two-dimensional code is displayed on the first page (authentication-dedicated paper) that turns the cover of a notebook, the notebook will not be purchased by a third party in the store after taking a picture with a mobile phone digital camera. It is preferable from the viewpoint that it is easy to use while preventing activation of the client who purchased the client.

For example, a mobile phone (smart phone) that has the functions of a personal portable computer, a mobile phone that does not have a function of a mobile computer, a personal computer, and the like can be used as the terminal here. In addition, another personal computer designated by the client may be used.
As the digital camera, for example, a CCD camera or a CMOS camera can be employed.
The “character image data” here is handwritten character image data (still image data) obtained by photographing a handwritten character entered in a notebook with a digital camera. For example, bitmap data (Bitmap Data) can be used.

The type of two-dimensional code reader (two-dimensional bar code reader) is not limited.
As the communication network, for example, the Internet can be employed.
As the terminal-side communication unit, a communication device for wireless transmission can be employed. In addition, a communication device for wired transmission can be employed as the server-side communication unit. In general, text data is stored in a storage on the server side, and when necessary, a client accesses the server from a communication network using a terminal and uses the text data stored in the storage. Alternatively, the acquired text data may be returned from the server to the terminal.
The text data is data composed purely of characters without layout information or decoration information.

  Here, the “text data conversion processing unit” is, for example, text data obtained by two or more operators reading handwritten characters written in a handwritten space of a notebook and transmitting (inputting) the reading result from the operator terminal. A processing unit can be employed. In addition, not only the entry date (date) handwritten in the header of the notebook, the company number and the reference number written in the footer, but also OCR that optically reads handwritten characters in the handwritten space and converts them into text data may be used. Alternatively, a combination of both a processing unit with an operator and OCR may be used. When the OCR unit is provided, the determination result transmitted from the operator terminals of two or more OCR determination operators for determining whether or not the text data conversion by the OCR unit is correctly performed is stored in the text data conversion processing unit. It is desirable to provide an OCR determination unit that determines whether or not it is done. This increases the accuracy of automatic reading of handwritten characters by the OCR unit.

“Authenticating the use of a notebook by receiving the reading data of the two-dimensional code” here means that the notebook information (such as the number of papers with a handwritten space) that is patterned in the two-dimensional code is two-dimensional. It is read by a code reader and approves the use of the text data conversion processing unit by the target note. In addition to this, information on how many times the text data conversion processing can be used using the target note is acquired. It is also possible to check whether the code is actually issued to prevent forgery.
The “note authentication unit” is software that performs activation (license authentication).

  Further, according to the present invention, the text data conversion processing unit includes each of an image dividing unit that divides character image data into a plurality of partial character image data and two or more reading operators that individually read one partial character image data. Partial text determination unit for determining whether or not a plurality of partial text data as a result of reading partial character image data transmitted from the terminal one by one matches, and partial text data determined to match the partial text determination unit The text data is created by connecting the partial text data of the entire text data stored in the storage unit and the partial text data of the text data stored in the storage in accordance with an arrangement in which the character image data is divided into a plurality of partial character image data. A text creation unit, and after creating text data, the server side communication unit uses the communication line network. The person who was to return the text data to the terminal is desirable.

Here, “divide character image data into a plurality of partial character image data” means that character image data is subdivided by a predetermined number of characters and by a predetermined paper area (predetermined interval, every predetermined number of lines). Say. The number of divisions and the division size of character image data are arbitrary.
“Partial character image data” is data obtained by dividing character image data by an image dividing unit.
The “image dividing unit” is software for dividing the character image data.
“Two or more reading operators that individually read one partial character image data” refers to one selected partial character image data among a plurality of partial character image data obtained by dividing character image data. Two or more reading operators who read at their own discretion without consulting each other.

“A plurality of partial text data as a result of reading partial character image data transmitted one by one from each operator terminal” is transmitted from the operator terminals individually owned by two or more reading operators to the partial text determination unit ( It means the reading result of the input partial character image data of each operator. One reading result is obtained from each operator terminal.
The “partial text determination unit” is software that determines whether a plurality of partial text data transmitted one by one from each operator terminal match (match) or do not match. As a determination when there are three or more partial text data to be determined, for example, a determination method of whether or not all partial text data matches can be employed. In this case, the accuracy of reading partial character image data increases as the number of reading operators increases. In addition, it may be determined whether or not one set (two) or more partial text data matches except for the determination method of whether or not all the partial text data match.

As the “storage unit”, for example, various storages possessed by the host computer of the server can be employed.
The “partial text data for the entire text data stored in the storage unit” here is a partial text determination unit that is individually performed for all partial character image data corresponding to the total amount of character image data to be read. Means partial text data (all partial text data necessary for composing text data) obtained by the match determination (including deemed match determination).
Here, “partial text data for the entire text data is connected in accordance with an array in which character image data is divided into a plurality of partial character image data” means that character image data is divided into a plurality of partial character image data. By connecting all the partial text data stored in the storage unit, that is, all the partial text data determined to match (including the deemed match determination) by the partial text determination unit in the arrangement order when To create text data.
The “text creation unit” is software that creates (assembles) text data from all the partial text data provided in the host computer of the server.

  Further, the present invention provides a partial text data as a partial text determination unit that is transmitted as a result of reading from the operator terminal of the first reading operator and a part as a result of reading that is transmitted from the operator terminal of the second reading operator. A reading result transmitted from the operator terminal of the third reading operator when the determination results of the first partial determination unit and the first partial determination unit that determine whether or not the text data match are the same Whether or not the partial text data matches the partial text data transmitted from the operator terminal of the first reading operator, and is transmitted from the operator terminal of the third reading operator It is determined whether or not the partial text data The partial text data stored in the storage unit is the partial text data determined by the first partial determination unit or the first part. It is desirable to use partial text data that has been determined that the second partial determination units after determining that the determination units do not match.

The partial text data stored in the storage unit is partial text data determined by the first partial determination unit to match. Or it becomes the partial text data determined as the 2nd partial determination part in the case where it determines with a 1st partial determination part not matching.
Furthermore, if the result of the determination by the second partial determination unit does not give a match, the partial text transmitted from the operator terminal of the fourth or later (fifth, sixth,..., Nth) reading operator It is determined by the third and subsequent (fourth, fifth,..., N−1th) partial determination units whether the data and the partial text data acquired so far match. You may do it. Further, the fourth and subsequent reading operators refer to the partial text data acquired so far, and if there is partial text data to be stored in the storage unit, the corresponding partial text from the acquired partial text data. Data may be selected, and the selected partial text data may be transmitted as partial text data of the reading result of the operator from the terminal of the fourth or later reading operator.

According to the first aspect of the present invention, the two-dimensional code of the notebook is read by the two-dimensional code reader, and thereafter, the authentication information of the notebook as a result of reading the two-dimensional code and the characters written in the notebook are read using the communication network. The text data use count information is transmitted to the note authentication unit of the server. Thereby, the activation (authentication process) of the note is performed, and the number of times of use of the note as text data is also transmitted to the server side. As a result, before the client requests the text data conversion processing service, it is possible to grasp the standard of the notebook (number of pages, etc.). Therefore, for example, it is possible to prevent erasure of handwritten characters, unauthorized use of notes using copy notes, and consequently unauthorized use of a text data conversion processing service for handwritten characters by the text data conversion processing unit.
Next, character image data captured by a digital camera of a terminal (for example, a smartphone) is transmitted to a server as a character reading center using a communication network, and handwritten characters are converted by a text data conversion processing unit included in the server. Text data conversion processing is performed on the imaged character image data. Thereby, the text data conversion processing of handwritten characters can be performed with high accuracy.

  In particular, according to the second aspect of the present invention, the character image data is divided into a plurality of partial character image data by the image dividing unit, and thereafter, each partial character image data is individually separated by the partial text determining unit. It is determined whether or not a plurality of pieces of partial text data transmitted one by one from each operator terminal of two or more reading operators that match each other match. Here, the partial text data determined to be matched by the partial text determination unit is stored in the storage unit. In the same process, after the partial text data for the entire text data is stored in the storage unit, the text creation unit stores the partial text data into a plurality of partial character image data in accordance with the arrangement obtained by dividing the character image data. Create text data by connecting all the partial text data. As a result, it becomes possible to convert the text data of handwritten characters with higher accuracy than OCR. In addition, by dividing the character image data into a plurality of partial character image data by the image dividing unit, each operator cannot visually see the entire character image data. Thereby, information leakage can be prevented.

According to the third aspect of the present invention, when the partial text determination unit determines, the partial text data as the read result transmitted from the operator terminal of the first reading operator and the second reading operator The first partial determination unit determines whether or not the partial text data as a reading result transmitted from the operator terminal matches. Here, when it is determined that the first partial determination units do not match, the partial text data as the read result transmitted from the operator terminal of the third reading operator and the operator terminal of the first reading operator Whether or not the partial text data transmitted from the second partial text data and the partial text data transmitted from the operator terminal of the second reading operator match. The determination is made by the second determination unit.
After the determination by the partial text determination unit performed in such a plurality of steps, the partial text data stored in the storage unit is the partial text data determined by the first partial determination unit to match or the second part The partial text data determined to match the determination units. As a result, the number of operator terminals used can be reduced, and handwritten character text data can be processed with higher accuracy than in the case of claim 2.

BRIEF DESCRIPTION OF THE DRAWINGS It is a perspective view which shows the whole structure of the handwritten character text conversion system which concerns on Example 1 of this invention. It is a principal part block diagram of the handwritten character text conversion system which concerns on Example 1 of this invention. It is a flow sheet of note authentication of the handwritten character text conversion system concerning Example 1 of this invention. It is a whole flow sheet of handwritten character text conversion processing of the handwritten character text conversion system concerning Example 1 of this invention. It is a flow sheet of the text data conversion processing step of the handwritten character text conversion system according to Embodiment 1 of the present invention. It is an enlarged front view of the paper surface which has the handwriting space of the notebook of the handwritten character text conversion system which concerns on Example 1 of this invention. It is a flowchart of the partial text determination process of the text data conversion process of the handwritten character text conversion system which concerns on Example 1 of this invention.

  Examples of the present invention will be specifically described below.

  In FIG. 1 and FIG. 2, reference numeral 10 denotes a handwritten character text conversion system (hereinafter referred to as a text conversion system) according to the first embodiment. S, a handwritten note 15 on which a QR code (two-dimensional code) 14 for note authentication is displayed, a CMOS camera (digital camera) 16 that captures handwritten characters entered in the handwriting space S, and a QR code A two-dimensional code reader 17 that reads 14, a terminal-side communication unit 19 that uses the Internet (communication network) 18, and a terminal (here, a smartphone) 21 that includes an application 20 for text data conversion processing, and a terminal QR code 14 by the two-dimensional code reader 17 respectively transmitted from the communication unit 19 on the side A server 22 that receives read data and character image data of handwritten characters by the CMOS camera 16 using the Internet 18, a server-side communication unit 23 that uses the Internet 18, and the server 22 are provided in the character image. The text data conversion processing unit 24 that reads data and converts it into text data, and is provided in the server 22 and receives the read data of the QR code 14, thereby authenticating the use of the notebook 15 and the text data generated by the notebook 15. And a note authenticating unit 25 that acquires data on the number of times the data processing unit 24 is used.

  As shown in FIG. 1 and FIG. 6, the notebook 15 has 29 sheets (20 to 50 sheets with instructions) of the same size as the terminal 21 (60 mm in width and 115 mm in length) that are bound vertically. This is a booklet in which this is sandwiched between a thick cover 12 and a back cover 13. Therefore, the maximum use count of the text data conversion processing service is 29 times. In addition, between the cover 12 and the first sheet 11, an authentication-dedicated paper 12A in which a two-dimensional code is displayed on the front surface and usage of the notebook including note authentication is described on the back surface is inserted. . In addition, a company name and a logo of a partner company (not shown) are printed at the center of the surface of the cover 12.

  An 8-digit rectangular small frame is printed on the upper end of the surface of each paper 11 in order to handwritten the entry date (date entered) 26 corresponding to the header. Further, a footer code (classification code) 27 of a 6-digit number (which may be an alphabetic character) corresponding to a footer is printed on the lower end portion of the surface of each paper 11. The footer code 27 includes an advertisement (company) code from the left to the third digit (for example, 000; no advertisement, 001: first house advertisement, 002; second house advertisement, etc.), and the fourth digit from the left is a function code ( For example, 0; no function, 1; calendar, 2; tag, calendar, etc.) The fifth digit from the left is the code of the character image data division method (for example, 0; current fixed coordinate division, 1; manual division, etc.), left The sixth digit (right end) is a check digit code which is a check digit. The check digit is calculated by adding all the digits of the advertisement code, doubling the number of the function code, multiplying the number of the divided code by 3, and adding them up to the total number of 10 The remainder of the divided answer is used as a check digit. These are stored in the storage unit 33 of the server 22.

  The handwritten entry date 26 is automatically read by an OCR unit 28 (to be described later), and the type footer code 27 is automatically read by an OCR unit (read by an operator if it cannot be recognized) 52 on the terminal side. When the determination date is used to read the entry date 26 handwritten in the header and the type of footer code 27, the characters entered in these areas may be read simultaneously. Further, guide marks (classification marks) 29 for distortion correction and note information acquisition are printed at the four corners of the surface of each paper 11. When the guide marks 29 at the four corners are photographed by the camera, the distortion of the character image data at the time of camera photographing is corrected by the distortion correcting unit 53 installed in the application 20 described later. In addition, the application 20 is used to read information such as the note size and note grade (text data provision speed and quality may be changed) from the shape of the guide mark 29 and the arrangement at the four corners.

  As shown in Table 1, the guide mark 29 includes six types (horizontal alignment) for the header (left corner) and six types (vertical alignment) for the footer (left corner). The six types of headers are: 1; a combination of two squares close to each other up and down; 2; a combination of one square and one long rectangle close to each other up and down; 3; two squares 4; a combination of two slightly oblong rectangles close to the left and right; 5; a combination of one square and one oblong rectangle close to the left and right; 6; two squares Combinations separated from each other can be mentioned. In addition, the six types for the footer are: 1; a combination in which two squares are close to each other up and down; 2; a combination in which one vertical rectangle and one square are close to each other in a vertical direction; 3; two squares 4; a combination of two slightly horizontal squares close to the left and right 5; a combination of one square and one horizontal rectangle close to the left and right 6; two squares Is a combination separated from side to side. The data in Table 1 is stored in advance in the storage unit 33 of the server 22.

  Further, a handwritten space S is formed in a portion excluding the outer peripheral portion of the surface of each paper 11. A specific formation position of the handwriting space S is an area between the entry date 26 and the footer code 27 on the surface of each paper 11.

Such a handwriting space S includes an uppermost title entry field 30 and six partial character image data display fields 31 in which the area below the uppermost part is equally divided by a total of eight horizontal rules (solid lines) a. It is comprised by. The horizontal rule a differs depending on the note size. Further, a calendar cooperation time description column, a tag description column, and the like may be provided. Each partial character image data display column 31 is written in two lines, one line above and below, with a horizontal ruled line b of a broken line arranged in parallel in the middle part in the width direction. Although it is normal for handwritten characters to be entered in two upper and lower lines in each partial character image data display column 31, the handwritten characters are large characters that straddle the broken horizontal ruled line b and are displayed in one partial character image data display column 31. One line may be written.
Also, one two-dimensional code 14 for note authentication is printed at the center of the surface of the authentication-dedicated paper 12A of the note 15. On the server 22 side, the two-dimensional code 14 is read by the two-dimensional code reader 17 to cause the client to activate the note 15 (authentication process) in advance. In addition, information on the number of times the purchased notebook 15 is converted into text data is acquired.

Next, the client terminal 21 will be described in detail with reference to FIG. 1 and FIG.
The terminal 21 is a smartphone that is a type of mobile phone, and is connected to the CPU 50 with respect to the CPU 50, the communication unit 19 on the terminal side, the application 20 dedicated to text data conversion installed on the terminal 21 from the net store, and the back surface. A CMOS camera 16, which is a kind of digital camera embedded in the upper part, a two-dimensional code reader 17 that reads the two-dimensional code 14, and a storage unit 32 are electrically connected to each other. The application 20 includes a distortion correction unit 53 for correcting distortion of a photographed image of the note paper imaged by the CMOS camera 16 and an OCR unit 52 that reads the footer code 27. The distortion correction unit 53 uses the plurality of guide marks 29 displayed at the four corners of the sheet 11 of each sheet 11 having the handwriting space S, and corrects distortion of the captured image on the sheet by a trapezoidal correction method (projection conversion). It is. The OCR unit 52 is simple enough to read printed numbers.

Next, the configuration of the server 22 will be described in detail with reference to FIGS. 2 and 7.
The server 22 has a server communication unit 23, a text data conversion processing unit 24 that converts handwritten character image data into text data, and a note authentication unit 25 that authenticates the notebook 15 purchased by the client. Are electrically connected.

  Among these, as shown in FIG. 2, the text data conversion processing unit 24 includes an image dividing unit 34 that divides the character image data uploaded from the terminal 21 to the server 22 into seven partial character image data including the title. Two persons who judge whether or not the text data conversion of the OCR unit 28 that optically reads the header entry date 26 and the handwritten text, the OCR unit 28 of the server 22 and the OCR unit 52 of the terminal 21 has been performed correctly. It is determined whether or not the text data of the OCR unit 28 of the server 22 and the OCR unit 52 of the terminal 21 match each other as a determination result transmitted from the operator terminals 21α and 21β of the OCR determination operators α and β. Each of the operator terminals 21A-2 of the two or more reading operators that individually read one partial character image data. A partial text determination unit 40 for determining whether or not a plurality of partial text data as a result of reading partial character image data transmitted one by one from E (FIG. 7) match, and this partial text determination unit 40 The storage unit 33 that stores the partial text data determined to match, and the partial text data for the entire text data stored in the storage unit 33 are arranged in accordance with an arrangement in which character image data is divided into a total of seven partial character image data. Thus, a text creation unit 41 that creates text data is provided. The created text data is stored in the storage unit 33 of the server 22, and when necessary, the client accesses the server 22 from the terminal 21 or a personal computer to acquire the text data. At this time, the OCR determination operator or the reading operator may perform work by accessing the server 22 using his / her terminal at a convenient time while staying at home, instead of being an operator at normal work. The number of partial character image data is not limited to seven (for example, 11 may be sufficient).

Here, a specific configuration of the partial text determination unit in which data entry is performed will be described with reference to the flowchart of FIG.
The partial text determination unit 40 includes partial text data as a reading result transmitted from the operator terminal 21A of the first reading operator A and partial text as a reading result transmitted from the operator terminal 21B of the second reading operator B. Transmitted from the operator terminal 21C of the third reading operator C when the determination results of the first partial determination unit 36 and the first partial determination unit 36 that determine whether or not the data match are the same. Whether or not the partial text data as the read result matches the partial text data transmitted from the operator terminal 21A of the first reading operator A, and the third partial text data and the second reading operator Partial text data transmitted from the operator terminal 21B of B If the determination result by the second partial determination unit 37 indicates that they are not consistent with each other, the result is 4 The partial text data as a reading result transmitted from the operator terminal 21D of the human reading operator D to the storage unit 33 of the server 22 is regarded as a matching determination result in the partial text determination unit 40.

  Therefore, the partial text data stored in the storage unit 33 is the partial text data determined by the first partial determination unit 36 to match or after the first partial determination unit 36 determines that they do not match. From the operator terminal 21D of the fourth reading operator D after determining that the second partial determination unit 37 matches the partial text data determined to match or the second partial determination unit 37 does not match, the server 22 Any of the partial text data transmitted to the storage unit 33. That is, the reading operator D can browse the reading results of the reading operators A, B, and C, and can transmit the reading results from the operator terminal 21D to the storage unit 33 at his / her own judgment.

Next, with reference to FIGS. 1 to 7, a text data conversion processing method for handwritten characters entered in the note 15 by the client by the text conversion system 10 according to the first embodiment of the present invention will be described.
First, the client connects the terminal 21, which is a smartphone, to the Internet 18, installs an application 20 dedicated to text data conversion from a net store, and purchases a notebook 15 from a net shop or the like.
Next, before using the text data conversion processing service by the text conversion system 10, the client rolls the cover 12 of the notebook 15 and the two-dimensional code 14 of the authentication dedicated paper 12A as shown in the flow sheet of FIG. The code information is obtained by reading with a two-dimensional (QR) code reader 17. Then, using the Internet 18 from the communication unit 19 on the terminal side, the authentication information of the note 15 including the text data conversion usage number information (usage number 29 times) as a result of reading the two-dimensional code 14 is transmitted to the server 22. These are received by the server side communication unit 23 and transmitted to the note authentication unit 25. Thereafter, the note authentication unit 25 authenticates the note 15, and after the authentication is completed, an authentication completion notification is returned from the note authentication unit 25 to the terminal 21. As a result, the activation (authentication process) of the note 15 is performed, and the text data use count information of the note 15 is transmitted to the server 22. As a result, for example, it is possible to prevent erasure of handwritten characters, unauthorized use of the notebook 15 using a copy note, and consequently unauthorized use of this text data conversion processing service.

Next, referring to the respective flow sheets of FIGS. 4, 5 and 7 and FIGS. 1 and 6, a text data conversion processing method for handwritten characters entered in a note 15 performed using the text conversion system 10 will be described. explain.
First, as shown in FIGS. 1 and 6, the authentication-dedicated paper 12A of the notebook 15 is turned, and the entry date 26 is shown in the eight small frames of the header on the surface of the paper 11, for example, “2011-08-06”. Handwriting with a pen. However, “20” in the Christian era is already printed, and the 6-digit “555165” is also printed in the footer code 27 in advance. Thereafter, the diary is handwritten in the handwriting space S using a pen. In the entry, the title “Purchase MV pen” is handwritten in the title entry field 30 and the diary is handwritten in two lines per line in the partial character image data display field 31 from the top three. Specifically, “smart analog memo → iphone → KYBER” is handwritten in the upper and lower two rows in the uppermost partial character image data display column 31, and the partial character image data display column 31 in the second row from the top. Handwritten “Or Evernote can be saved on the ground” in the upper and lower two rows, and handwritten “I am struggling ...” only in the upper row of the partial character image data display column 31 in the third row from the top. .

  Thereafter, as shown in the flow sheet of FIG. 4, the entire surface of the handwritten paper 11 is photographed by the CMOS camera 16 with the application 20 of the terminal (smartphone) 21 activated. The photographed character image data uses guide marks 29 printed at the four corners of the surface of the paper 11, and a distortion correction unit (distortion correction function) 53 installed in the application 20 corrects distortion at the time of shooting with the camera 16. Is done. That is, the distortion correction unit 53 searches the character image data for the guide marks 29 displayed at the four corners of the paper 11 having the handwriting space S, and uses these to make known the distortion of the photographed image on the paper. Correct by the keystone correction method (projective transformation). Further, the footer code 27 is read by the OCR unit 52 mounted on the application 20.

The corrected character image data, the read data of the guide marks 29 at the four corners processed by the application 20 and the read data of the footer code 27 are transmitted from the terminal side transmission unit 19 to the server as a character text conversion center using the Internet 18. 22, the server-side communication unit 23 receives this and transmits it to the text data conversion processing unit 24. Here, it is first determined whether or not the number of times the text data conversion processing service is used is within the prescribed number (29 times). If the number is within the prescribed number, the process proceeds to the next text data conversion processing step. On the other hand, if the specified number of times is exceeded, a notification of exceeding the specified number of times is sent to the application 20 of the terminal 21 via the server-side communication unit 23, the Internet 18, and the terminal-side communication unit 19 in order. That the number of times has been exceeded is displayed on the liquid crystal screen of the terminal 21.
Next, the text data conversion processing unit 24 converts the character image data into text data. The obtained text data is stored in the storage unit 33 of the server 22. By doing so, the text data can be extracted from the storage unit 33 of the server 22 when accessed from the client terminal 21. The text data may be transmitted from the communication unit 23 on the server side to the communication unit 19 on the terminal side, displayed on the liquid crystal screen of the terminal 21, and stored in the storage unit (memory) 32 of the terminal 21. .

Here, the text data conversion processing step by the text data conversion processing unit 24 will be described in detail with reference to the flow sheets of FIGS.
In the text data conversion processing unit 24, first, the image dividing unit 34 divides the character image data into a header portion, a footer portion, a title portion, and a body portion of handwritten characters. Among these, the entry date 26 and the body part handwritten in the header part are optically read by the OCR unit 28 and converted into text data. Since the footer code 27 is read in advance by the OCR unit 52 on the terminal side, each partner company linked to the URL of the partner company on the liquid crystal screen of the terminal 21 through the Internet 18 based on the determination result by the OCR determination unit 35. Display the company name or logo. As a result, not only analog advertisement on the surface of the cover 12 but also digital advertisement effect can be obtained at the same time.

  Based on the read data of the guide marks 29 at the four corners, the application 20 recognizes the paper size and grade of the notebook 15 (the text data provision speed and quality may be changed). That is, combinations of the shape and arrangement of the plurality of guide marks 29 printed at the left corner of the header and the shape and arrangement of the plurality of guide marks 29 printed at the left corner of the footer are classified into the guide marks 29 shown in Table 1. The paper size and grade of the notebook 15 can be recognized by checking against the table. For example, in the classification table of Table 1, if the combination for the header is “1” and the footer is “3”, it can be seen that the combination is for the iPhone (ordinary price). reference). Based on the data transmitted from the terminal 21, the character image data is divided into partial character image data.

  Next, the advertisement code, the function code, and the character image data are acquired from the read data of the footer code 27 consisting of a 6-digit number transmitted from the terminal 21. The footer code 27 includes an advertisement (company) code (000; no advertisement, 001; first house advertisement, 002; second house advertisement) from the left to the third digit, and a function code (0; No function, 1; calendar, 2; tag, calendar), the fifth digit from the left is the code of the character image data division method (0; current fixed coordinate division, 1; manual division), the sixth digit from the left (right end) ) Is a check digit code which is a check digit. The check digit is calculated by adding the number (555) of each digit of the advertisement code (5 + 5 + 5 = 15), doubling the number (1) of the function code (1 × 2 = 2), and the number of divided codes (6 ) Is multiplied by 3 (6 × 3 = 18), and the total number (15 + 2 + 18 = 35) obtained by adding these is divided by 10 (35 ÷ 10 = 3... 05) “5” is the check digit. To do.

  Thereafter, as shown in the flow sheets of FIGS. 5 and 7, whether the date of entry 26 by the OCR unit 28, the footer code 27 by the OCR unit 52 on the terminal side, and the body text of the handwritten characters have been correctly converted into text data. The two OCR determination operators α and β determine whether or not. After that, whether the entry date 26 and the footer code 27 as the determination results transmitted one by one from the operator terminals 21α and 21β of the two OCR determination operators α and β match the text data of the handwritten character body. The OCR determination unit 35 determines whether or not, and if they match, a check by a dedicated operator E described later is performed. If they do not match, one selected partial character image A data entry type reading process is performed on the data by interposing the first to fourth operators A to D and the operator E dedicated to checking.

  Specifically, partial text data as a reading result transmitted from the operator terminal 21A of the first reading operator A, and partial text data as a reading result transmitted from the operator terminal 21B of the second reading operator B Are determined by the first partial determination unit 36. If they match, the common partial text data is transmitted to the operator terminal 21E of the operator E dedicated to checking. On the other hand, if they do not match (the mismatch rate is about 5%), the partial text data as the reading result transmitted from the operator terminal 21C of the third reading operator C and the operator terminal of the first reading operator A Whether or not the partial text data transmitted from 21A matches, and whether or not the third partial text data matches the partial text data transmitted from the operator terminal 21B of the second reading operator B Is determined by the second partial determination unit 37.

If the determinations in the second partial determination unit 37 match, the common partial text data is transmitted to the operator terminal 21E dedicated to the check. On the other hand, when a determination result that none of the two matches is obtained by the determination by the second partial determination unit 37 (the mismatch rate is about 1%), the server terminal 21D of the fourth reading operator D receives the server The partial text data as a reading result transmitted to the storage unit 33 of 22 is regarded as a matching determination result in the partial text determination unit 40, and the regarded partial text data is transmitted to the operator terminal 21E dedicated to checking. The
Thus, the partial text data obtained corresponding to the selected partial character image data is sequentially stored in the storage unit 33 of the server 22. At this time, the partial text data stored in the storage unit 33 is determined to be the partial text data determined by the first partial determination unit 36 to match or the first partial determination unit 36 does not match. Later, it is determined from the operator terminal 21D of the fourth reading operator D after determining that the second partial determination unit 37 matches the partial text data or the second partial determination unit 37 does not match. One of the partial text data transmitted to the storage unit 33 of the server 22. As a result, it is possible to process the text data of handwritten characters with higher accuracy than the OCR unit 28.

Thereafter, it is determined whether or not all the partial image data has been converted to text. If it has been completed, all the partial text data for the entire character image data stored in the storage unit 33 is converted into text. The creation unit 41 creates text data by connecting the character image data to a plurality of partial character image data according to an array obtained by dividing the character image data. As a result, it is possible to process the text data of handwritten characters with higher accuracy than the OCR unit 28.
Since the text data conversion processing unit 24 described above performs the text data conversion processing of character image data, the text data conversion processing of handwritten characters can be performed with high accuracy. Moreover, by dividing the character image data into a plurality of partial character image data by the image dividing unit 34, the operators A to D cannot visually see the entire character image data. Thereby, information leakage can be prevented.

  Further, the affiliated company is specified based on the reading result of the footer code 27 by the OCR unit 52 of the terminal 21. Therefore, at the time of reading, the server 22 accesses, for example, the URL of the affiliated company through the Internet 18 and arranges the icon and banner of the affiliated company in the character image data and the text data on the application 20 of the terminal 21 or the client PC screen. Display can be made. As a result, it is possible to provide a new advertising method in which an analog corporate advertisement printed on the surface of the cover 12 of the notebook 15 is linked with a digital corporate advertisement using icons and banners on the liquid crystal screen. In addition, an advertisement of a sponsor company or a company name logo may be displayed as a splash window for activation.

  The present invention is useful as a technique for converting handwritten characters entered in a notebook into text data using a text data conversion processing unit of a server.

10 Handwritten text conversion system,
14 QR code (two-dimensional code),
15 notes,
16 CMOS camera (digital camera),
17 2D code reader,
18 Internet (communication network),
19 Communication part on the terminal side,
21 terminals,
21A-21E operator terminal,
22 servers,
23 server side communication unit 24 text data conversion processing unit,
25 Note authentication part,
33 storage unit,
34 Image segmentation unit,
40 partial text determination unit,
41 Text creation part,
A First reading operator,
B Second reading operator,
C Third reading operator,
D 4th reading operator,
S Handwritten space.

Claims (3)

A handwritten note having a handwriting space for characters and displaying a two-dimensional code for note authentication;
A terminal equipped with a digital camera that captures handwritten characters entered in the handwriting space, a two-dimensional code reader that reads the two-dimensional code, and a communication unit on the terminal side that uses a communication network;
Receives the two-dimensional code reading data by the two-dimensional code reader respectively transmitted from the communication unit on the terminal side and the character image data of the handwritten characters by the digital camera using the communication line network. A server equipped with a server-side communication unit;
A note authenticating unit that is provided in the server and that authenticates the use of the note by receiving the read data of the two-dimensional code, and obtains data on the number of times the text data conversion processing unit is used by the note; ,
A handwritten character text conversion system comprising: a text data conversion processing unit that reads the character image data and converts it into text data. The text data conversion processing unit
An image dividing unit for dividing the character image data into a plurality of partial character image data;
Whether or not a plurality of partial text data as a result of reading the partial character image data transmitted one by one from each operator terminal of two or more reading operators that individually read one partial character image data match. A partial text determination unit for determining
A storage unit for storing the partial text data determined to match the partial text determination unit;
Text for creating the text data by connecting the partial text data for the entire text data stored in the storage unit in accordance with an array obtained by dividing the character image data into the plurality of partial character image data The handwritten character text conversion system according to claim 1, further comprising a creation unit. The partial text determination unit
Whether or not the partial text data as the reading result transmitted from the operator terminal of the first reading operator matches the partial text data as the reading result transmitted from the operator terminal of the second reading operator A first partial determination unit for determining
Partial text data as a reading result transmitted from the operator terminal of the third reading operator when the determination result of the first partial determination unit is inconsistent, and transmitted from the operator terminal of the first reading operator A process for determining whether or not the partial text data matches, and whether or not the third partial text data matches the partial text data transmitted from the operator terminal of the second reading operator. 2 partial determination units,
The partial text data stored in the storage unit is the partial text data determined by the first partial determination unit to match, or after determining that the first partial determination unit does not match, The handwritten character text conversion system according to claim 2, wherein the second partial determination unit is partial text data determined to match.

Images