JP2013161143A - Character text making system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a character text making system capable of reading a character to be digitalized in a high accuracy, and besides, capable of preventing unauthorized use exceeding a use amount of a text data making processing service.SOLUTION: A character text making system can transmit character image data imaged by digital photographing means mounted on a terminal or the like to a server using a communication line network, and converts handwritten character data into text data by means of a text data making processing unit, so as to perform text data making processing of characters including a handwritten character at a high accuracy. The system performs a use authentication of a text data making processing service to a client before using the service by using an authentication code displayed on a display body, and acquires information on the number of characters capable of being subjected to digital conversion, which are bought by the client, and prevents, on the basis of the information, an unauthorized use which exceeds a use amount of a text data making processing service.

Description

  The present invention relates to a character text conversion system, and more particularly to a character text conversion system that converts, for example, handwritten characters written in a notebook into text data using a text data conversion processing unit of a server.

In recent years, as a technique for converting handwritten contents into electronic data, for example, handwritten characters entered in a notebook are photographed with a digital camera of a smartphone, and character image data is displayed on a screen and stored in a storage by an application installed on the smartphone. A “shot note” manufactured by King Jim Co., Ltd. has been developed (Non-Patent Document 1).
In Non-Patent Document 1, the guide marks marked at the four corners of the paper are used to correct the distortion of the character image data by the distortion correction unit mounted on the application, and a simple OCR (Optical Character Reader) mounted on the application. ) Function to optically read only the header date and reference number into text data and store them in the application storage in association with the character image data, making it easy to search for handwritten content in the notebook .

[Search on September 25, 2011], Internet <URL: http: // www. kingjim. co. jp / sp / shotnote />

However, in the “shot note” of Non-Patent Document 1, handwritten characters are displayed on the screen as character image data and stored in the application storage. Therefore, for example, when the client (user) wants to send the minutes recorded as character image data in the storage to the related parties using e-mail as a document, the application is first started, and the header of the text data is converted. Using the date and reference number as an index, it was necessary to read the character image data of the minutes on the display of the smartphone and input the contents into text data by inputting it into a word processing software on a personal computer.
Therefore, in order to eliminate the annoyance of manually inputting such a client, the application is equipped with a high-accuracy OCR that optically reads handwritten characters, instead of the conventional simple OCR that only reads numbers. Can be considered. However, OCR that can read handwritten characters in this way cannot be said to have sufficient reading accuracy.

Therefore, as a result of diligent research, the present inventors transmit character image data captured by digital photographing means mounted on a terminal to a center server using a communication line network, where the character image data is converted into text data. I came up with the idea of converting. That is, when a text data conversion processing unit for character image data is provided in the server, it has been found that high-precision text data conversion processing of handwritten characters (including printed characters) can be performed.
However, on the server side, before the client uses this text data conversion processing service, it is necessary to know the amount of text data conversion service usage (number of text conversion characters). The reason is, for example, to prevent unauthorized use exceeding the acquired usage amount. Therefore, the inventor displays an authentication code such as a QR code (registered trademark) on the display body and uses this to cause the client (user) to perform activation (authentication processing) in advance. The present invention has been completed by discovering that all the above-mentioned problems can be solved if it is configured to acquire character count data corresponding to the charge of the data processing service.
Further, as a result of earnest research, the present inventors mark only the text conversion area from characters including handwritten characters with a marker for color recognition, and only this marking portion is converted into text data by the text data conversion processing unit. This configuration eliminates the need for text conversion-only notebooks that were required in the past, and the necessary parts (character areas) for various documents written with characters including printed characters as well as handwritten characters. As a result, the present invention has been completed.

An object of the present invention is to provide a text-to-text system that can read characters to be digitized with high accuracy and can prevent unauthorized use exceeding the text data processing fee.
Another object of the present invention is to provide a character text conversion system that does not require a text conversion dedicated notebook or the like and can convert only necessary portions into text data for characters written in various documents. It is said.

  In the present invention, the use of a text data conversion processing service that converts text including handwritten characters into text is authenticated, and an authentication code for acquiring the number of characters that can be converted into text using the text data conversion processing service is displayed. A display, a digital image acquisition means for optically inputting characters including the handwritten characters into character image data, a code reader for reading the authentication code, and a communication unit on the terminal side using a communication line network Using the communication line network, the installed terminal, the read data of the authentication code by the code reader transmitted from the communication unit on the terminal side, and the character image data acquired by the digital image acquisition means, A server on which a server-side communication unit for receiving is mounted, and the server is provided with the server, and the authentication code is read Receiving data to authenticate the use of the text data conversion processing service and to obtain the character number data that can be converted into text, and to convert the text image data into text data by reading the character image data Is a character text conversion system.

The “character text conversion system” is a system that embodies a text data conversion service that converts text including handwritten characters into text.
“Characters including handwritten characters” refer to handwritten characters or printed characters printed by a printing machine. In addition, "handwritten characters" refers to characters (kanji, hiragana, katakana, numbers, alphabets, etc.) written manually on the document using a writing instrument such as a pencil, mechanical pencil, ballpoint pen, magic pen, or brush. It is.

  Although the thing in which the character containing a handwritten character was written is generally a document (a book and a document are included), it is not limited to this. Specific examples of documents include notebooks, memo papers, copy papers, books, magazines, flyers, posters, and the like. The material of the document is arbitrary. For example, paper, cloth, leather, wood, plastic, metal, ceramics, etc. may be used. The shape of the document is arbitrary such as a rectangular shape. Examples of the document (paper) size of the document include A7, A5, and A1. A horizontal rule, a vertical rule, a grid, and the like may be written on the document. The “notebook” here is a notebook (book, booklet, etc.) in which a plurality of sheets (page paper, paper) are bound sideways or vertically. Further, only one sheet of paper such as memo paper may be used. In a notebook or memo paper, the handwritten paper (document) is, for example, the surface of the paper constituting the notebook or the surface of the memo paper.

Further, the distortion of the captured image of characters may be corrected by, for example, the distortion correction function of the application of the terminal. As a correction method, a trapezoidal correction method (projective transformation) or the like can be employed. Specifically, the document (paper surface) is photographed by the digital image acquisition means, and the four corners of the document are searched from the photographed images by, for example, color shading, and the image is photographed by the trapezoid correction method using the found four corners. Convert the image into a rectangular, well-written document image.
“Usage authentication” here is a process that is performed by an application activation function and gives a use permission when a client uses a text data conversion service using a terminal. Further, here, the number-of-characters data that can be converted into text at the time of use authentication is also acquired.
The “character number data that can be converted into text” here is information on the maximum number of characters that can be converted into text data using the text data conversion service. As a method of acquiring the character count data, for example, a numerical value set in the server may be referred to at the time of use authentication and registered in the client account.

The “display body” here is an arbitrary object on which an authentication code is displayed at an arbitrary position on the exposed surface. Examples of the display body include exposed surfaces of various objects such as cards, text data processing service usage instructions, plates, pens including markers, packaging bags, pencil cases, rulers, erasers, and the like. The material of the display body is not limited to paper. For example, cloth, leather, wood, plastic, metal, ceramics, etc. may be used. Moreover, as a shape of a display body, sheet | seat shape, plate shape, block shape, etc. are arbitrary.
The display position of the authentication code on the display body is arbitrary. For example, when the display body is a card, it may be the front or back surface of the card cover. The card is preferably stored in a colored package (bag) from the viewpoint that it cannot be easily read before purchase and is easy to use after purchase.
Here, the “authentication code” is obtained by marking and printing data such as authentication characters and numbers on the display surface of the display body. Specific examples include encryption, (one-dimensional) barcode, two-dimensional code (two-dimensional barcode), and the like. The two-dimensional code is data such as authentication characters and numbers displayed on the display body as a two-dimensional graphic pattern. Examples thereof include a matrix type two-dimensional code (QR code) and a stack type two-dimensional code. It is done.

The “digital image acquisition means” here is a character image data obtained by optically photographing or scanning a character written on a document. A digital camera can be used for shooting, and a single scanner or a scanner of a digital multi-function peripheral can be used for scanning. As the digital camera, for example, a CCD camera or a CMOS camera can be employed. The character image data acquired by the scanner or the scanner unit of the digital multifunction device is once transmitted to the smartphone by the wireless LAN (scanner side communication unit) from the scanner or the digital multifunction device, and then taken by the digital camera of the smartphone. The process is similar to that of character image data. In addition, the character image data may be transmitted directly from the communication unit on the scanner side to the server using electronic (E) mail.
The “character image data” here is image data (still image data) of characters including handwritten characters obtained by photographing or scanning characters including handwritten characters entered in a document by a digital image acquisition means. . For example, bitmap data (Bitmap Data) can be used.

The type of code reader (two-dimensional bar code reader) is not limited.
As the communication network, for example, the Internet can be employed.
As the terminal-side communication unit, a communication device for wireless transmission can be employed. In addition, a communication device for wired transmission can be employed as the server-side communication unit. In general, text data is stored in a storage on the server side, and when necessary, a client accesses the server from a communication network using a terminal and uses the text data stored in the storage. Alternatively, the acquired text data may be returned from the server to the terminal.
The text data is data composed purely of characters without layout information or decoration information.

For example, a mobile phone (smart phone) that has the functions of a personal portable computer, a mobile phone that does not have the function of a mobile computer, a personal computer, and the like can be used as the terminal here. Alternatively, another personal computer designated by the client may be used.
The server here has a host computer, receives the read data of the authentication code and the character image data from the digital image acquisition means via the communication line network and the communication unit on the server side, and is received by the authentication unit. The text data conversion service is authenticated for use, and the text data conversion processing unit reads the character image data to convert characters including handwritten characters into text data.
The “authentication unit” is software that performs activation (license authentication). Here, data of the maximum number of characters (total number of characters) of characters including handwritten characters that can be converted into text whose conversion region is designated by “marker” is also obtained.

  As used herein, the “text data conversion processing unit” refers to a processing unit that converts two or more operators to text data by reading characters including handwritten characters and transmitting (inputting) the read results from the operator terminal. Can be adopted. In addition, for example, an OCR unit that optically reads a handwritten character and converts it into text data may be used. Alternatively, a combination of both the processing unit and the OCR unit intervening by the operator may be used. When the OCR unit is provided, the determination result transmitted from the operator terminals of two or more OCR determination operators for determining whether or not the text data conversion by the OCR unit is correctly performed is stored in the text data conversion processing unit. It is desirable to provide an OCR determination unit that determines whether or not it is done. This increases the accuracy of automatic reading of handwritten characters by the OCR unit.

The text data conversion processing unit may convert the text into all or a part of characters including handwritten characters written on the document. In some of these cases, for example, various conversion pens are used to fill the text conversion area, or to enclose the area with a line, thereby specifying the conversion area.
“Authentication of use of text data processing service by receiving read data of authentication code” here means use authentication of text data processing service encrypted or figured in the authentication code, for example Information (such as an authentication number) is read by a code reader, and the number of characters used by the text data conversion processing unit is approved. In addition to this, in order to confirm the amount of text data conversion processing purchased by the client, information on the number of characters that can be converted into text is also acquired. It is also possible to check whether the code is actually issued to prevent forgery.

In addition, the present invention recognizes a text conversion area of a character including a handwritten character from a marker for designating a text conversion area of a character including a handwritten character by marking, and a marking color provided by the marker. It is desirable to have a color recognition unit and the text data conversion processing unit read only characters including handwritten characters in the text conversion area recognized by the color recognition unit and convert them into text data.
The marker here is preferably a marker in which the filled characters can be seen through even if the marking is performed so as to fill the characters. Specifically, a fluorescent marker (fluorescent pen) using fluorescent ink is used. The color of the marker ink is arbitrary. For example, red, blue, yellow, orange, pink, etc. are arbitrary.
The text data conversion processing unit extracts the marked text conversion area and converts only this portion into text data.

As a method for recognizing and extracting a text conversion area, for example, a terminal application is provided with a color recognition unit for recognizing the ink color of a marked marker to recognize the text conversion area, and then provided in a text data processing unit. Alternatively, the text conversion area whose color has been recognized may be extracted by the conversion area extraction unit. In addition, a color recognition unit may be provided in the server.
The display body of the authentication code may be stored and sold in a marker package (packaging bag, packaging container, etc.) so that it cannot be easily seen from the outside. In this way, the text conversion processing unit can be set so that only the text conversion region marked with the color of the marker ink stored in the same package as the display body is subjected to text conversion processing. As a result, the effect of preventing unauthorized use of the text data conversion processing service can be enhanced.

Furthermore, in the present invention, it is desirable to employ a smartphone as a terminal.
A “smart phone” is a mobile phone that has the functions of a mobile computer.
Furthermore, in the present invention, the digital image acquisition means is a scanner having a scanner-side transmission unit connected to the terminal, and the scanner scans characters including handwritten characters, and obtains the obtained character image data on the scanner side. It is desirable to transmit to the terminal or server by the transmission unit.
The “scanner” here refers to, for example, scanning a document, and a photoelectric conversion sensor receives light reflected from the document and converts it into an electrical signal. As the scanner, the scanner (image scanner) alone or a scanner unit of a digital multi-function peripheral can be employed. Here, the “digital multifunction device” refers to an electronic device in which at least a copy unit, a scanner unit, and a transmission unit on the multifunction device side are combined into one unit. In addition to these functions, a digital multifunction peripheral having a printer function may be employed.

The “scanner-side transmission unit” here refers to a communication unit that transfers character image data acquired by scanning with a scanner unit to a terminal via a communication line (wireless LAN or the like), for example, or a scanner Refers to a communication unit that transfers data to the server via e-mail. Among these, the character image data transmitted to the terminal is transmitted from the transmission unit on the terminal side to the server via the communication line network. When character image data is directly transmitted from the scanner to the server, it is necessary to employ a server having a color recognition unit or a scanner having a color recognition unit.
Further, according to the present invention, the text data conversion processing unit includes each of an image dividing unit that divides character image data into a plurality of partial character image data and two or more reading operators that individually read one partial character image data. Partial text determination unit for determining whether or not a plurality of partial text data as a result of reading partial character image data transmitted from the terminal one by one matches, and partial text data determined to match the partial text determination unit The text data is created by connecting the partial text data of the entire text data stored in the storage unit and the partial text data of the text data stored in the storage in accordance with an arrangement in which the character image data is divided into a plurality of partial character image data. A text creation unit, and after creating text data, the server side communication unit uses the communication line network. The person who was to return the text data to the terminal is desirable.

Here, “divide character image data into a plurality of partial character image data” means that character image data is subdivided by a predetermined number of characters and by a predetermined paper area (predetermined interval, every predetermined number of lines). Say. The number of divisions and the division size of character image data are arbitrary. When subdividing every predetermined number of lines, for example, when there is a ruled line, the ruled line is used as a reference, and when there is no ruled line, it is subdivided based on the line spacing of handwritten characters. However, it is not limited to this.
“Partial character image data” is data obtained by dividing character image data by an image dividing unit.
The “image dividing unit” is software for dividing the character image data.
“Two or more reading operators that individually read one partial character image data” refers to one selected partial character image data among a plurality of partial character image data obtained by dividing character image data. Two or more reading operators who read at their own discretion without consulting each other.

“A plurality of partial text data as a result of reading partial character image data transmitted one by one from each operator terminal” is transmitted from the operator terminals individually owned by two or more reading operators to the partial text determination unit ( It means the reading result of the input partial character image data of each operator. One reading result is obtained from each operator terminal.
The “partial text determination unit” is software that determines whether a plurality of partial text data transmitted one by one from each operator terminal match (match) or do not match. As a determination when there are three or more partial text data to be determined, for example, a determination method of whether or not all partial text data matches can be employed. In this case, the accuracy of reading partial character image data increases as the number of reading operators increases. In addition, it may be determined whether or not one set (two) or more partial text data matches except for the determination method of whether or not all the partial text data match.

As the “storage unit”, for example, various storages possessed by the host computer of the server can be employed.
The “partial text data for the entire text data stored in the storage unit” here is a partial text determination unit that is individually performed for all partial character image data corresponding to the total amount of character image data to be read. Means partial text data (all partial text data necessary for composing text data) obtained by the match determination (including deemed match determination).
Here, “partial text data for the entire text data is connected in accordance with an array in which character image data is divided into a plurality of partial character image data” means that character image data is divided into a plurality of partial character image data. By connecting all the partial text data stored in the storage unit, that is, all the partial text data determined to match (including the deemed match determination) by the partial text determination unit in the arrangement order when To create text data.
The “text creation unit” is software that creates (assembles) text data from all the partial text data provided in the host computer of the server.

  Further, the present invention provides a partial text data as a partial text determination unit that is transmitted as a result of reading from the operator terminal of the first reading operator and a part as a result of reading that is transmitted from the operator terminal of the second reading operator. A reading result transmitted from the operator terminal of the third reading operator when the determination results of the first partial determination unit and the first partial determination unit that determine whether or not the text data match are the same Whether or not the partial text data matches the partial text data transmitted from the operator terminal of the first reading operator, and is transmitted from the operator terminal of the third reading operator It is determined whether or not the partial text data The partial text data stored in the storage unit is the partial text data determined by the first partial determination unit or the first part. It is desirable to use partial text data that has been determined that the second partial determination units after determining that the determination units do not match.

The partial text data stored in the storage unit is partial text data determined by the first partial determination unit to match. Or it becomes the partial text data determined as the 2nd partial determination part in the case where it determines with a 1st partial determination part not matching.
Furthermore, if the result of the determination by the second partial determination unit does not give a match, the partial text transmitted from the operator terminal of the fourth or later (fifth, sixth,..., Nth) reading operator It is determined by the third and subsequent (fourth, fifth,..., N−1th) partial determination units whether the data and the partial text data acquired so far match. You may do it. Further, the fourth and subsequent reading operators refer to the partial text data acquired so far, and if there is partial text data to be stored in the storage unit, the corresponding partial text from the acquired partial text data. Data may be selected, and the selected partial text data may be transmitted as partial text data of the reading result of the operator from the terminal of the fourth or later reading operator.

According to the first aspect of the present invention, the authentication code of the display body is read by the code reader, and then the authentication information as the result of reading the authentication code is transmitted to the authentication unit of the server using the communication line network. . As a result, the text data conversion service is activated (authentication process) and, for example, on the server side, reference is made to preset maximum character number data (information) that can be converted into text, and this maximum character number data is stored in the client. Register for an account. Therefore, unauthorized use of the text data conversion processing service by the text data conversion processing unit can be prevented.
Next, character image data obtained by photographing characters including handwritten characters by a digital image acquisition unit (for example, a digital camera) of a terminal (for example, a smartphone) is transmitted to a server as a character reading center using a communication line network. Is converted into text data by the text data conversion processing unit. Thereby, text data conversion processing of characters including handwritten characters can be performed with high accuracy.

  In particular, according to the second aspect of the present invention, the text conversion area of the character including the handwritten character is marked with the marker, and this is transmitted to the text data conversion processing unit of the server. For example, transmission to the server with only the marking portion may be performed. In the text data conversion processing section, only the characters in the marked text conversion area are read and converted into text data. In this way, the use of markers eliminates the need for text conversion-only notebooks that were required in the past, and for various documents written with characters that include not only handwritten characters but also type. Only the necessary part (character area) can be converted into text data.

  According to the invention described in claim 3, when a smartphone is adopted as a terminal, the terminal can be carried, so that necessary characters (sentences) can be converted into text data regardless of time or place. Can do.

  Furthermore, according to the invention described in claim 4, a character (including a handwritten character) is scanned by a scanner (including a scanner unit of a digital multi-function peripheral), and the obtained character image data is transferred from the transmission unit on the scanner side to the terminal or Send to server. Among these, the character image data transmitted to the terminal is transmitted from the transmission unit of the terminal to the server via the communication line network. With this configuration, for example, when there are a large number of subjects (documents), a large amount of character image data can be acquired by a single scan. In addition, since there is almost no focus shift at the time of acquiring an image image (character image data), it is possible to acquire character image data with higher accuracy than shooting with a digital camera.

  According to the fifth aspect of the present invention, the character image data is divided into a plurality of partial character image data by the image dividing unit, and then each partial character image data is individually read by the partial text determining unit. It is determined whether or not a plurality of partial text data transmitted one by one from each operator terminal of two or more reading operators match. Here, the partial text data determined to be matched by the partial text determination unit is stored in the storage unit. In the same process, after the partial text data for the entire text data is stored in the storage unit, the text creation unit stores the partial text data into a plurality of partial character image data in accordance with the arrangement obtained by dividing the character image data. Create text data by connecting all the partial text data. As a result, it becomes possible to convert the text data of handwritten characters with higher accuracy than OCR. In addition, by dividing the character image data into a plurality of partial character image data by the image dividing unit, each operator cannot visually see the entire character image data. Thereby, information leakage can be prevented.

According to the sixth aspect of the present invention, when the partial text determination unit makes the determination, first, the partial text data as the reading result transmitted from the operator terminal of the first reading operator and the second reading operator's The first partial determination unit determines whether or not the partial text data as a reading result transmitted from the operator terminal matches. Here, when it is determined that the first partial determination units do not match, the partial text data as the read result transmitted from the operator terminal of the third reading operator and the operator terminal of the first reading operator Whether or not the partial text data transmitted from the second partial text data and the partial text data transmitted from the operator terminal of the second reading operator match. The determination is made by the second determination unit.
After the determination by the partial text determination unit performed in such a plurality of steps, the partial text data stored in the storage unit is the partial text data determined by the first partial determination unit to match or the second part The partial text data determined to match the determination units. As a result, the number of operator terminals used can be reduced, and handwritten character text data can be processed with higher accuracy than in the case of claim 2.

BRIEF DESCRIPTION OF THE DRAWINGS It is a perspective view which shows the whole structure of the character text conversion system based on Example 1 of this invention. It is a front view including the partial cross section figure which shows the whole structure of the display body on which the marker and authentication code which are used for the text-text-izing system based on Example 1 of this invention were displayed. It is a principal part block diagram of the character text conversion system which concerns on Example 1 of this invention. It is an authentication flow sheet of the text-to-text system according to Embodiment 1 of the present invention. It is an enlarged front view which shows the reading operation | work of the authentication code on the application screen of the terminal used for the text-text-izing system based on Example 1 of this invention. It is a whole flow sheet | seat of the character text-ized process of the character text-ized system which concerns on Example 1 of this invention. It is a flow sheet of the text data conversion processing step of the character text conversion system according to Embodiment 1 of the present invention. It is an enlarged front view which shows the paper surface of the memo paper by which the text conversion area | region was marked by the marker of the memo paper used for the text-text-izing system based on Example 1 of this invention. It is an enlarged front view which shows the paper surface of the note paper immediately after camera photography displayed on the application screen of the terminal used for the character text-ized system which concerns on Example 1 of this invention. It is an enlarged front view which shows the paper surface of the memo paper after color recognition of the text conversion area | region displayed on the application screen of the terminal used for the character text-ized system which concerns on Example 1 of this invention. It is an enlarged front view which shows the paper surface of the memo paper after distortion correction | amendment displayed on the application screen of the terminal used for the character text-ized system which concerns on Example 1 of this invention. It is a front view which shows many partial text conversion area | regions divided | segmented by the image division part of the server used for the character text-ized system based on Example 1 of this invention. It is a flow sheet of the partial text determination process of the text data conversion process of the character text conversion system concerning Example 1 of this invention. It is a perspective view which shows the marker and digital compound machine which are used for the character text-izing system based on Example 2 of this invention. It is a principal part block diagram of the character text conversion system which concerns on Example 2 of this invention. It is a whole flow sheet | seat of the character text-ized process of the character text-ized system which concerns on Example 2 of this invention.

  Examples of the present invention will be specifically described below.

  1 to 3, reference numeral 10 denotes a character text conversion system (hereinafter referred to as a text conversion system) according to Embodiment 1 of the present invention. The text conversion system 10 stores handwritten characters written on a handwritten memo paper 15. QR code (authentication code) 14 for authentication that obtains character number data (number of converted characters: 5000 characters) that can be converted into text using the text data conversion processing service while performing authentication of text data conversion processing service for text conversion A rectangular card 61 (display body) on which is displayed, a CMOS camera (digital image acquisition means) 16 that captures (optically inputs) handwritten characters into character image data, and a two-dimensional code that reads the QR code 14 A reader (code reader) 17, a communication unit 19 on the terminal side using the Internet (communication network) 18, and QR code 14 read data by the two-dimensional code reader 17 respectively transmitted from the terminal (here, a smartphone) 21 each equipped with the application 20 for processing the digitized data, the communication unit 19 on the terminal side, and the CMOS camera 16 A server 22 that receives handwritten character image data using the Internet 18, a server-side communication unit 23 that uses the Internet 18, and the server 22 are provided to read the character image data and convert it into text data. The text data conversion processing unit 24 and the server 22 that receive the read data of the QR code 14 authenticate the use of the text data conversion processing service and acquire the number of characters that can be converted into text. Part 25.

As shown in FIGS. 1 and 8, the memo paper 15 is one A4 paper. There are no ruled lines or grids on the paper, and it is blank.
As shown in FIGS. 1 and 2, the marker 60 is an orange and bold fluorescent pen. The marking of the handwritten characters by the marker 60 is performed by filling only the characters in the text conversion area 67 to be converted into text in the handwritten text written on the surface of the memo paper 15 (filled areas 67A and 67B). When it extends over several lines, it marks so that the part may be enclosed (enclosed area 67C).
The marker 60 is housed in a vertically long plastic packaging bag 62. The marker 60 may be stored in a paper box or a plastic box instead of the packaging bag 62. The packaging bag 62 stores a card 61 made of a plastic sheet displaying the QR code 14 in addition to the marker 60. The card 61 may be placed, for example, in an opaque child bag so that the QR code 14 cannot be easily read from the outside. On the back surface of the packaging bag 62, a method for using the text data conversion service is described. Separately from this, a user manual may be enclosed. Note that only the card 61 may be sold separately from the marker 60 in order to add the number of converted characters.

Next, the client terminal 21 will be described in detail with reference to FIG.
The terminal 21 is a smartphone that is a type of mobile phone, and is connected to the CPU 50 with respect to the CPU 50, the communication unit 19 on the terminal side, the application 20 dedicated to text data conversion installed on the terminal 21 from the net store, and the back surface. A CMOS camera 16, which is a kind of digital camera embedded in the upper part, a two-dimensional code reader 17 that reads the QR code 14, and a storage unit 32 are electrically connected to each other. In the application 20, a distortion correction unit 53 for correcting distortion of a photographed image of the memo paper 15 photographed by the CMOS camera 16 and the color (orange) of the fluorescent ink of the marker 60 are detected by a digital color sensor. Thus, the color recognition unit 52 that recognizes the text conversion area 67 marked by the marker 60 in the text of the memo paper 15 as a coordinate value on the paper surface of the memo paper 15 is mounted. The distortion correction unit 53 is software that uses the four corners of the paper surface of the memo paper 15 to correct distortion of the captured image on the paper surface by a trapezoidal correction method (projection conversion).

Next, the configuration of the server 22 will be described in detail with reference to FIGS. 3 and 12.
The server 22 mainly authenticates the use of the communication unit 23 for the server, the text data conversion processing unit 24 that converts the handwritten character image data into text data, and the text data conversion service purchased by the client. The authentication unit 25 to be connected to is electrically connected.
Among these, as shown in FIG. 3, the text data conversion processing unit 24 extracts the text conversion area 67 from the character image data uploaded from the terminal 21 to the server 22, and converts this into the partial text conversion for each line. Dividing into regions (partial character image data) (FIG. 12) It is determined whether or not the image dividing unit 34, the OCR unit 28 for optically reading a handwritten body, and text data conversion of the OCR unit 28 have been performed correctly. It is determined whether or not the text data of the OCR unit 28 of the server 22 matches as a determination result transmitted one by one from each of the two OCR determination operators α and β operator terminals 21α and 21β (FIG. 13). Each of the operator terminals 21A to 21 of the OCR determination unit 35 and two or more reading operators that individually read one partial text conversion area The partial text determination unit 40 for determining whether or not a plurality of partial text data as the read result of the partial text conversion area transmitted one by one from FIG. Then, the storage unit 33 for storing the determined partial text data and the partial text data for the entire text data stored in the storage unit 33 are arranged in accordance with an array in which character image data is divided into partial text conversion areas for each row. And a text creation unit 41 that creates text data.

  The extraction of the text conversion area 67 from the character image data may be performed by an independent conversion area extraction unit separated from the image dividing unit 34. The created text data is stored in the storage unit 33 of the server 22, and when necessary, the client accesses the server 22 from the terminal 21 or a personal computer to acquire the text data. At this time, the OCR determination operator or the reading operator may perform work by accessing the server 22 using his / her terminal at a convenient time while staying at home, instead of being an operator at normal work. The division of the partial text conversion area is not limited to each line (for example, every half line, every second line, every third line, etc.).

Here, a specific configuration of the partial text determination unit 40 in which data entry is performed will be described with reference to the flow sheet of FIG.
The partial text determination unit 40 includes partial text data as a reading result transmitted from the operator terminal 21A of the first reading operator A and partial text as a reading result transmitted from the operator terminal 21B of the second reading operator B. Transmitted from the operator terminal 21C of the third reading operator C when the determination results of the first partial determination unit 36 and the first partial determination unit 36 that determine whether or not the data match are the same. Whether or not the partial text data as the read result matches the partial text data transmitted from the operator terminal 21A of the first reading operator A, and the third partial text data and the second reading operator Partial text data transmitted from the operator terminal 21B of B If the determination result by the second partial determination unit 37 indicates that they are not consistent with each other, the result is 4 The partial text data as a reading result transmitted from the operator terminal 21D of the human reading operator D to the storage unit 33 of the server 22 is regarded as a matching determination result in the partial text determination unit 40.

  Therefore, the partial text data stored in the storage unit 33 (FIG. 3) is the partial text data determined by the first partial determination unit 36 to match or the first partial determination unit 36 does not match. The operator terminal of the fourth reading operator D after determining that the second partial determination unit 37 does not match, or after determining that the second partial determination unit 37 does not match Any of the partial text data transmitted from 21D to the storage unit 33 of the server 22. That is, the reading operator D can browse the reading results of the reading operators A, B, and C, and can transmit the reading results from the operator terminal 21D to the storage unit 33 at his / her own judgment.

Next, a text data conversion processing method for handwritten characters by the text conversion system 10 according to the first embodiment of the present invention will be described with reference to FIGS.
First, as shown in FIG. 1, the client connects the terminal 21, which is a smartphone, to the Internet 18, installs an application 20 dedicated to text data conversion processing from a net store, and uses a marker 60 shown in FIG. 2 from a net shop or the like. To buy.
Next, before using the text data conversion processing service by the text conversion system 10, the client takes out the card 61 displaying the QR code 14 from the packaging bag 62 of the marker 60 as shown in the flow sheet of FIG. The QR code 14 is read by a two-dimensional (QR) code reader 17 to obtain code information (FIG. 5). Thereafter, using the Internet 18 from the communication unit 19 on the terminal side, the usage authentication information is transmitted to the server 22 as a result of reading the QR code 14, and these are received by the communication unit 23 on the server side and transmitted to the authentication unit 25. (FIG. 3). Thereafter, use authentication is performed by the authentication unit 25. After the authentication is completed, an authentication completion notification is returned from the authentication unit 25 to the terminal 21, and the number-of-characters data that can be converted into text using the text data conversion processing service held by the server ( As a result, the number of convertible characters is added to the client account. As a result, it is possible to prevent unauthorized use of the text data conversion service beyond the number of characters that can be converted to text.

Next, referring to the flow sheets of FIGS. 4, 6, 7 and 13 and FIGS. 1, 5, 8 to 12, fill in a note paper 15 which is performed using the text system 10. A method for converting the handwritten characters into text data will be described.
First, as shown in FIG. 1 and FIG. 8, a sentence is handwritten on a memo paper 15 prepared by a client, and a portion (text conversion area 67) in which text conversion is desired is filled with a marker 60 (one line filling). (Area 67A, 67B) and the area is enclosed in a square (enclosed area 67C extending over 5 lines).
Then, as shown in the flow sheet of FIG. 6, after confirming the remaining usage amount of the text data conversion processing service in the state where the marker usage mode of the application 20 of the terminal (smart phone) 21 is activated, the CMOS camera 16 performs handwriting. Photographing is performed so that the entire surface of the memo paper 15 is within the field of view of the camera (FIG. 9). If there is no remaining usage amount when the marker usage mode is set, shooting with the CMOS camera 16 cannot be performed. Next, in the photographed character image data, the marked text conversion area 67 is recognized as the coordinate value of the paper surface of the note paper 15 by the orange color of the fluorescent ink of the marker 60 by the color recognition unit 52 of the application 20. Thereafter, using the four corners of the paper surface of the memo paper 15, the distortion correction unit (distortion correction function) 53 installed in the application 20 corrects the distortion of the paper surface when the camera 16 is photographed (FIGS. 10 and 11). That is, the distortion correction unit 53 searches the character image data for the four corners of the memo paper 15 based on the color shading, and uses these to detect distortion of the photographed image on the paper, using a known trapezoidal correction method (projection conversion). Correct by

The corrected character image data (including the text conversion area 67) is transmitted from the terminal-side transmission unit 19 to the server 22 as the character text conversion center using the Internet 18, and the server-side communication unit 23 transmits the data. It is received and transmitted to the text data conversion processing unit 24 (FIG. 3). It should be noted that only the text conversion area 67 may be transmitted to the character text conversion center.
Next, the text data conversion processing unit 24 converts the character image data into text data. The obtained text data is stored in the storage unit 33 of the server 22 together with the character image data. By doing so, the character image data and text data can be extracted from the storage unit 33 of the server 22 when accessed from the client terminal 21. The text data (the same applies to the character image data) is transmitted from the communication unit 23 on the server side to the communication unit 19 on the terminal side, displayed on the liquid crystal screen of the terminal 21, and stored in the storage unit (memory) 32 of the terminal 21. You may make it memorize | store. Further, only character image data (without text conversion) may be stored in the storage unit 33 of the server 22 or the storage unit 16 of the terminal 21 as desired by the client. In the converted text data, first, it is determined whether the usage amount of text data conversion processing service (number of characters that can be converted to text) exceeds the remaining number of specified characters (5000 characters). At 21, the processed text data can be acquired. On the other hand, if the number of characters exceeds the specified number of characters, a notification of exceeding the specified number of characters is sequentially transmitted to the application 20 of the terminal 21 via the communication unit 23 on the server side, the Internet 18, and the communication unit 19 on the terminal side. The state exceeding the number of characters is displayed on the liquid crystal screen of the terminal 21. In this state, if the client wishes to acquire text data, the number of characters that can be acquired by purchasing the marker 60 and authenticating it again may be added.

Here, the text data conversion processing step by the text data conversion processing unit 24 will be described in detail with reference to the flowcharts of FIGS.
In the text data conversion processing unit 24, the image dividing unit 34 (FIG. 3) first extracts the text conversion area 67 from the character image data from the coordinate value data based on the detection signal of the digital color sensor, and the extracted text. The conversion area 67 is divided for each row. Specifically, in the text written on the surface of the memo paper 15, one line of the filled areas 67 </ b> A and 67 </ b> B filled with the marker 60, and five lines surrounded by a square with the marker 60. Each of the regions 67C is extracted, and the enclosed region 67C is divided into five partial text conversion regions 67a to 67e by using a trimmer as a division criterion (FIG. 12). Note that the surrounding area 67C is not necessarily divided into the five partial text conversion areas 67a to 67e, and the text conversion processing may be performed as a group of surrounding areas 67C.
Thereafter, OCR of each partial text conversion area is performed by the OCR unit 28, and then all the partial text conversion areas are read by the two OCR determination operators α and β, respectively (FIG. 13). Thereafter, the OCR determination unit 35 determines whether or not the text data in the text conversion area 67 as a determination result transmitted one by one from the operator terminals 21α and 21β of the two OCR determination operators α and β match. If they match, a final check is performed by a check-dedicated operator E described later. On the other hand, if they do not match, the selected partial character image data is subjected to a data entry type reading process in which the first to fourth operators A to D and the operator E for checking are interposed. .

  Specifically, partial text data as a reading result transmitted from the operator terminal 21A of the first reading operator A, and partial text data as a reading result transmitted from the operator terminal 21B of the second reading operator B Are determined by the first partial determination unit 36. If they match, the common partial text data is transmitted to the operator terminal 21E of the operator E dedicated to checking. On the other hand, if they do not match (the mismatch rate is about 5%), the partial text data as the reading result transmitted from the operator terminal 21C of the third reading operator C and the operator terminal of the first reading operator A Whether or not the partial text data transmitted from 21A matches, and whether or not the third partial text data matches the partial text data transmitted from the operator terminal 21B of the second reading operator B Is determined by the second partial determination unit 37.

If the determinations in the second partial determination unit 37 match, the common partial text data is transmitted to the operator terminal 21E dedicated to the check. On the other hand, when a determination result that none of the two matches is obtained by the determination by the second partial determination unit 37 (the mismatch rate is about 1%), the server terminal 21D of the fourth reading operator D receives the server The partial text data as a reading result transmitted to the storage unit 33 of 22 is regarded as a matching determination result in the partial text determination unit 40, and the regarded partial text data is transmitted to the operator terminal 21E dedicated to checking. The
In this way, the partial text data obtained corresponding to the selected partial text conversion area is sequentially stored in the storage unit 33 of the server 22. At this time, the partial text data stored in the storage unit 33 is determined to be the partial text data determined by the first partial determination unit 36 to match or the first partial determination unit 36 does not match. Later, it is determined from the operator terminal 21D of the fourth reading operator D after determining that the second partial determination unit 37 matches the partial text data or the second partial determination unit 37 does not match. One of the partial text data transmitted to the storage unit 33 of the server 22. As a result, it is possible to process the text data of handwritten characters with higher accuracy than the OCR unit 28.

After that, it is determined whether or not the text conversion of all the partial text conversion areas has been completed. If it has been completed, all the partial text data for the entire text conversion area 67 stored in the storage unit 33 is stored. Then, the text creation unit 41 creates the text data by connecting the text conversion area 67 in accordance with the arrangement obtained by dividing the text conversion area 67 into a plurality of partial text conversion areas. As a result, it is possible to process the text data of handwritten characters with higher accuracy than the OCR unit 28.
Since the text data conversion processing unit 24 described above performs the text data conversion processing of the character image data, the text data conversion processing of handwritten characters (the same applies to the type) can be performed with high accuracy. Moreover, by dividing the text conversion area 67 extracted from the character image data by the image dividing unit 34 into a plurality of partial text conversion areas, the operators A to D cannot visually see the entire character image data. Thereby, information leakage can be prevented.

In addition, since only the necessary part in the sentence is selected by marking the text conversion area 67 with the marker 60 and only the text conversion area 67 is converted into text by the text data conversion processing unit, the text that was conventionally required A conversion-only notebook or the like is not required, and only a necessary portion (character area) can be converted into text data for various documents including general memo paper 15.
In addition, since a portable smartphone is adopted as a terminal, necessary characters (sentences) can be converted into text data regardless of time and place.

Next, with reference to FIG. 14 to FIG. 16, a character text conversion system according to Embodiment 2 of the present invention will be described.
As shown in FIGS. 14 to 16, a character text conversion system (hereinafter referred to as a text conversion system) 10 </ b> A according to the second embodiment is characterized by a smartphone terminal for a memo paper 15 in which a text conversion area 67 is marked by a marker 60. Instead of acquiring the character image data using the 21 CMOS camera 16, the acquisition using the digital multi-function device 63 is adopted.

  The digital multifunction device 63 includes an electronic device including a scanner unit (scanner) 66 and a multifunction device side transmission unit (scanner side transmission unit) 64 using a wireless LAN, in addition to the main function copy unit 65. It is. In the digital multi-function device 63, the paper surface of the memo paper 15 is scanned by scanning, and the photoelectric conversion sensor receives the reflected light from the paper surface and converts it into an electrical signal, thereby acquiring character image data. The transmission unit 64 on the multifunction device side transfers character image data to the terminal 21 that is a smartphone using a wireless LAN. The transmission destination from the transmission unit 64 on the multifunction machine side may be the server 22 instead of the terminal 21. In this case, the character image data is transmitted directly to the server 22 without going through the terminal 21 equipped with the color recognition unit 52. The color recognition unit 52 needs to be provided in the digital multi-function peripheral 63 or the server 22.

Next, a text data conversion processing method for handwritten characters on the memo paper 15 by the text conversion system 10A according to the second embodiment will be described with reference to FIGS.
In the text data conversion process of the second embodiment, first, as shown in FIG. 16, the scanner unit 66 of the digital multi-function peripheral 63 scans the paper surface of the memo paper 15 to obtain character image data on the paper surface. Thereafter, the acquired character image data is transferred from the transmission unit 64 on the multifunction machine side to the terminal 21 via the wireless LAN. The character image data transferred to the terminal 21 is then color-recognized according to the flow of FIG.
As described above, since the digital multi-function device 63 is used as the image acquisition means, the digital multi-function device 63 provided in the office or the like can be used for image input by scanning without using the CMOS camera 16 of the smartphone. Since there is almost no focus shift at the time of shooting, a highly accurate text data processing service based on high-quality character image data can be received. Also, a large amount of character image data can be acquired by a single scan.
Since other configurations, operations, and effects are in a range that can be estimated from the first embodiment, the description thereof is omitted.

  The present invention is useful as a technique for converting characters including handwritten characters into text data using a text data conversion processing unit of a server.

10, 10A text-to-text system,
14 QR code (authentication code),
15 note paper,
16 CMOS camera (digital image acquisition means),
17 Two-dimensional code reader (code reader),
18 Internet (communication network),
19 Communication part on the terminal side,
21 terminals,
21A-21E operator terminal,
22 servers,
23 server side communication unit 24 text data conversion processing unit,
25 Authentication section,
33 storage unit,
34 Image segmentation unit,
40 partial text determination unit,
41 Text creation part,
60 marker 61 card (display),
63 Digital MFP,
64 MFP-side transmitter (scanner-side transmitter),
66 Scanner section (scanner),
67 Text conversion area,
A First reading operator,
B Second reading operator,
C Third reading operator,
D Fourth reading operator.

Claims (6)

A display body on which an authentication code for obtaining character number data that can be converted into text using the text data conversion processing service is displayed while performing use authentication of the text data conversion processing service for converting text including handwritten characters into text,
A terminal equipped with digital image acquisition means for optically inputting characters including the handwritten characters into character image data, a code reader for reading the authentication code, and a communication unit on the terminal side using a communication line network; ,
On the server side that receives the read data of the authentication code by the code reader transmitted from the communication unit on the terminal side and the character image data acquired by the digital image acquisition means using the communication line network A server equipped with a communication unit;
An authentication unit that is provided in the server and receives the authentication code read data to authenticate the use of the text data conversion processing service, and acquires the text-convertible character count data;
A character text conversion system comprising: a text data conversion processing unit that reads the character image data and converts it into text data. A marker for designating a text conversion area of characters including the handwritten character by marking;
A color recognition unit that is provided in the terminal or the server and recognizes a text conversion area of a character including the handwritten character from a color of marking by the marker;
The character text conversion system according to claim 1, wherein the text data conversion processing unit reads only characters including handwritten characters in the text conversion area recognized by the color recognition unit and converts them into the text data.   The character text conversion system according to claim 1, wherein the terminal is a smartphone. The digital image acquisition means is a scanner including a scanner-side transmission unit connected to the terminal,
The character which includes the handwritten character is scanned by the scanner, and the obtained character image data is transmitted to the terminal or the server by the transmission unit on the scanner side. The text-to-text system described in the section. The text data conversion processing unit
An image dividing unit for dividing the character image data into a plurality of partial character image data;
Whether or not a plurality of partial text data as a result of reading the partial character image data transmitted one by one from each operator terminal of two or more reading operators that individually read one partial character image data match. A partial text determination unit for determining
A storage unit for storing the partial text data determined to match the partial text determination unit;
Text that creates the text data by connecting the partial text data for the entire text data stored in the storage unit in accordance with an array obtained by dividing the character image data into the plurality of partial character image data The character text conversion system according to any one of claims 1 to 4, further comprising a creation unit. The partial text determination unit
Whether or not the partial text data as the reading result transmitted from the operator terminal of the first reading operator matches the partial text data as the reading result transmitted from the operator terminal of the second reading operator A first partial determination unit for determining
Partial text data as a reading result transmitted from the operator terminal of the third reading operator when the determination result of the first partial determination unit is inconsistent, and transmitted from the operator terminal of the first reading operator A process for determining whether or not the partial text data matches, and whether or not the third partial text data matches the partial text data transmitted from the operator terminal of the second reading operator. 2 partial determination units,
The partial text data stored in the storage unit is the partial text data determined by the first partial determination unit to match, or after determining that the first partial determination unit does not match, 6. The character text conversion system according to claim 5, which is partial text data determined by the second partial determination unit to match.

Images