KR102060818B1 - A Digital output method of handwritten text and mathematical expressions - Google Patents

Abstract

The present invention provides a digital output method for handwriting texts and equations. The method comprises: a handwriting data receiving step of receiving handwriting data from a user through an input unit; a unit handwriting information type identification step of determining a unit handwriting information type of each of a plurality of unit handwriting information included in the handwriting data, and comparing the handwriting information with pre-stored symbol data to be classified into a non-formal text type or a mathematical symbol type; a correlation determination step of determining the correlation between the unit handwriting information adjacent to each other in the handwriting data; an equation tree determination step of determining a final equation tree among equation trees which are equation structures capable of being combined with the plurality of unit handwriting information included in the handwriting data, when the type of the unit handwriting information is the mathematical symbol type, based on the result values of the unit handwriting information type identification step and the correlation determination step; and a digital conversion step of being performed to convert the handwriting data into a digitized equation corresponding to the final equation tree and visually output the same to an output unit based on the final equation tree determined in the equation tree determination step, and when the type of the unit handwriting information is a non-formal text type, recognize the handwriting data as non-formal text which does not contain a formula, and convert the handwriting data into digitized non-formal text to be output. The present invention can increase accuracy by increasing the recognition rate of the handwriting text and the equations.

Description

A digital output method of handwritten text and mathematical expressions}

The present invention relates to a digital output method of handwritten texts and mathematical expressions that can digitize handwritten data including texts and mathematical expressions input manually by a user and output them accurately and quickly.

In general, the terms and products of touch panel or touch screen were originally developed for military use. Since then, it has been used for a long time in the military, electronics and IT industries. It has been closely used in real life since early 2000. In the early stages of industrial applications, monitors using a touch interface were mainly employed in kiosks.

Recently, as tablets and smartphones are generally used, the spread of touch panels or touch screens has been activated, and together with handwritten data input through input devices such as tablets and smartphones, motion data obtained when texts are input together is used. Therefore, the importance of the character recognition technology capable of outputting digital characters is emerging.

However, as the motion data obtained at the time of handwriting is different for each user, the accuracy of the digital character desired by the user is still low, especially when writing an equation, a completely different result is output due to an error in recognition of the order of the input character. There was a problem.

On the other hand, unlike the case where the text and the equation are separately written and written by hand, when the text and the equation are mixed and written by hand, the digitized text and the mathematical expression due to the occurrence of an error are output. There was a problem that was impossible to do.

The present invention has been made to solve the above problems, an object of the present invention is to convert the text and mathematical expressions entered by hand, as well as outputting the mixed text and mathematical expressions entered by the user by hand In the output, it is intended to improve the accuracy of the finalized digitized text and equations.

The present invention provides a digital output method of handwritten text and equations, the method comprising receiving handwritten data from a user through an input unit, and handwritten information of each of the plurality of unit handwritten information included in the handwritten data. Determining a type, and comparing the unit handwriting information with previously stored symbol data and classifying the unit handwriting information into a non-formal text type or a mathematical sign type, and between the unit handwriting information neighboring each other in the handwriting data. If the type of the unit handwriting information is the mathematical symbol type, it is included in the handwriting data based on a result of the correlation relationship determining step for determining the correlation, the unit handwriting information type identification step, and the result of the correlation determination step. Among the mathematical trees that can be combined with the plurality of unit information information On the basis of the equation tree determination step of determining the final equation tree and the final equation tree determined in the equation tree decision step, the handwritten data is converted into a digital equation corresponding to the final equation tree to the output unit. When the type of the handwritten information is a non-formal text type while visually outputting, the digital conversion step of recognizing handwritten data as non-formal text not including a formula and converting the handwritten data into digitized non-formal text and outputting it Characterized in that it comprises a.

More specifically, the input unit is a touch screen or a handwriting recognition panel that receives a user's handwriting trajectory as an input signal in a pressure sensitive or capacitive manner, and the handwriting data receiving step receives the handwritten data from the user through the input unit. After the predetermined time has elapsed, the handwritten data may be executed to determine the recognition target.

In addition, the input unit is a camera module and the handwriting data receiving step is characterized in that is executed to determine the image information photographed and obtained through the input unit as handwritten data.

In more detail, the unit handwriting information type identification step may include: a unit handwriting information detecting step of detecting a plurality of handwriting information which is handwriting information included in the handwriting data; and each of the handwriting information is set within a preset unit area. A boundary division step of setting a unit area so as to be accommodated, dividing the unit area into a plurality of pixels, and a retention information extraction step of extracting retention information of the unit note information for each pixel for each unit note information; On the basis of the result of comparing the characteristic information for each digital symbol information and comparing the characteristic information for each digital symbol information stored in the memory unit with the retention information of the unit handwriting information, Determining digital symbol information corresponding to unit handwritten information of the digital symbol information; After the digital identification information output step of outputting the digital identification information of the information and the digital identification information output step, the type of dividing the type of digital symbol information output through the digital identification information output step into a non-formal text type or mathematical symbol type Characterized in that it is executed to include a classification step.

In more detail, the retaining information of the unit note information or the feature information for each digital symbol information may include an absolute coordinate value, a relative coordinate value, a slope of a stroke, a curvature value, an area of the retained information included in the unit area. And at least one information of a value, center of gravity, correlation, and any feature value.

More specifically, the mutual relationship determination step is a relationship analysis for grouping a plurality of neighboring unit handwritten information among the plurality of unit handwritten information, and setting a relationship analysis area for accommodating the grouped plurality of handwritten information. An area setting step, a boundary dividing step of dividing the relationship analysis area into a plurality of pixels, extracting retention information of the relationship analysis area for each pixel for each relationship analysis area, and extracting the retention information of the extracted relationship analysis area. Extracting retention relationship information for extracting retention relationship information between a plurality of units of handwritten information in the relationship analysis region based on the characteristics of digital relationship information between the retention relationship information of the plurality of units of handwritten information and symbols stored in a memory unit; Comparing the characteristic information for each digital relation information between symbols and comparing the information; On the basis of the result value of the feature information comparison step for each information, the inter-symbol digital relationship information determining step for determining the inter-symbol digital relationship information corresponding to each of the retention relationship information and outputting the digital identification information of the inter-symbol digital relationship information; And the step of outputting the digital identification information.

More specifically, the equation tree determining step may include a first call step of calling digital identification information of all digital symbol information output in the unit handwriting information type identification step, and a full symbol output in the correlation determination step; A second call step of calling the digital identification information of the digital relation information between the two; and a mathematical tree generation step of generating a plurality of mathematical structure trees possible by combining the information of the first call step and the information of the second call step. And an equation structure tree for determining a coincidence rate with handwritten data for each of the plurality of equation structure trees and an equation structure tree having the highest coincidence rate in the coincidence rate evaluation step as a final equation structure tree. Determination step; characterized in that it is executed to include.

In more detail, the digital conversion step may be performed to include a first conversion output step of converting the handwritten data into a digital equation corresponding to an equation structure tree having the highest match rate and outputting the digital equation.

In more detail, the digital conversion step may include at least one or more next-order digital equations corresponding to the following structure tree of the next-order coincidence ratio in a neighboring output area of the digital equation output in the first conversion output step. A candidate group temporary output step of outputting, a change output step of changing the output of the candidate order digital equation to the selected next-order digital equation when the selection signal for the at least one next-order digital equation is received within the preset time, and the preset time And if the selection signal for the at least one or more next-order digital equations is not received therein, finally determining, outputting, and storing the digital equations in the first transformed output step.

More specifically, in the digital identification information outputting step, when unit symbol information for forming a mathematical tree structure is not found among the plurality of unit handwriting information, the handwritten data is converted into non-expression text containing no formula. And recognize and convert the handwritten data into digitized non-formal text for output.

The digital output method of handwritten text and mathematical expression according to the present invention improves the recognition rate of handwritten text and mathematical expression through the unit handwriting information type identification step and mutual relationship determination step for identifying handwritten data. It can provide an effect that can further improve the accuracy of the digitized text and equations.

In addition, through the unit information information type identification step, it is possible to determine the type of the text and equations inputted by hand, and then to output the digitized text and equations according to the determined type, to the text and equations It is possible to output continuous digitized text and mathematical expressions, thereby providing an effect of improving the user's input convenience.

In addition, through the formula tree determining step, the order of handwritten data input by the user can be correctly divided and arranged so that the order of each symbol is not changed in the formula, so that the user can accurately output the desired formula. Can provide an effect.

In addition, when the digital conversion step is performed, at least one or more next-order digital equations may be output for a predetermined time, so that the user may selectively input a desired equation, thereby making the handwritten text and equations wrong. By correcting this when input, it is possible to provide an effect of improving the reliability of the final digitized text and mathematical expressions.

1 is a schematic block diagram of a system for performing a digital output method of handwritten text and mathematical expression of the present invention.
2 is a schematic flowchart showing the overall procedure of the digital output method of handwritten text and equation of the present invention.
FIG. 3 is an exemplary diagram schematically illustrating a state in which a handwritten data receiving step is performed and output to an output unit in the digital output method of handwritten text and equation of the present invention.
4 is a flowchart illustrating a series of processes in which the unit handwriting information type identification step according to an embodiment of the present invention is performed.
5 is a schematic diagram illustrating an example of unit handwritten information type identification step according to an embodiment of the present invention.
6 is a flowchart illustrating a series of processes in which the mutual relationship determination step according to an embodiment of the present invention is performed.
7 is a schematic diagram illustrating an example of a mutual relationship determination step according to an embodiment of the present invention.
8 is a flowchart illustrating a series of processes in which a mathematical tree determination step is performed according to an embodiment of the present invention.
9 is a schematic diagram illustrating an example of a mathematical tree determination step according to an embodiment of the present invention.
10 is a flowchart illustrating a series of processes in which a digital conversion step is performed according to an embodiment of the present invention.
11 is an exemplary diagram illustrating an example of a digital conversion step according to an embodiment of the present invention.

Advantages and features of the present invention, and methods for achieving them will be apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings.

However, the present invention is not limited to the embodiments disclosed below, but may be embodied in various different forms, and the present embodiments merely make the disclosure of the present invention complete, and those of ordinary skill in the art to which the present invention belongs It is provided to fully inform the person having the scope of the invention, and the invention is defined only by the scope of the claims.

Also, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase. As used herein, "comprises" and / or "comprising" does not exclude the presence or addition of components other than the mentioned components.

Unless otherwise defined, all terms used in the present specification (including technical and scientific terms) may be used in a sense that can be commonly understood by those skilled in the art.

Hereinafter, with reference to the accompanying drawings will be described in detail with respect to the digital output method of handwritten text and equations according to the present invention.

First, FIG. 1 is a block diagram schematically illustrating a system for performing a digital output method of handwritten text and mathematical expression of the present invention.

As shown in FIG. 1, the controller 100, the input unit 200, the output unit 300, the communication unit 400, and the memory unit 500 may be used to perform the digitized text and mathematical expression output method of the present invention. A system comprising a can be built.

In this case, the controller 100 may process signals, data, information, or the like input or output through the components when the digitalized text and mathematical output method of the present invention is performed.

In addition, in addition to the operations related to the digitized text and mathematical output method of the present invention, the controller 100 generally controls the overall operations of the system.

In addition, the controller 100 may control at least some of the above-described components including the input unit 200, the output unit 300, the communication unit 400, and the memory unit 500. In addition, the controller 100 may operate by combining at least two or more of the components included in the system in order to proceed with the digitized text and mathematical output method of the present invention.

Next, the input unit 200 may include a touch screen capable of inputting character information such as handwritten data, a handwriting recognition panel, a camera module, an optical character recognizer (OCR), a scanner, and the like.

Next, the output unit 300 is provided to output information related to the user's time, and is provided with a mobile phone, smart phone, notebook computer, digital broadcasting terminal, provided with a display capable of checking in real time the information directly input by the user, Personal digital assistants (PDAs), portable multimedia players (PMPs), navigation, tablet PCs, and the like.

In addition, the output unit 300 may form a mutual layer structure or integrally with the input unit 200 to implement a touch screen.

The touch screen may function as a user input unit that provides an input interface between the output unit 300 and the user, and may also provide an output interface between the output unit 300 and the user.

Next, the communication unit 400 may include one or more components for performing wired / wireless communication between the control unit 100 and each component constituting the system.

For example, the communication unit 400 may include a mobile communication module, a short range communication module, and the like.

The mobile communication module transmits and receives a radio signal with at least one of a base station, an external terminal, and a server on a mobile communication network. Here, the wireless signal may include various types of data according to transmission and reception of a voice call signal, a video call signal, or a text / multimedia message.

The short range communication module refers to a module for short range communication. As a short range communication technology, Bluetooth, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra-Wideband (UWB), ZigBee, and the like may be used.

In addition, the memory unit 500 is provided to store various data supporting various functions required for outputting handwritten data in digitized text and mathematical expressions, and for smooth control in the control unit 100, Data and instructions including feature information for digital symbol information, which is a criterion for non-formal text type or mathematical symbol type determination, can be stored.

Next, FIG. 2 is a schematic flowchart showing the overall procedure of the digital output method of handwritten text and equation of the present invention, and FIG. 3 is a step of receiving handwritten data in the digital output method of handwritten text and equation of the present invention. FIG. 1000 is an exemplary diagram schematically illustrating a state output to the output unit at step S1000.

As shown in Figure 2, the digital text output method of the handwritten text and equation of the present invention, the handwriting data receiving step (S1000), unit handwriting information type identification step (S1100) correlation relationship step (S1200), equation tree A decision step S1300 and a digital conversion step S1400 are included.

First, the handwritten data receiving step S1000 is performed so that the control unit receives handwritten data from the user through the input unit.

As illustrated in FIG. 1, the controller may control the input unit to receive handwritten data from the user through the input unit.

In this case, the handwriting data receiving step (S1000) may be performed by determining, as handwritten data, image information captured and acquired through the input unit including a touch screen, a handwriting recognition panel, and a camera module.

In this case, as shown in FIG. 3, the controller may control the input unit to receive the user's writing trace as an input signal in the input / output layer m in the input / output layer m.

When the handwritten data is input into the recognition area disposed in the input / output layer m, the control unit receives the handwritten data from the user through the input unit, and receives the handwritten data after a predetermined time has elapsed. It can be determined and executed.

Thereafter, the controller may control the output unit to output information on handwritten data currently input to the user through the output unit.

Next, the unit handwriting information type identification step (S1100), after the handwriting data receiving step (S1000) in the control unit, determines the handwritten information type of each of the plurality of handwritten information included in the handwritten data, the unit Pre-stored handwritten information A step of dividing the information into feature information for each symbol information into a non-formal text type or a mathematical symbol type will be described in more detail with reference to FIGS. 4 and 5.

4 is a flowchart showing a series of processes in which the unit handwriting information type identification step S1100 according to an embodiment of the present invention is performed, and FIG. 5 is a step number identification information identification step according to an embodiment of the present invention. It is a schematic diagram which showed an example of S1100.

4 and 5, the unit handwriting information type identification step (S1100) is a unit handwriting information detection step (S1110), boundary division step (S1120), holding information extraction step (S1130), digital symbols in the control unit The feature information for each information is compared to a step S1140, a digital decision step S1150, a digital identification information output step S1160, and a type classification step S1170.

First, the unit handwriting information detecting step (S1110) is performed so that the control unit detects a plurality of unit handwriting information which is handwritten information included in the handwritten data, respectively.

In this case, as shown in (a) of FIG. 5, the plurality of unit handwriting information may include mathematical symbols, texts, and special characters that may be input to the input unit by a handwriting of a user, and may include a predetermined area. It can be input in the set recognition area (W).

In this case, as shown in b of FIG. 5, the boundary division step S1120 may be performed in the controller for text and mathematical symbols, respectively.

In addition, the text may be divided into one or more language units selected from syllables, morphemes, words, words, phrases, sentences, and paragraph units, and the boundary division step (S1120) to be described later may be performed by the controller. .

Subsequently, as shown in (c) of FIG. 5, the boundary dividing step (S1120) performed on the mathematical symbol among the text and the mathematical symbol is described. The unit area L1 is set so that each of the information C1 is accommodated in the preset unit area L1, and the unit area L1 is divided into a plurality of pixels.

In addition, although not particularly illustrated in the drawing, the boundary dividing step S1120 may be performed in the same manner with respect to the text.

In this case, the unit area L1 may be determined according to the resolution, and the recognition of the handwritten data input through the high resolution unit area L11 or the touch screen or the handwriting recognition panel for image recognition acquired through the camera module is performed. It may include a low resolution unit region (L12) for.

In addition, the boundary dividing step (S1120) may be performed on each of a plurality of unit handwritten information, thereby improving the recognition rate for handwritten data recognition.

Thereafter, the holding information extraction step (S1130) is performed by the controller to extract the holding information of the unit note information for each pixel, for each unit note information (C1), after the boundary division step (S1120).

At this time, as shown in (d) of FIG. 5, the retaining information of unit handwritten information for each pixel in each handwritten image or pixel column may include absolute coordinate values, relative coordinate values, It may include information on at least one or more of the inclination, curvature value, area value, center of gravity, correlation, and any feature value of the stroke.

Thereafter, the feature information comparison step for each digital symbol information (S1140) is performed by the controller to compare the retention information of each unit handwriting information with the feature information R1 for each digital symbol information stored in the memory unit.

In this case, the feature information R1 for each digital symbol information includes absolute coordinate values, relative coordinate values, inclination of strokes, curvature values, area values, centers of gravity, correlations, and arbitrary features with respect to handwritten information included in the unit region. At least one of the values may include information.

In addition, in order to extract a feature value for handwritten information included in the unit region in the feature information comparison step according to the digital symbol information (S1140), an artificial neural network may be applied.

In addition, although not specifically illustrated in the drawing, when the handwritten information included in the unit area may be classified into a text type in the comparison of feature information for each digital symbol information (S1140), the handwritten information that may be classified into the text type by the controller may be used. It can be evaluated by quantifying the possibility of classifying text types for.

Subsequently, as shown in FIG. 5E, the digital symbol information determining step S1150 is performed based on a result value of the feature information comparison step for each digital symbol information in the controller, respectively, corresponding to each unit handwritten information. Proceeds to determine the digital symbol information (D1) of.

And, as shown in Figure 4, the digital identification information output step (S1160) proceeds to output the digital identification information of the digital symbol information (D1) from the controller.

In addition, the type classification step (S1170) is a digital output through the digital identification information output step (S1160), if there is no unit number information remaining after the digital identification information output step (S1160), the control unit; The type of symbol information is divided into non-formal text type or mathematical symbol type.

Next, as shown in FIG. 2, in the mutual relationship determination step (S1200), after the unit handwriting information type identification step (S1100), the unit handwriting information identified through the unit handwriting information type identification step is a mathematical symbol type. In this case, the control unit is a step of determining the mutual relationship between neighboring unit handwriting information in the handwritten data, which will be described in more detail with reference to FIGS. 6 and 7.

FIG. 6 is a flowchart illustrating a series of processes in which the correlation determining step S1200 is performed according to an embodiment of the present invention, and FIG. 7 is a flowchart of the correlation determining step S1200 according to an embodiment of the present invention. A schematic diagram illustrating one example.

As shown in Fig. 6 and 7, the correlation determination step (S1200) is a digital identification information call step (S1210), the relationship analysis region setting step (S1220), boundary division step (S1230), retention relationship information in the control unit It is executed to include an extraction step (S1240), a feature information comparison step for each digital symbol information between symbols (S1250), a digital relationship information determination step (S1260) between digital symbols, and a digital identification information output step (S1270).

As shown in FIG. 6 and FIG. 7, the digital identification information call step (S1210) calls digital identification information D2 of each neighboring unit handwriting information which is a conversion target of electronic data to be finally output from the controller. To proceed.

In this case, as shown in FIG. 7A, the plurality of neighboring unit handwriting information may include mathematical symbols, texts, and special characters that may be input to an input unit by a handwriting of a user, and may include a predetermined recognition having a predetermined area. It can be entered in the area W.

In the setting of a relation analysis region (S1220), the controller groups the plurality of unit manuscript information into a plurality of neighboring unit manuscript information, and sets a relation analysis region to accommodate the grouped plural unit manuscript information. To proceed.

In this case, the relation analysis region setting step (S1220) may be performed such that one group of the plurality of unit handwritten information C2 and the other grouped plurality of handwritten unit information overlap each other.

By doing so, the relationship analysis region L2 is executed so that a plurality of neighboring unit handwriting information overlap each other, so that more accurate digital relationship information can be obtained, and as a result, the digitized text and mathematical transformations are obtained. It can provide an effect that can further improve the accuracy.

The boundary dividing step (S1230) is a step in which the control unit divides the relationship analysis region L2 into a plurality of pixels.

In particular, as shown in (b) and (c) of FIG. 7, each of the relationship analysis region C2 detected through the unit handwritten information detection step by the control unit is accommodated in a preset relationship analysis region L2. The relation analysis region L2 is set so that the relation analysis region L2 is divided into a plurality of pixels.

In this case, the relationship analysis region L2 may be determined according to the resolution, and the high resolution relationship analysis region L21 for image recognition obtained through the camera module or the handwritten data input through the touch screen or the handwriting recognition panel may be used. A low resolution relationship analysis region L22 for recognition may be included.

In the holding relationship information extracting step (S1240), the controller extracts the holding information of the relationship analysis region for each pixel for each relationship analysis region, and based on the extracted retention information of the relationship analysis region, Extracting the retention relationship information between a plurality of unit handwriting information.

In this case, as shown in (d) of FIG. 7, the retaining information of the relation analysis region or the feature information for each digital relation information between symbols may include absolute coordinate values of the retaining information included in the relation analysis region L2, It may include at least one or more of a relative coordinate value, the slope of the stroke, the curvature value, the area value, the center of gravity, the correlation and any characteristic value.

In addition, the artificial neural network may be applied to extract a feature value for holding relationship information between a plurality of pieces of unit information included in the unit area in the holding relationship information extracting step (S1240).

In the comparing of the feature information for each piece of digital relation information between symbols (S1250), the control unit compares the retention relationship information C2 of the plurality of unit handwriting information and the feature information for each piece of digital relationship information between symbols stored in a memory unit. The comparison is made to judge.

Subsequently, as shown in FIG. 7E, the ongoing inter-digital digital information determination step (S1260) is performed by the control unit based on the result of the comparison of the feature information for each digital-signal information between the symbols. Proceed to determine the interdigital digital relationship information D2 corresponding to the relationship information.

The digital identification information outputting step (S1270) is performed by the control unit to control the output unit to output digital identification information of the digital relation information between symbols through the output unit.

On the other hand, after the digital identification information output step (S1270) of the correlation determination step (S1200), if a mathematical symbol does not exist in one grouped unit handwriting information, the control unit does not include a formula in the handwritten data The output unit may be controlled to recognize the non-mathematical text and convert the handwritten data into digitized text and equations.

Next, as shown in FIG. 2, in the equation tree determination step S1300, after the correlation determination step S1200, a control unit determines a result value of the unit handwriting information type identification step and the correlation relationship determination step. If the type of the unit handwriting information is a mathematical symbol type based on the step, determining the final mathematical tree from among the mathematical tree of the mathematical structure which can be combined into the plurality of unit handwritten information included in the handwritten data; This will be described in more detail with reference to FIGS. 8 and 9.

8 is a flowchart illustrating a series of processes in which the mathematical tree determination step S1300 according to an embodiment of the present invention is performed, and FIG. 9 is a mathematical tree determination step S1100 according to an embodiment of the present invention. Is a schematic diagram illustrating an example.

As shown in Figure 8 and 9, the equation tree determination step (S1300) is the first call step (S1310), the second call step (S1320), the equation tree generation step (S1330), the match rate evaluation in the control unit It is executed to include the step S1340 and the equation structure tree determining step S1350.

First, as shown in (a) of FIG. 9, the first call step S1310 allows the controller to call the digital identification information of the entire digital symbol information D1 output in the unit handwriting information type identification step. Proceed.

And, as shown in (b) of FIG. 9, the second call step (S1320), the controller proceeds to call the digital identification information of the overall interdigital digital relationship information (D2) output in the correlation determination step. do.

Thereafter, in operation S1330, the control unit generates a plurality of equation structure trees by combining the information of the first call step and the information of the second call step.

In this case, as shown in FIG. 9C, the plurality of mathematical structure trees T may include a first mathematical symbol representing an equal sign or an inequality sign among the mathematical symbols constituting the mathematical expression. A second mathematical symbol, which is a mathematical symbol excluding letters, numbers, and the first mathematical symbol, may be disposed below the first mathematical symbol.

In addition, when the second modification symbol is disposed below the first modification symbol, a third modification symbol that is the same as or different from the second modification symbol may be disposed below the second modification symbol. have.

In addition, the artificial neural network may be applied to generate a plurality of mathematical structure trees in the mathematical tree generation step (S1330).

As described above, in the case where any one of the mathematical symbols is arranged in the mathematical tree, another mathematical symbol can be arranged below the one of the mathematical formulas, and the letters or numbers are replaced with the mathematical symbols. By dividing the data as a reference, it is possible to provide an effect of improving the accuracy of the finalized digitized equation.

Subsequently, as described in (d) of FIG. 9, the coincidence rate evaluation step (S1340) is performed by the controller to evaluate the coincidence rate with the above-described handwritten data for each of the plurality of mathematical structure trees (T).

In operation S1350, the controller determines the equation structure tree having the highest agreement rate as the final equation structure tree in the evaluation of the agreement rate.

Next, as shown in FIG. 2, in the digital conversion step (S1400), when the type of the unit handwriting information is a text type among the plurality of unit handwriting information, the digital conversion unit converts the digitized non-formal text into an output unit. When the type of unit handwriting information is a mathematical symbol type, the handwriting data is visually output to the final formula tree after the step of determining the formula tree (S1300). Converting to a corresponding digitized equation to control the visual output from the output unit.

Particularly, only when no unit symbol information for forming a mathematical tree structure to be described later is found among the plurality of unit handwriting information, the control unit recognizes handwritten data as non-formal text in the output unit, and outputs the The output unit may be controlled to output the digitized non-formal text from the unit.

The digital conversion step S1400 will be described in more detail with reference to FIGS. 10 and 11.

10 is a flowchart illustrating a series of processes in which the digital conversion step S1400 is performed according to an embodiment of the present invention, and FIG. 11 is an example of the digital conversion step S1400 according to an embodiment of the present invention. It is a schematic diagram illustrating.

As shown in FIGS. 10 and 11, the digital conversion step (S1400) is performed by the control unit in a first conversion output step (S1410), candidate group temporary output step (S1420), a change output step (S1430), and a determined output step (S1440). ) To be executed.

First, the first transform output step S1410 is performed by the controller to convert the handwritten data into a digital equation corresponding to an equation structure tree having the highest match rate.

The candidate group temporary output step (S1420) may include at least one next-order digital equation corresponding to the equation structure tree of the next-order match rate in a neighboring output area of the digital equation output from the first transformed output step by the controller. This is a step for outputting for a set time.

At this time, as shown in (a) of FIG. 11, the control unit proceeds with the first transformed output step so that the digital equation (P) corresponding to the equation structure tree of the highest match rate in the input / output layer (m) of the output unit. The output unit is controlled to output.

The controller controls the output unit to output the at least one next-order digital equation in the input / output layer m of the output unit for a preset time by performing the candidate group temporary output step.

Next, as illustrated in FIG. 10, when the selection signal for the at least one or more next-order digital equations is received by the control unit, the change output step (S1430) may be changed to the next-order digital equations selected by the control unit. This is the step that proceeds to output.

In particular, when the change output step (S1430) proceeds, when a selection signal for the at least one or more next-order digital equations obtained from the input unit is received within a predetermined time period, the control unit outputs the next-order digital equation. Proceed to change the output.

In addition, when the selection signal for at least one or more next-order digital equations is not received within the preset time, the deterministic output step (S1440) is finally determined by outputting and storing the digital equations in the first conversion output step. to be.

At this time, as shown in (b) of FIG. 11, the control unit finally finalizes the digital equation of the first transformed output step, thereby digitally verifying the final transformed output step of the first transformed output step in the input / output layer m of the output unit. Controls the output of equations.

The controller may control not only the digital equations but also the digital texts to be output at the same time, thereby finally outputting the digital equations and the digital texts to the output unit in a mixed state.

Through this, by outputting and converting continuous digital equations and digital texts with respect to the text and equations, it is possible to provide an effect of further improving the user's input convenience.

In addition, the control unit stores the digital equation of the first transform output step to the memory unit side, and then can be received again the digital equation of the first transform output step, the digital of the handwritten text and equation of the present invention The effect of improving the speed of the output method can be provided.

While the present invention has been described with reference to the embodiments shown in the drawings, it is only for the purpose of illustrating the invention, and those skilled in the art to which the present invention pertains various modifications or equivalents from the detailed description of the invention. It will be appreciated that one embodiment is possible.

Therefore, the true scope of the present invention should be determined by the technical spirit of the claims.

100: control unit
200: input unit
300: output unit
400: communication unit
500: memory
S1000: Receiving Handwritten Data
S1100: Unit handwriting information type identification step
S1200: Interaction judgment step
S1300: equation tree determination step
S1400: Digital conversion step

Claims (10)

As a digital output method of handwritten text and equations,
Handwriting data receiving step of receiving handwriting data from a user through an input unit;
Determining the type of unit handwriting information of each of the plurality of handwriting information included in the handwritten data, and the unit handwriting information type identification step of dividing the unit handwriting information with the pre-stored symbol data into a non-modal text type or mathematical symbol type ;
A mutual relationship determining step of determining mutual relations between neighboring units of handwritten information in the handwritten data;
Based on the result of the unit handwriting information type identification step and the correlation determination step, when the type of the handwriting information is the mathematical symbol type, mathematics that can be combined into the plurality of handwriting information included in the handwriting data An equation tree determination step of determining a final equation tree among equation trees having an equation structure; And
And a digital conversion step of converting the handwritten data into an equation corresponding to the final equation tree and visually outputting it to an output unit based on the final equation tree determined in the equation tree determination step.
The digital conversion step,
If the unit symbol information for forming a mathematical tree structure is not found among the plurality of unit handwritten information, the handwritten data is recognized as non-formal text without a formula and the handwritten data is converted into digitized non-formal text. Handwritten text and digital output method characterized in that it is executed to output. The method of claim 1,
The input unit may be a touch screen or a handwriting recognition panel that receives a user's handwriting trace as an input signal in a pressure sensitive or capacitive manner.
The handwritten data receiving step,
Handwriting text and mathematical digital output method characterized in that the execution after the predetermined time elapses after receiving the handwritten data from the user through the input unit to determine the handwritten data to be recognized. The method of claim 1,
The input unit includes any one or more selected from a camera module, an optical character recognizer (OCR), a scanner,
The handwritten data receiving step,
The digital output method of handwritten text and equations, characterized in that the image information taken and obtained through the input unit is determined to be handwritten data. The method according to any one of claims 1 to 3,
The unit handwriting information type identification step,
A unit handwriting information detecting step of detecting a plurality of unit handwritten information which is handwritten information included in the handwritten data, respectively;
A boundary division step of setting a unit area so that each of the unit handwritten information is accommodated in a preset unit area, and dividing the unit area into a plurality of pixels;
A retention information extraction step of extracting retention information of the unit note information for each pixel for each unit note information;
Comparing the characteristic information for each digital sign information by comparing the retention information of each unit note information and the characteristic information for each digital sign information stored in the memory unit;
A digital symbol information determining step of determining respective digital symbol information corresponding to the respective unit handwriting information based on the result value of the feature information comparison step for each digital symbol information;
A digital identification information outputting step of outputting digital identification information of the digital symbol information; And
Handwritten text, characterized in that it is executed after the digital identification information outputting step, the type classification step of dividing the type of digital symbol information output through the digital identification information output step into a non-formal text type or a mathematical symbol type; And digital output method of equation. The method of claim 4, wherein
Retention information of the unit handwriting information or feature information for each digital symbol information,
And at least one of information about absolute coordinates, relative coordinates, strokes, curvatures, area values, centers of gravity, correlations, and arbitrary feature values of the retained information included in the unit area. Digital output method of handwritten text and mathematical expression. The method of claim 4, wherein
The mutual relationship determination step,
A relationship analysis area setting step of grouping a plurality of neighboring unit handwriting information among the plurality of unit handwriting information into a plurality of units, and setting up a relationship analysis area for accommodating a plurality of grouped handwriting information;
A boundary division step of dividing the relationship analysis region into a plurality of pixels;
Retention information is extracted for each relationship analysis area for each pixel, and retention information is extracted between a plurality of units of handwritten information in the relationship analysis area based on the extracted retention information of the relationship analysis area. Relationship information extraction step;
Comparing the feature information for each piece of digital relationship information between the symbol information for comparing the retention relationship information of the plurality of pieces of unit information with the feature information for each piece of digital relationship information stored in a memory unit;
A symbol-to-symbol digital relationship information determining step of determining symbol-to-symbol digital relationship information corresponding to respective retention relationship information based on a result value of the feature information comparison step for each symbol-to-digital relationship information; And
And digital identification information outputting step of outputting digital identification information of the inter-symbol digital relationship information. The method of claim 6,
The equation tree determination step,
A first calling step of calling digital identification information of all digital symbol information output in the unit handwriting information type identification step;
A second calling step of calling the digital identification information of the digital relation information between the entire symbols output in the correlation determination step;
A mathematical tree generation step of generating a plurality of mathematical structure trees possible by combining the information of the first calling step and the information of the second calling step;
A matching rate evaluation step of evaluating a matching rate with handwritten data for each of the plurality of mathematical structure trees; And
And a formula structure tree determining step of determining a formula structure tree having the highest match rate as a final formula structure tree in the matching rate evaluating step. The method of claim 7, wherein
The digital conversion step,
And a first transformed output step of converting the handwritten data into a digital equation corresponding to an equation structure tree having the highest match rate. The method of claim 8,
The digital conversion step,
A candidate group temporary output step of outputting at least one or more next-order digital equations corresponding to the equation structure tree of the next-order matching rate in the output region of the digital equation output in the first transformed output step for a predetermined time; and
A change output step of changing and outputting the selected next-order digital equation when the selection signal for the at least one next-order digital equation is received within the preset time; and
And if the selection signal for the at least one or more next-order digital equations is not received within the preset time period, finally determining, outputting and storing the digital equations in the first transformed output step. Digital output method of handwritten text and mathematical expressions. delete

Images