JP5703125B2 - Display device and display program - Google Patents

Description

  The present invention includes a touch panel formed integrally with a display panel, and an image representing a document image based on a line segment input by handwriting on the touch panel so as to overlap image data representing the document image displayed on the display panel The present invention relates to a display device and a display program capable of generating a curved line segment filled with a color and transparency so as to transmit data and displaying the curved line segment on a display panel.

  Electronic blackboard systems, personal computers, mobile phone terminals, electronic notebooks, etc. include handwritten characters that include a display panel such as a liquid crystal display and a touch panel formed integrally with the display panel. Is provided. The display device acquires image data representing the document image based on the input coordinate data of the locus handwritten by an input operation on the touch panel, and displays the acquired image data representing the document image on the display panel. The display device acquires an electronic document file such as a PDF data file as image data representing a document image, and displays the obtained image data representing the document image on a display panel.

  In such a display device, a technique has been proposed in which a modification symbol for attention is added to a desired portion of image data representing a document image displayed on a display panel.

  For example, Patent Document 1 discloses a technique related to a modification editing function that adds a line segment that fills a character string to a character string portion desired by a user of image data representing a document image displayed on a display panel. Has been. According to the technique disclosed in Patent Document 1, an arbitrary character is input by handwriting a line segment via a touch panel corresponding to an arbitrary character string of image data representing a document image displayed on a display panel. A straight line segment is displayed so as to overlap the image of the column and transmit the image of the character string.

  In this way, the user can simply display a handwritten line segment as a straight line segment without damaging the visibility of the character string by simply inputting the line segment by handwriting in response to an arbitrary character string desired. it can. Therefore, it is possible to modify and edit a character string so that a straight line is overwritten on a given character string using a ruler with a highlighter.

JP 2001-282785 A

  The technique disclosed in Patent Document 1 modifies an arbitrary character string arranged in a straight line by adding a straight line segment. For this reason, in the technique disclosed in Patent Document 1, image data representing a document image based on a locus handwritten on the touch panel, image data representing a document image based on an electronic document file such as a PDF data file, and the like are used. The character strings that are not arranged in a straight line cannot be modified and edited as if they were overwritten with a highlighter.

  Accordingly, an object of the present invention is to provide a display device capable of modifying and editing a character string arranged in a curved line as if it was overwritten with a highlighter in image data representing a document image displayed on a display unit. It is to provide a display program.

The present invention includes a document image data acquisition unit that acquires image data representing a document image;
A display unit for displaying a document image based on image data representing the document image acquired by the document image data acquisition unit;
An input unit that is laminated on the surface of the display unit, is formed integrally with the display unit, and outputs coordinate data representing a position of a point designated by handwriting;
Based on the coordinate data corresponding to the locus of handwriting input output from the input unit, the input line segment input by handwriting so as to overlap the document image displayed on the display unit is recognized, and both end points of the input line segment An input line segment recognition unit for acquiring the coordinate data of
Based on the coordinate data of both end points of the input line segment, a character area is extracted from a document image overlapping the input line segment, a character string included in the extracted character area is recognized, and a character trajectory for each character A character recognition unit for obtaining coordinate data representing
Based on the coordinate data for each character acquired by the character recognition unit, two characters that are closest to each of both end points of the input line segment among characters constituting the character string recognized by the character recognition unit A character extraction unit that extracts arbitrary characters including a first end character and a second end character, sets a circumscribed rectangle of each extracted character, and obtains coordinate data of the circumscribed rectangle;
A first tangent line segment and a second tangent line segment that are two tangent line segments for each character, each including two sides extending in the character array direction of the character string in the circumscribed rectangle of each character extracted by the character extraction unit. A character tangent generator for generating each character,
A first curve that is approximated by a curve so as to pass through an arbitrary point on each first tangent line segment for each character generated by the character tangent line generation unit, and an arbitrary one on each second tangent line segment for each character An approximate curve generation unit that generates a second curve that is approximated to pass through a point;
A band-shaped area having a width defined between the first curve and the second curve generated by the approximate curve generation unit is generated, and the generated band-shaped area is transmitted in a color that transmits the document image. It is a display apparatus characterized by including the strip | belt-shaped area | region production | generation part displayed on a display part.

  In the display device of the present invention, the document image data acquisition unit acquires image data representing a document image based on coordinate data corresponding to a locus of handwritten input to the input unit.

  In the display device according to the aspect of the invention, the approximate curve generation unit generates an approximate curve that passes through both end points of each first tangent line for each character generated by the character tangent line generation unit as the first curve. An approximate curve passing through both end points of each second tangent line segment is generated as the second curve.

  In the display device according to the aspect of the invention, the belt-shaped region generation unit may include a straight line passing through an outer end point of the first tangent line segment and the second tangent line segment corresponding to the first end character, A straight line passing through the outer end points of the first tangent line segment and the second tangent line segment corresponding to the end character, the first curve, and a region surrounded by the second curve are generated as a band-like region. It is characterized by doing.

  According to another aspect of the present invention, there is provided a display program that causes a computer including an input unit and a display unit to function as the display device.

  According to the present invention, the display device is a device that displays a document image on the display unit based on image data representing the document image acquired by the document image data acquisition unit. Furthermore, the display device includes an input unit, an input line segment recognition unit, a character recognition unit, a character extraction unit, a character tangent line generation unit, an approximate curve generation unit, and a band-shaped region generation unit.

  The input unit is laminated on the surface of the display unit and is formed integrally with the display unit, and outputs coordinate data representing the position of a point designated by handwriting input. The input line segment recognition unit recognizes the input line segment input by handwriting and inputs the line segment when the user inputs the line segment to the input unit so as to overlap the document image displayed on the display unit, for example. Get the coordinate data of both end points of the line segment. The character recognition unit extracts a character area from the document image overlapping the input line segment based on the coordinate data of both end points of the input line segment, recognizes a character string included in the extracted character area, and Get coordinate data representing the trajectory of a character. The character extraction unit is the first closest to each of both end points of the input line segment among the characters constituting the character string recognized by the character recognition unit based on the coordinate data for each character acquired by the character recognition unit. An arbitrary character including the end character and the second end character is extracted, a circumscribed rectangle of each extracted character is set, and coordinate data of the circumscribed rectangle is acquired.

  The character tangent generation unit generates a first tangent line segment and a second tangent line segment, which are two tangent line segments for each character extracted by the character extraction unit, for each character. The first tangent line segment and the second tangent line segment are line segments each including two sides extending in the character arrangement direction of the character string in the circumscribed rectangle of each character. The approximate curve generation unit includes a first curve that is approximated so as to pass an arbitrary point on each first tangent line segment for each character generated by the character tangent line generation unit, and each second tangent line for each character. A second curve approximated to pass through an arbitrary point on the minute is generated. Then, the band-shaped area generation unit generates a band-shaped area having a width defined between the first curve and the second curve generated by the approximate curve generation unit, and transmits the document image through the generated band-shaped area. Display on the display unit in various colors.

  In the display device of the present invention, the first curve and the second curve generated by the approximate curve generation unit are the first end characters among the characters constituting the character string overlapping the input line segment input by handwriting in the input unit. And a curved line extending along the character arrangement direction in contact with any plurality of characters including the second end character. The band-shaped area generating unit generates a band-shaped area whose width is defined between the first curve and the second curve as described above, and has a color and transparency that allows the document image to pass through the generated band-shaped area. To display on the display. Thereby, in the document image displayed on the display unit, the character strings arranged in a curved line can be modified and edited as if they were overwritten with the highlighter pen.

  According to the invention, the document image data acquisition unit acquires image data representing the document image based on the coordinate data corresponding to the locus of handwriting input to the input unit. Image data representing a document image based on a locus obtained by handwriting input by a user input operation on the input unit, acquired by the document image data acquisition unit, is displayed on the display unit.

  In the display device of the present invention, the band-shaped area is generated by the band-shaped area generation unit based on the input line segment input by handwriting so as to overlap the document image displayed on the display unit as described above. The generated band-shaped area includes a first curve and a second curve that are in contact with any of a plurality of characters including a first end character and a second end character among characters constituting the character string overlapping the input line segment. This is a region that is filled with a color and transparency such that the width is defined between the two and the document image is transmitted. In this way, the display device modifies and edits the character string arranged in a curved line included in the document image based on the locus handwritten to the input unit as if it was overwritten with the highlighter pen. Can do.

  According to the invention, the approximate curve generation unit generates an approximate curve that passes through both end points of each first tangent line segment for each character as a first curve, and both ends of each second tangent line segment for each character. An approximate curve passing through the points is generated as the second curve. In this manner, the approximate curve generation unit includes a plurality of arbitrary plural characters including the first end character and the second end character among the characters constituting the character string overlapped with the input line segment input by handwriting in the input unit. A first curve and a second curve extending along the character arrangement direction can be generated in contact with the character.

  According to the invention, the band-shaped region generating unit corresponds to a straight line passing through the outer end points of the first tangent line segment and the second tangent line segment corresponding to the first end character, and the second end character. A region surrounded by a straight line passing through the outer end points of the first tangent line segment and the second tangent line segment, the first curve, and the second curve is generated as a band-shaped region. In this manner, the band-shaped area generation unit generates a band-shaped area whose width is defined between the first curve and the second curve, and has a color and transparency that allows the document image to pass through the generated band-shaped area. It can be displayed on the display unit.

  According to the invention, the display program is a program for causing a computer including an input unit and a display unit to function as the display device. Such a display program uses a software to perform a display process that allows a text image arranged in a curved line in a document image displayed on a display unit to be modified and edited as if it was overwritten with a highlighter. Can be controlled.

It is a block diagram which shows the structure of the display apparatus 1 which concerns on one Embodiment of this invention. 3 is a flowchart showing the operation of the display device 1. 4 is a flowchart illustrating an operation of input line segment recognition processing in the display device 1. 6 is a diagram for explaining an operation of input line segment recognition processing in the display device 1. FIG. 4 is a flowchart showing an operation of a modification editing range setting process in the display device 1. FIG. 10 is a diagram for explaining an operation of a modification editing range setting process in the display device 1. 4 is a flowchart showing an operation of character recognition processing in the display device 1. FIG. 6 is a diagram for explaining an operation of character recognition processing in the display device 1. FIG. 6 is a diagram for explaining an operation of character recognition processing in the display device 1. FIG. 6 is a diagram for explaining an operation of character recognition processing in the display device 1. 4 is a flowchart showing an operation of character extraction processing in the display device 1. 6 is a diagram for explaining an operation of a character extraction process in the display device 1. FIG. 4 is a flowchart showing an operation of a tangent line generation process in the display device 1. 6 is a diagram for explaining an operation of a tangent line generation process in the display device 1. FIG. 4 is a flowchart showing an operation of approximate curve generation processing in the display device 1. 6 is a diagram for explaining an operation of an approximate curve generation process in the display device 1. FIG. 4 is a flowchart showing an operation of a band-shaped area generation process in the display device 1. 6 is a diagram for explaining an operation of a band-shaped area generation process in the display device 1. FIG.

  FIG. 1 is a block diagram showing a configuration of a display device 1 according to an embodiment of the present invention. The display device 1 includes a transparent touch panel 2 as an input unit, a display panel 3 as a display unit, a touch panel data processing unit 4, a display data processing unit 5, a CPU 6 as control means, a memory, and the like. First storage means 7 and second storage means 8 composed of an HDD or the like. The display device 1 includes document data based on electronic document files such as image data (hereinafter referred to as “document image data”) representing a document image based on a trajectory input by handwriting on the touch panel 2 or a PDF data file. Is displayed on the display panel 3.

  In the display device 1 of the present embodiment, the touch panel 2 has an input surface in which orthogonal coordinates are set, and is laminated on the surface of the display panel 3 and is formed integrally with the display panel 3. This touch panel 2 is input by handwriting on the input surface and outputs coordinate data representing the input coordinate points. The touch panel data processing unit 4 is realized by a touch panel driver, and the display data processing unit 5 is realized by a display driver.

  As an input method other than the above, it is possible to use an infrared blocking method that blocks infrared rays, a capacitive coupling method, an electromagnetic induction method, a camera method, or the like.

  The CPU 6 includes a document image data acquisition unit 60, a coordinate data acquisition unit 61, a locus display instruction unit 62, an input line segment recognition unit 63, a modified editing range setting unit 64, a character recognition unit 65, and a character extraction unit. 66, a character tangent line generation unit 67, an approximate curve generation unit 68, and a band-shaped region generation unit 69.

  The document image data acquisition unit 60 acquires document image data based on the coordinate data corresponding to the locus input by handwriting on the touch panel 2. The document image data acquisition unit 60 acquires document image data based on an electronic document file such as a PDF data file. A document image is displayed on the display panel 3 based on the document image data acquired by the document image data acquisition unit 60.

  The coordinate data acquisition unit 61 acquires coordinate data output from the touch panel 2. The trajectory display instruction unit 62 transmits handwritten input coordinate data input to the touch panel 2 to the display panel 3 and causes the display panel 3 to display a handwritten input trajectory based on the coordinate data.

  The input line segment recognition unit 63 recognizes an input line segment input by handwriting so as to overlap the document image displayed on the display panel 3 based on the coordinate data corresponding to the locus of handwriting input output from the touch panel 2. The coordinate data of both end points of the input line segment is acquired. The input line segment recognized by the input line segment recognition unit 63 is set by an input operation on the input surface of the touch panel 2 by the user.

  Based on the coordinate data of the both end points of the input line segment acquired by the input line segment recognition unit 63, the modified edit range setting unit 64 is a specific area that overlaps the input line segment in the document image displayed on the display panel 3. Set the qualifying edit range indicating the range of. The modification edit range set by the modification edit range setting unit 64 is a band-shaped area generated by a band-shaped area generation unit 69 described later, and becomes a target area to be modified and edited as if overwritten with a highlighter.

  The character recognizing unit 65 extracts a character area from the document image in the modified editing range set by the modified editing range setting unit 64, recognizes a character string included in the extracted character area, and obtains a character trajectory for each character. Get representing coordinate data. The character recognition unit 65 recognizes characters included in the character area using OCR (Optical Character Recognition) technology. The character recognition unit 65 sets a circumscribed rectangle for each recognized character, and acquires coordinate data of the circumscribed rectangle, inclination of each character with respect to the X axis, and size data of each character.

  The character extraction unit 66 is closest to each of the end points of the input line segment among the characters constituting the character string recognized by the character recognition unit 65 based on the coordinate data for each character acquired by the character recognition unit 65. An arbitrary character including the first end character and the second end character to be extracted is extracted, a circumscribed rectangle of each extracted character is set, and coordinate data of the circumscribed rectangle is acquired. In addition, the character extracting unit 66 acquires the inclination of each extracted character and the size data of each character.

  The character tangent line generation unit 67 generates a first tangent line segment and a second tangent line segment that are two tangent line segments for each character extracted by the character extraction unit 66 for each character. The first tangent line segment and the second tangent line segment are line segments each including two sides extending in the character arrangement direction of the character string in the circumscribed rectangle of each character.

  The approximate curve generation unit 68 includes a first curve that is approximated by a curve so as to pass through an arbitrary point (control point) on each first tangent line segment for each character generated by the character tangent line generation unit 67, and for each character. A second curve that is approximated to pass through an arbitrary point (control point) on each second tangent line segment is generated.

  The band-shaped area generation unit 69 generates a band-shaped area having a width defined between the first curve and the second curve generated by the approximate curve generation unit 68, and transmits the document image through the generated band-shaped area. Display on the display panel 3 with various colors and transparency.

  In the display device 1 of the present embodiment, the first curve and the second curve generated by the approximate curve generation unit 68 are the first of the characters constituting the character string overlapping the input line segment input by handwriting on the touch panel 2. The curved line extends along the character arrangement direction in contact with a plurality of arbitrary characters including the end character and the second end character. Then, the band-shaped area generator 69 generates a band-shaped area whose width is defined between the first curve and the second curve as described above, and a color that allows the document image to pass through the generated band-shaped area, The display panel 3 is displayed with transparency. As a result, in the document image displayed on the display panel 3, the character strings arranged in a curved line can be modified and edited as if they were overwritten with the highlighter.

  The first storage means 7 has a function as a working memory in the display device 1. The first storage means 7 includes a display information storage unit 70, a coordinate data storage unit 71, a recognized character information storage unit 72, a recognized character number storage unit 73, an extracted character information storage unit 74, and a tangent line segment information storage unit. 75, an approximate curve generation control point storage unit 76, a standard character size information storage unit 77, and a character sample information storage unit 78.

  The display information storage unit 70 stores display information displayed on the display panel 3. The coordinate data storage unit 71 stores input line segment coordinate data output from the touch panel 2 and coordinate data based on a handwritten input locus. The recognized character information storage unit 72 stores the information of the recognized character such as the coordinate data of the circumscribed rectangle of the recognized character acquired by the character recognition unit 65, the character inclination with respect to the X axis, and the size data. The recognized character number storage unit 73 stores the number of characters recognized by the character recognition unit 65. The extracted character information storage unit 74 stores extracted character information such as coordinate data of a circumscribed rectangle of the extracted character acquired by the character extraction unit 66, character inclination with respect to the X axis, and size data.

  The tangent line segment information storage unit 75 stores the coordinate data of the first tangent line segment and the second tangent line segment generated by the character tangent line generation unit 67 and corresponding to each extracted character. The approximate curve generation control point storage unit 76 uses arbitrary control points located on the first tangent line segment and the second tangent line segment that are used when the approximate curve generation unit 68 generates the first curve and the second curve. Is stored. The standard character size information storage unit 77 stores information related to a predetermined standard character size (height and width) used when the modified edit range setting unit 64 sets the modified edit range. The character sample information storage unit 78 stores information related to the shape of the character sample for comparison, which is used when the character recognition unit 65 executes the character recognition process.

  The second storage unit 8 includes a software storage unit 80, a driver storage unit 81, and a user setting display screen storage unit 82. The software storage unit 80 stores software for operating the display device 1. The driver storage unit 81 stores a touch panel driver that implements the touch panel data processing unit 4 and a display driver that implements the display data processing unit 5. The user setting display screen storage unit 82 stores information related to the setting of the display panel 3 set by the user who operates the display device 1.

  Next, the operation of the display device 1 according to the present embodiment will be described with reference to FIG. FIG. 2 is a flowchart showing the operation of the display device 1.

  In step s1, the document image data acquisition unit 60 acquires document image data based on, for example, coordinate data corresponding to a locus input by handwriting on the touch panel 2. A document image is displayed on the display panel 3 based on the document image data acquired by the document image data acquisition unit 60.

  Next, in step s2, after an instruction signal indicating an instruction to execute the modification editing is input via the touch panel 2, the display device 1 performs an input line segment operation by performing a handwriting input input operation on the touch panel 2. Perform recognition processing.

  FIG. 3 is a flowchart showing the operation of the input line segment recognition process in the display device 1. FIG. 4 is a diagram for explaining the operation of the input line segment recognition process in the display device 1.

  As shown in FIG. 4A, the input line segment recognition unit 63 performs a handwriting input input operation on the input surface 21 of the touch panel 2 in a state where a document image is displayed on the display surface 31 of the display panel 3. Thus, the input line segment recognition process is performed. In FIG. 4A, a document image composed of a character string “mark a character to be emphasized” based on a locus input by handwriting is displayed on the display surface 31 of the display panel 3. Each character in the character string “mark the character to be emphasized” is arranged in a curved shape so as to be arranged on the lower side in the Y-axis direction from the X-axis left end side to the X-axis right end side of the display surface 31. It is arranged.

First, in step S2-1, the input line recognition unit 63 determines whether or not the handwriting input of the input line segment L ₁ with respect to the input surface 21 of the touch panel 2 is started. Input line recognition unit 63 determines that the coordinate data corresponding to the input starting point T ₁ of the handwriting input when it is output from the touch panel 2, handwriting input of the input line segment L ₁ is started. If the if the input line recognition unit 63 determines that the handwriting input line segment L ₁ is started, the process proceeds to step S2-2, it is determined that the handwriting input line segment L ₁ is not started Step s2-1 is repeated. Note that coordinate data corresponding to the input start point T ₁ output from the touch panel 2 is stored in the coordinate data storage unit 71.

In step S2-2, the input line recognition unit 63 obtains the coordinate data corresponding to the input starting point T ₁ from the coordinate data storing unit 71.

Next, at step S2-3, the input line recognition unit 63 determines whether or not the handwriting input of the input line segment L ₁ with respect to the input surface 21 of the touch panel 2 is completed. Input line recognition unit 63 judges that the input operation of the input surface 21 of the touch panel 2 even after the lapse of predetermined interval of time is when not done, handwriting input of the input line segment L ₁ is completed. If the input line recognition unit 63 proceeds to step s2-4 if handwriting input of the input line segment L ₁ is determined to have ended, handwriting input line segment L ₁ is determined not to be completed Step s2-3 is repeated. The coordinate data corresponding to the locus handwritten input of the input line segment L _1, the coordinate data of the input end point T ₂ it is also stored in the coordinate data storage unit 71 included.

In step S2-4, the input line recognition unit 63 obtains the coordinate data corresponding to the input end point T ₂ from the coordinate data storing unit 71.

As described above, the input line segment recognition unit 63, as shown in FIG. 4B, the input line segment L ₁ that is input by handwriting so as to overlap the document image displayed on the display surface 31 of the display panel 3. recognizes, acquires coordinate data of a both end points of the input line input start point T _1, and the coordinate data of the input end point T _2.

  Returning to FIG. 2, when the input line segment recognition process is completed in step s2, the process proceeds to step s3. In step s3, the display device 1 performs a modified editing range setting process.

FIG. 5 is a flowchart showing the operation of the modification editing range setting process in the display device 1. FIG. 6 is a diagram for explaining the operation of the modification editing range setting process in the display device 1. The modification edit range setting unit 64 displays the display panel 3 based on the coordinate data of both end points (input start point T ₁ and input end point T ₂ ) of the input line segment L ₁ acquired by the input line segment recognition unit 63. among the displayed document image on the surface 31, to set the overlapped modified edit selection and input line L _1.

  First, in step s 3-1, the modification edit range setting unit 64 reads standard character size information from the standard character size information storage unit 77. The width size (size in the X-axis direction) of the standard character is “Sx”, and the height size (size in the Y-axis direction) is “Sy”.

Next, at step S3-2, modified editing range setting unit 64, as shown in FIG. 6 (a), "Sx / 2" from the input start point _{T 1} in the X-axis direction outward, Y-axis upward direction The position moved by “Sy / 2” to the side is set as the first reference point R ₁ . Next, at step S3-3, modified editing range setting unit 64, "Sx / 2" from the input end point T ₂ in the X-axis direction outward and moves by "Sy / 2" in the Y axis direction lower side to set position as the second reference point R _2.

Then, in step S3-4, modified editing range setting unit 64, as shown in FIG. 6 (b), a rectangular area to the first reference point R ₁ and the second reference point R ₂ and diagonal apexes of The modification editing range R is set, and the modification editing range setting process is terminated.

  Returning to FIG. 2, when the modification edit range setting process ends in step s3, the process proceeds to step s4. In step s4, the display device 1 performs a character recognition process.

  FIG. 7 is a flowchart showing the operation of character recognition processing in the display device 1. 8A to 8C are diagrams for explaining the operation of the character recognition process in the display device 1. The character recognizing unit 65 extracts a character region from the document image within the modified editing range R set by the modified editing range setting unit 64, and recognizes a character string included in the extracted character region.

First, in step S4-1, the character recognition unit 65, as shown in FIG. 8A (a), to the peripheral edge right end of the display surface 31 from the input start point _{T 1,} generates a horizontal line _{L 2} extending parallel to the X axis To do. The character recognition unit 65, as shown in FIG. 8A (b), to a position where the horizontal line L ₂ does not overlap with the characters in the modified edit range R, to move the horizontal line L ₂ in the Y-axis direction upper side. As a result, the horizontal line L ₂ is a line extending in parallel with the X axis from the point T ₃ to the right edge of the peripheral edge of the display surface 31.

Next, in step s4-2, the character recognition unit 65, as shown in FIG. 8A (c), about the point T ₃ horizontal lines L ₂ until it contacts one of the characters and the point within the modified edit range R It is rotated to generate a character recognition reference line L _3. Next, at step S4-3, the character recognition unit 65, as shown in FIG. 8B (d), through the point _{T 3,} to produce a line perpendicular _{L 4} in the character recognition reference line _{L 3.}

Next, at step S4-4, the character recognition unit 65, as shown in FIG. 8B (e), the vertical line L _4, is translated toward the peripheral right end of the display surface 31 so that the inclination does not change while, calculates the number of intersections a ₁ of the perpendicular L ₄ and characters. Next, at step S4-5, the character recognition unit 65 creates a histogram based on the number of intersections A ₁ of the perpendicular L ₄ and character binarizes the histogram.

  Next, in step s4-6, the character recognizing unit 65 separates the document image in the modified editing range R for each character based on the binarization result as shown in FIG. 8B (f).

  In step s4-7, as shown in FIG. 8C (g), the character recognition unit 65 performs character recognition on the separated characters, acquires coordinate data representing the trajectory of each character, and recognizes it. Get information about the characters that were created. Specifically, as shown in FIG. 8C (h), the character recognizing unit 65 reads a standard character sample for comparison corresponding to each separated character from the character sample information storage unit 78, and reads the character. Recognize recognized character information by comparing the sample and each separated character. First, the character recognition unit 65 matches each separated character while rotating the character sample, and calculates the inclination angle of the most matched character sample with respect to the X axis as the inclination “Ri” of each separated character. . Next, the character recognizing unit 65 changes the size of the character sample for each separated character, and sets the size of the character sample most matched to the size data (width size Wi, height size) of each separated character. Calculated as Hi).

Using the calculated character inclination “Ri” and character size data, the character recognition unit 65 sets a circumscribed rectangle of each separated character, and acquires coordinate data of the circumscribed rectangle. The character recognition unit 65 uses, as coordinate data of the circumscribed rectangle, four vertices of the circumscribed rectangle, that is, an apex P _U1 on the X axis left end side of the upper side, an apex P _U2 on the right end side of the X axis on the upper side, and an X axis on the lower side The coordinate data of the apex P _L1 on the left end side and the apex P _L2 on the X axis right end side on the lower side is acquired.

  The recognized character information such as the character inclination “Ri”, the character size data, and the circumscribed rectangle coordinate data acquired by the character recognition unit 65 is stored in the recognized character information storage unit 72.

  Returning to FIG. 2, when the character recognition process ends in step s4, the process proceeds to step s5. In step s5, the recognized character number storage unit 73 stores the number of characters recognized by the character recognition unit 65. For example, when the character recognition unit 65 recognizes a character for each character from the document image data “mark the character to be emphasized”, “strong”, “tone”, “shi”, “ta”, “i” ”,“ Sentence ”,“ Character ”,“ O ”,“ Ma ”,“-”, and“ K ”, a total of 11 characters are recognized. 11 "is stored.

  Next, in step s6, the display device 1 performs a character extraction process. FIG. 9 is a flowchart showing the operation of the character extraction process in the display device 1. FIG. 10 is a diagram for explaining the operation of the character extraction process in the display device 1. The character extraction unit 66 extracts an arbitrary character from the characters constituting the character string recognized by the character recognition unit 65.

First, in step S6-1, the character extraction unit 66, as shown in FIG. 10, in the character of the character recognition unit 65 recognizes, first end which is closest to the input start point T ₁ of the input line segment L ₁ extracting a part character _{C 1.} In Figure 10, the first end character _{C 1} is "strong".

Next, in step s6-2, the character extraction unit 66, in the character the character recognition unit 65 recognizes, the second end character C _n closest to the input end point T ₂ of the input line segment L ₁ extract To do. In FIG. 10, the second end character C _n is “ku”.

  Next, in step s6-3, the character extraction unit 66 reads information on the number of recognized characters from the recognized number-of-characters storage unit 73, and determines whether or not the number of recognized characters “n” is equal to or less than a predetermined number of characters “N”. If the character extraction unit 66 determines that the number of characters is “N” or less, the process proceeds to step s6-4, and if it is determined that the number of characters exceeds “N”, the process proceeds to step s6-5.

In step S6-4, the character extraction unit 66 extracts from the second character of the character recognition result to (n-1) th character as an intermediate character _{C S.}

In step s6-5, the character extraction unit 66 selects (N−2) characters so that the number of characters between the second character to the (n−1) th character in the character recognition result is equal. Are extracted as intermediate characters Cs. For example, a total of 11 “strong”, “tone”, “shi”, “ta”, “i”, “sentence”, “character”, “wo”, “ma”, “-”, and “ku” When (recognized character number n = 11) characters are arranged and the predetermined character number “N” is “3”, the number of intermediate characters Cs is (N−2) = 1, and the intermediate character C _S "Sentence" is extracted as An intermediate character C _S "statement" is the second character from the "tone" (n-1) of the up character of the "over", between the first is an end character C ₁ "strong" Four characters “tone”, “shi”, “ta”, and “i” are arranged, and “letter”, “o”, “ma” are placed between the second end character C _n “ku”. , And “−” are arranged.

Further, a recognition number "n" is "10", if the number of characters "N" is "3", the character extraction unit 66, (N-2) = as one intermediate character C _S, for example 5 characters Extract eye characters. In this case, three characters are arranged between the first end character C ₁ and the intermediate character C _S, and four characters are arranged between the second end character C _n and the intermediate character C _S. Even in such a case, it is considered that the number of characters between them is equal.

In addition, the character extracting unit 66 extracts the character inclination data “Ri”, character size data (width size Wi, high character) of the extracted first end character C ₁ , second end character C _n , and intermediate character C _S. Information of the extracted character such as the size size Hi) and the coordinate data of the circumscribed rectangle.

  The extracted character information such as the character inclination “Ri”, the character size data, and the circumscribed rectangle coordinate data acquired by the character extraction unit 66 is stored in the extracted character information storage unit 74.

  Returning to FIG. 2, when the character extraction process is completed in step s6, the process proceeds to step s7. In step s7, the display device 1 performs a tangent line segment generation process.

FIG. 11 is a flowchart showing the operation of the tangent line segment generation process in the display device 1. FIG. 12 is a diagram for explaining the operation of the tangent line generation processing in the display device 1. The character tangent line generation unit 67 generates, for each character, a first tangent line segment S ₁ and a second tangent line segment S ₂ that are two tangent line segments for each character extracted by the character extraction unit 66.

  First, in step s <b> 7-1, the character tangent line generation unit 67 reads extracted character information from the extracted character information storage unit 74.

Next, in step s7-2, the character tangent generation unit 67 extends in the character arrangement direction (X-axis direction) of the character string in the circumscribed rectangle of each character extracted by the character extraction unit 66, as shown in FIG. the first tangent line segment S ₁ containing the upper side generates for every character. In the present embodiment, the character tangent line generating unit 67 converts a line segment extending from the vertex P _U1 on the X axis left end side of the upper side of the circumscribed rectangle to the vertex P _U2 on the X axis right end side of the upper side as a first tangent line segment. generated as S _1. The first tangent line segment S ₁ is the length of the line segment is a length corresponding to the width size Wi character, the inclination corresponds to the inclination Ri character.

Next, in step s7-3, the character tangent generation unit 67 includes a second tangent line including a lower side extending in the character arrangement direction (X-axis direction) of the character string in the circumscribed rectangle of each character extracted by the character extraction unit 66. to generate a minute S ₂ on a character-by-character basis. In the present embodiment, the character tangent generation unit 67, from the apex P _L1 in the X-axis left end side of the circumscribed rectangle of the lower side, a line segment extending to the apex P _L2 of the X-axis right end portion side of the lower, second tangent line segments to generate as S _2. The second tangent line segment S _2, the length of the line segment is a length corresponding to the width size Wi character, the inclination corresponds to the inclination Ri character.

The character tangent generation unit 67 acquires the coordinate data of both end points S _U1 and S _U2 of the first tangent line segment S ₁ generated as described above. The end point S _U1 corresponds to the vertex P _U1 , and the end point S _U2 corresponds to the vertex P _U2 . In addition, the character tangent line generation unit 67 acquires coordinate data of both end points S _L1 and S _L2 of the second tangent line segment S ₂ . The end point S _L1 corresponds to the vertex P _L1 , and the end point S _L2 corresponds to the vertex P _L2 .

Note that the coordinate data of both end points S _U1 and S _U2 of the first tangent line segment S ₁ and the coordinate data of both end points S _L1 and S _L2 of the second tangent line segment S ₂ acquired by the character tangent line generation unit 67 are approximate. It is stored in the curve generation control point storage unit 76.

  Returning to FIG. 2, when the tangent line generation process is completed in step s7, the process proceeds to step s8. In step s8, the display device 1 performs an approximate curve generation process.

FIG. 13 is a flowchart showing the operation of the approximate curve generation process in the display device 1. FIG. 14 is a diagram for explaining the operation of the approximate curve generation process in the display device 1. The approximate curve generation unit 68 generates a first curve B _U and a second curve B _L.

First, in step S8-1, the approximate curve generating section 68, from the approximate curve generating control point storage unit 76, corresponding to each character the character extraction unit 66 has extracted, end points of the first tangent line segment S ₁ The coordinate data of S _U1 and S _{U2 and} the coordinate data of both end points S _L1 and S _L2 of the second tangent line segment S ₂ are read out.

Next, at step S8-2, the approximate curve generating section 68, as shown in FIG. 14, corresponding to each character the character extraction unit 66 has extracted, each of the first tangential ends of the line segments S ₁ of each character An approximate curve (Bézier curve) passing through the points S _U1 and S _U2 is generated as the first curve B _U.

Then, in step S8-3, the approximate curve generating section 68 corresponds to each character the character extraction unit 66 has extracted, through the end points _{S L1, S L2} of each second tangent line segments per character _{S 2} An approximate curve (Bézier curve) is generated as the second curve _BL .

  Returning to FIG. 2, when the approximate curve generation process is completed in step s8, the process proceeds to step s9. In step s9, the display device 1 performs a band-shaped region generation process.

FIG. 15 is a flowchart showing the operation of the band-shaped area generation process in the display device 1. FIG. 16 is a diagram for explaining the operation of the band-shaped area generation process in the display device 1. The band-shaped area generation unit 69 generates a band-shaped area that has a width defined between the first curve _BU and the second curve _BL generated by the approximate curve generation unit 68 and transmits the document image.

First, in step S9-1, the strip area generation unit 69, as shown in FIG. 16 (a), a first tangent line segment _{S 1} of the end point _{S U1,} a second end point _{S L1} tangent line _{S 2} generating a straight line _{L 5} connecting. Next, at step S9-2, the strip area generator 69 generates a first tangent line segment _{S 1} of the end point _{S U2,} a line _{L 6} connecting the end points _{S L2} of the second tangent line segment _{S 2.}

Then, in step S9-3, the strip area generation unit 69, as shown in FIG. 16 (b), surrounded by a first curve _{B U,} a second curve _{B L,} the straight line _{L 5,} the straight line _{L 6} the region was generated as a band-like region O _L, color like the generated strip-shaped region O _L transmits document images on the display panel 3 in transparency. The band region O _L, the color so as to transmit the document image, a region filled with transparency.

As described above, in the display device 1 of the present embodiment, the first curve B _U and the second curve B _L generated by the approximate curve generating unit 68, overlapped with the input line L ₁ which is handwritten on the touch panel 2 of the characters constituting the character string, the curve in contact with any of a plurality of characters including a first end character C ₁ and the second end letter C _n, extending along the character array direction. Then, the strip area generator 69 generates a band-like region O _L which width is defined between the first curve B _U and the second curve B _L as described above, document the generated strip-shaped region O _L The image is displayed on the display panel 3 with a color and transparency that can transmit the image. As a result, in the document image displayed on the display panel 3, the character strings arranged in a curved line can be modified and edited as if they were overwritten with the highlighter.

  As still another embodiment of the present invention, in order to cause a computer to function as the display device 1 described above, at least one of a display program (an execution format program, an intermediate code program, and a source program) to be executed by the computer. It is also possible to provide a computer-readable recording medium on which the display program is recorded.

  As a recording medium for recording the display program, a memory (not shown) such as a CD-ROM (Compact Disc-Read Only Memory) itself is a program medium because processing is performed by a microcomputer. Alternatively, although not shown, a program reading device may be provided as an external storage device, and the program medium may be read by inserting a recording medium therein.

  In any case, the stored display program may be configured to be accessed and executed by a microprocessor, or in any case, the display program is read and the read display program is a microcomputer. The program may be downloaded to a program storage area (not shown) and the display program is executed. It is assumed that this download program is stored in the main device in advance.

  The program medium is a recording medium configured to be separable from the main body, and is a tape system such as a magnetic tape or a cassette tape, a magnetic disk such as a floppy (registered trademark) disk or a hard disk, or a CD-ROM / MO (Magneto Optical disc). ) / MD (Mini Disc) / DVD (Digital Versatile Disc) optical discs, IC (Integrated Circuit) cards (including memory cards) / Optical cards, etc., Mask ROM, EPROM (Erasable Programmable Read) It may be a medium that carries a fixed program including a semiconductor memory such as an only memory), an EEPROM (registered trademark) (electrically erasable programmable read only memory), or a flash ROM.

  Further, it may be a medium that has a system configuration capable of connecting to a communication network including the Internet so that the display program is fluidly carried so as to download the display program from the communication network. When the display program is downloaded from the communication network as described above, the download program may be stored in the main device in advance or installed from another recording medium. The present invention can also be realized in the form of a computer data signal embedded in a carrier wave in which the display program is embodied by electronic transmission.

DESCRIPTION OF SYMBOLS 1 Display apparatus 2 Touch panel 3 Display panel 60 Document image data acquisition part 61 Coordinate data acquisition part 62 Trajectory display instruction part 63 Input line segment recognition part 64 Modified edit range setting part 65 Character recognition part 66 Character extraction part 67 Character tangent line generation part 68 Approximate curve generator 69 Band-shaped area generator

Claims (5)

A document image data acquisition unit for acquiring image data representing a document image;
A display unit for displaying a document image based on image data representing the document image acquired by the document image data acquisition unit;
An input unit that is laminated on the surface of the display unit, is formed integrally with the display unit, and outputs coordinate data representing a position of a point designated by handwriting;
Based on the coordinate data corresponding to the locus of handwriting input output from the input unit, the input line segment input by handwriting so as to overlap the document image displayed on the display unit is recognized, and both end points of the input line segment An input line segment recognition unit for acquiring the coordinate data of
Based on the coordinate data of both end points of the input line segment, a character area is extracted from a document image overlapping the input line segment, a character string included in the extracted character area is recognized, and a character trajectory for each character A character recognition unit for obtaining coordinate data representing
Based on the coordinate data for each character acquired by the character recognition unit, two characters that are closest to each of both end points of the input line segment among characters constituting the character string recognized by the character recognition unit A character extraction unit that extracts arbitrary characters including a first end character and a second end character, sets a circumscribed rectangle of each extracted character, and obtains coordinate data of the circumscribed rectangle;
A first tangent line segment and a second tangent line segment that are two tangent line segments for each character, each including two sides extending in the character array direction of the character string in the circumscribed rectangle of each character extracted by the character extraction unit. A character tangent generator for generating each character,
A first curve that is approximated by a curve so as to pass through an arbitrary point on each first tangent line segment for each character generated by the character tangent line generation unit, and an arbitrary one on each second tangent line segment for each character An approximate curve generation unit that generates a second curve that is approximated to pass through a point;
A band-shaped area having a width defined between the first curve and the second curve generated by the approximate curve generation unit is generated, and the generated band-shaped area is transmitted in a color that transmits the document image. A display device comprising: a band-shaped region generating unit to be displayed on the display unit.   The display device according to claim 1, wherein the document image data acquisition unit acquires image data representing a document image based on coordinate data corresponding to a locus of handwriting input to the input unit.   The approximate curve generation unit generates an approximate curve that passes through both end points of each first tangent line for each character generated by the character tangent line generation unit as the first curve, and each second tangent line for each character. The display device according to claim 1, wherein an approximate curve passing through both end points of the minute is generated as the second curve.   The band-shaped region generation unit is configured to correspond to the second end character corresponding to the straight line passing through the outer end point of the first tangent line segment and the second tangent line segment corresponding to the first end character. The straight line passing through the outer side end point of the 1 tangent line segment and the second tangent line segment, the area surrounded by the first curve, and the second curve are generated as a band-shaped area. The display device according to any one of 1 to 3.   A display program for causing a computer including an input unit and a display unit to function as the display device according to claim 1.

Images