JP4741363B2 - Image processing apparatus, image processing method, and image processing program - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing method, and an image processing program.

  2. Description of the Related Art Conventionally, there are apparatuses that read a content that has been added to paper on which content of an electronic document is printed, and perform a predetermined process. In such an apparatus, the user can hand-write a symbol or the like on the paper on which the content is printed, and read the paper with a scanner or the like, thereby determining the processing content and the target to be processed. For example, the following documents can be cited.

  Japanese Patent Laid-Open No. 2002-252732 (Patent Document 1) prints a menu sheet having a plurality of processing contents and a check column, adds the menu sheet to the check column of the menu sheet, and performs the menu sheet and processing. There is disclosed a system for inputting processing content desired by a user and content to be processed by causing the scanner to read the paper on which the content is printed.

Japanese Patent Application Laid-Open No. 2004-78973 (Patent Document 2) discloses an example of an apparatus having characters in a document and a database that manages link destinations corresponding to the characters. In this example, first, the user enters a predetermined mark on a paper on which a document and a barcode specifying the document are printed, and selects a character having the link. Next, by reading the paper with a scanner or the like, the position where the predetermined mark is written and the character printed at the position are detected. Then, a document linked to the character is acquired with reference to the database, and printed out.
JP 2002-252732 A JP 2004-78973 A

  However, in the system described in Patent Document 1, the menu paper on which the processing content is written and the content to be processed are input by being read by a scanner or the like. In addition to being able to perform different processing in each part and processing to specify the target range, there were problems such as unable to acquire character images correctly when reading content into text data.

  Further, the apparatus described in Patent Document 2 has a problem that hypertext corresponding to characters and link destinations is necessary in advance, and information that is not linked cannot be acquired.

  The present invention has been invented in order to solve these problems in view of the above points, and in the case of processing the content by printing the content on paper or the like and handwriting on the paper, An object of the present invention is to provide an image processing apparatus capable of designating contents, a processing range, and the like by an arbitrary symbol defined by a user.

  In order to achieve the above object, the image processing apparatus, the image processing method, and the image processing program of the present invention employ the following configurations.

In order to achieve the above object, an image processing apparatus of the present invention includes a content printed on a first paper, paper identification information for identifying the first paper printed on the first paper, and A paper identification information managing means for managing the paper identification information and the content in association with each other in the image processing apparatus for processing the content based on the first symbol handwritten on the first paper; Based on the result of reading a plurality of processing contents printed on the paper and a second symbol handwritten for each processing content, the first symbol handwritten on the first paper and the second symbol A processing symbol management unit that outputs a processing content corresponding to the second symbol for the content printed on the first paper when the symbol has a match, and the processing symbol management unit outputs In the process Based on, possess a process executing means for processing the content corresponding to the paper identification information, the processing execution unit, in response to the second determination of whether a match with the symbols, the a first symbol can Rukoto forming structure to overwrite the contents.

Thus, when processing the content by printing the content on paper and handwriting on the paper, the processing content, processing range, etc. are designated by an arbitrary symbol defined by the user, and the processed content portion and processing It is possible to provide an image processing apparatus that allows the user to easily confirm the contents .

  Note that the processing performed by the image processing apparatus in the claims and in the present specification refers to predetermined processing such as “content editing”, “content output by printing”, or “content playback”. Say.

In order to achieve the above object, the processing execution means included in the image processing apparatus of the present invention overwrites the content with the first symbol determined to be consistent with the second symbol. Can be configured.

As a result, it is determined that there is a match with the second symbol, and the user can easily confirm the processed content portion and the processing content .

In order to achieve the above object, the processing execution means included in the image processing apparatus of the present invention overwrites the content with the first symbol that has not been determined to be consistent with the second symbol. Can be configured.

Accordingly, it is not determined that the second symbol is consistent, and the user can easily confirm the processed content portion and the processing content .

In order to achieve the above object, in the image processing apparatus of the present invention , when the content is printed on the first paper, the paper identification information managing means identifies the paper identification information for identifying the first paper. It can comprise so that it may have a printing means which prints on the 1st paper of the previous period according to the matching to manage .

Thereby, the electronic data of the content printed on the paper, which is the object of processing, can be specified .

In order to achieve the above object, the second paper read by the image processing apparatus of the present invention can be configured to have identification information of the printed second paper .

Thereby, when the processing content is input using a plurality of sheets of paper , the plurality of sheets of paper can be distinguished .

In order to achieve the above object, the processing symbol management means included in the image processing apparatus of the present invention determines whether the first symbol and the second symbol are the same when determining whether or not they have the coincidence. It can be configured to determine that the similarity is within a predetermined range .

As a result, the user can arbitrarily determine a symbol for designating the processing content and the like. Further, when one user writes the first symbol and the second symbol, the degree of similarity is increased, and the determination is made. The accuracy can be increased .

In order to achieve the above object, the processing symbol management means included in the image processing apparatus of the present invention associates the range to be processed, the content to be processed, and the processing content with the second symbol. It can be configured to hold .

Accordingly, the user can arbitrarily specify the range to be processed, the content to be processed, and the processing details .

In order to achieve the above object, the processing symbol management means included in the image processing apparatus of the present invention includes a selection column for the range to be processed, the content to be processed, and the processing details. Based on the content selected by handwriting on the second paper, the second symbol and the processing content can be made to correspond to each other .

Thereby , the user can specify the processing content corresponding to the symbol.

  To achieve the above object, first stroke data when handwritten on the first paper instead of the first symbol handwritten on the first paper, and handwritten on the second paper Instead of the second symbol, the second stroke data when handwritten on the second paper can be used.

  Thereby, the accuracy in determining the similarity between the first stroke data and the second stroke data can be increased by using the stroke data having a larger amount of information than the symbol.

  In order to achieve the above object, the processing symbol management means of the image processing apparatus of the present invention uses the similarity between the first stroke data and the second stroke data, the coordinate value of the stroke, The writing speed value and the stroke color value can be determined using a value representing one or more strokes.

  Thereby, the precision of the judgment of the similarity between the first stroke data and the second stroke data can be improved.

  In order to achieve the above object, the processing symbol management means included in the image processing apparatus of the present invention includes a stroke color value, a stroke position value, and a stroke, which are the contents of the plurality of second stroke data. The degree of priority for performing the processing corresponding to the second stroke data is held by a value representing one or more strokes among the value of the stroke size and the value of the stroke writing speed. Can do.

  As a result, when a plurality of processes are performed using one first paper, a result obtained by performing all the processes can be unified.

  In order to achieve the above object, the processing symbol management means included in the image processing apparatus of the present invention is configured to determine a priority order for performing the processing for each processing corresponding to the plurality of second stroke data. Can do.

In order to achieve the above object, an image processing method of the present invention includes a content printed on a first paper, paper identification information for identifying the first paper printed on the first paper, and the An image processing method in an image processing apparatus having paper identification information management means for managing the content in association with each other based on the first symbol handwritten on the first paper A first reading step of reading a plurality of processing contents printed on the second paper and a second symbol handwritten for each processing content; a content printed on the first paper; The paper reading information for identifying one paper, the second reading step for reading the first symbol handwritten on the first paper, and the first and second reading steps, When a second symbol and the first symbol are coincident, the processing content corresponding to the second symbol is output, and the processing content is output by the processing symbol management step. based, possess a processing execution step of processing the content corresponding to the paper identification information, said processing execution step, in response to the second determination of whether a match with the symbols, the first One symbol can be configured to overwrite the content .

Thus, when processing the content by printing the content on paper and handwriting on the paper, the processing content, processing range, etc. are designated by an arbitrary symbol defined by the user, and the processed content portion and processing It is possible to provide an image processing method that allows the user to easily confirm the contents .

To achieve the above object, the processing execution step in the image processing method of the present invention is configured to overwrite the content with the first symbol determined to be consistent with the second symbol. be able to.

As a result, it is determined that there is a match with the second symbol, and the user can easily confirm the processed content portion and the processing content .

In order to achieve the above object, in the image processing method of the present invention, the processing execution step overwrites the content with the first symbol that has not been determined to be consistent with the second symbol. can do.

Accordingly, it is not determined that the second symbol is consistent, and the user can easily confirm the processed content portion and the processing content .

In order to achieve the above object, according to the image processing method of the present invention , when the content is printed on the first paper, the paper identification information managing means identifies the paper identification information for identifying the first paper. It can be configured to have a printing step for printing on the first paper in accordance with the managed association .

Thereby, the electronic data of the content printed on the paper, which is the object of processing, can be specified .

To achieve the above object, is read Oite the image processing method of the present invention, the second sheet can be configured to have identification information printed the second sheet.

Thereby, when the processing content is input using a plurality of sheets of paper , the plurality of sheets of paper can be distinguished .

In order to achieve the above object, the processing symbol management step in the image processing method of the present invention is similar to the first symbol and the second symbol when determining whether or not they have the coincidence. It can be configured to determine that the degree is within a predetermined range .

As a result, the user can arbitrarily determine a symbol for designating the processing content and the like. Further, when one user writes the first symbol and the second symbol, the degree of similarity is increased, and the determination is made. The accuracy can be increased .

In order to achieve the above object, the processing symbol management step in the image processing method of the present invention holds the range to be processed, the content to be processed, and the processing content in association with the second symbol. It can be set as the structure to do.

Accordingly, the user can arbitrarily specify the range to be processed, the content to be processed, and the processing details .

In order to achieve the above object, the processing symbol management step in the image processing method of the present invention includes a selection column for the range to be processed, the content to be processed, and the processing details. The second symbol and the processing content can be made to correspond to each other based on the content selected by handwriting on the paper .

Thereby , the user can specify the processing content corresponding to the symbol .

  To achieve the above object, first stroke data when handwritten on the first paper instead of the first symbol handwritten on the first paper, and handwritten on the second paper Instead of the second symbol, the second stroke data when handwritten on the second paper can be used.

  Thereby, the accuracy in determining the similarity between the first stroke data and the second stroke data can be increased by using the stroke data having a larger amount of information than the symbol.

  In order to achieve the above object, the processing symbol management step in the image processing method of the present invention includes the similarity between the first stroke data and the second stroke data, the value of the coordinate of the stroke, and the writing of the stroke. The determination can be made using a value representing one or more strokes among the velocity value and the stroke color value.

  Thereby, the precision of the judgment of the similarity between the first stroke data and the second stroke data can be improved.

  In order to achieve the above object, the processing symbol management step in the image processing method of the present invention includes a stroke color value, a stroke position value, a stroke position, which are the contents of the plurality of second stroke data. Among the size value and the stroke writing speed value, a value representing one or more strokes may be used to hold a priority level for performing processing corresponding to the second stroke data. it can.

  As a result, when a plurality of processes are specified using one first sheet, the result of all the processes can be uniquely determined.

  In order to achieve the above object, the processing symbol management step in the image processing method of the present invention is configured to determine a priority order for performing the processing for each processing corresponding to the plurality of second stroke data. it can.

  In order to achieve the above object, the image processing program of the present invention can be an image processing program for executing the image processing method according to any one of claims 17 to 32.

  Thereby, when processing the content by printing the content on paper and handwriting on the paper, it is possible to provide an image processing program that specifies the processing content, processing range, and the like by an arbitrary symbol defined by the user. .

  According to the present invention, when processing the content by printing the content on paper or the like and handwriting on the paper, the image processing that can specify the processing content, processing range, etc. by any symbol determined by the user An apparatus can be provided.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 19 is a structural principle diagram of the present invention.

  In FIG. 19, when a user fills in a symbol on a first sheet 1910 on which content or the like is printed and reads the first sheet, the image processing apparatus 1940 performs processing on the content and performs processing. Content 1960 is output.

  The image processing device 1940 is a device that processes the content printed on the first paper from the results of reading the first paper 1910 and the second paper 1920. The paper processing information management unit 1941, processing symbol management It comprises means 1942 and processing execution means 1943.

  The paper identification information management means 1941 acquires content data corresponding to the printed paper identification information 1911 from the storage device 1950 from the result of reading the first paper 1910, and outputs it to the process execution means. The paper identification information management means 1941 also compares the result of reading the first paper with the content data acquired from the storage device 1950, so that the first symbol entered by the user on the first paper 1910 1913 is extracted and output to the processing symbol management means 1942.

  The processing symbol management unit 1942 manages the processing content 1921 and the second symbol 1922 entered by the user corresponding to the processing content 1921 based on the result of reading the second paper 1920. When the first symbol 1913 is input from the paper identification information management unit 1941, the first symbol 1913 is compared with the second symbol 1922. When the similarity is within a predetermined range, The processing content 1921 corresponding to the second symbol 1922 is output to the processing execution means 1943.

  The processing execution unit 1943 performs processing of the processing content 1921 input from the processing symbol management unit 1942 on the data of the content 1912 input from the paper identification information management unit 1941, and outputs the processed content 1960.

  The first paper 1910 is paper on which paper identification information 1911 for identifying the paper and content 1912 are printed, and is a paper on which the user has entered the first symbol 1913 by handwriting.

  The second paper 1920 is a paper on which the processing content 1921 is printed, and is a paper on which the user has entered the second symbol 1922 corresponding to the processing content by handwriting. Here, paper identification information for identifying the second paper may be printed on the second paper.

The reading unit 1930 reads the contents printed on the first paper 1910 and the second paper 1920, and the first symbol and the second symbol entered by the user by handwriting, and acquires them as image data. . The storage device 1950 is a device that stores content data.
(Description of FIG. 1)
FIG. 1 shows functional blocks of an image processing apparatus 101 according to the present invention and peripheral devices. The image processing apparatus 101 is an image processing apparatus that processes content printed on paper based on symbols or the like that are handwritten on the paper, and includes a user I / F device 119 and a scanner 116 that is a paper input device. And a digitizer 117, a paper output device 118 such as a printer, and a disk device 107.

  The image processing apparatus 101 includes a user I / F control unit 102, a scanner I / F control unit 108, a digitizer I / F control unit 109, a paper information acquisition unit 110, a writing processing unit 111, a printer I / F control unit 112, and printing. The processing unit 113, the paper identification information management unit 114, and the disk I / F control unit 115 are configured.

  The user I / F control unit 102 receives a paper information acquisition command or a document print command from the external input device, and outputs it to the paper information acquisition unit 110 and the print processing unit 113. In addition, a signal requesting output to the outside input from the writing processing unit 111 is transmitted to an external output device such as the display 103 and the speaker 104. Further, the disk I / F control unit 115 controls the disk device 107 to acquire and store document data as content.

  The scanner I / F control unit 108 controls the scanner 116 to acquire a document image including paper identification information (ID) and a handwritten symbol, and transmits the acquired image to the paper information acquisition unit 110. . The digitizer I / F control unit 109 controls the digitizer 117 to acquire stroke data, which is information on operations when the user performs handwriting on paper, and paper identification information represented by an image, and a paper information acquisition unit 110. Send to.

  In this specification, the paper identification information is expressed as “ID”. The ID is a code indicating information for specifying a document printed on paper, and is, for example, a number uniquely assigned to all paper when the document is printed on paper. A paper identification information management unit 114 (to be described later) manages IDs and document information in association with each other. Document information includes, for example, “file name”, “page number”, “content type”, and “path name or address representing the location of the file on the disk” of the content stored in the storage device. Is managed in association with the ID, the data of the document can be read from a storage device or the like when the ID of the document printed on paper is acquired.

  Although detailed operations of the print processing unit 113, the paper identification information management unit 114, the paper information acquisition unit 110, and the writing processing unit 111 will be described later, an outline of the operation of each unit will be described here.

  The print processing unit 113 adds an ID to the document data stored in the disk device 107 or the like, and generates an image to be output by the printer 118 that is a paper output device. The printer I / F control unit 112 performs processing for printing an image such as a document on paper by controlling the printer 118.

  The paper identification information management unit 114 manages the page identification information management table by using a paper identification information management table so that the relationship between each page of the document and the ID becomes one-to-one, and performs ID registration, deletion, reference, and the like.

  The paper information acquisition unit 110 acquires an ID based on the image read by the scanner 116, the digitizer 117, and the like, and writes the document information corresponding to the ID as the paper information and the read image. Output to.

  Note that in this specification, the image of content such as a document and the document information corresponding to the ID are referred to as “paper information”. The document information includes, for example, “file name”, “page number”, “content type”, “path name or address indicating the position of the file on the disk”, and the like of the content stored in the storage device.

  The writing processing unit 111 acquires the document corresponding to the ID from the disk device 107 via the disk I / F control unit 115 from the information of the document corresponding to the ID input from the paper information acquisition unit 110, and the paper information By processing the image compared with the image input from the acquisition unit 110, a symbol or the like handwritten by the user is acquired, and processing corresponding to the symbol is executed on the document corresponding to the ID.

  The user I / F device 119 is an I / F that is input by the user to the image processing apparatus 110 or displayed by the image processing apparatus 110, and includes a display 103, a speaker 104, a keyboard 105, and a mouse 106. It consists of and. A keyboard 105 and a mouse 106 are devices for a user to input various commands, and input commands for obtaining paper information and commands for printing and outputting a document. The display 103 and the speaker 104 output information necessary for the image processing apparatus 110 when the user performs an operation such as inputting a command.

  The disk device 107 stores document data as content, and the printer 118 forms an image on a paper surface and outputs it.

The scanner 116 is a device that scans the paper surface and acquires an image that is two-dimensional image data. Also, the digitizer 117 of the present embodiment is configured to be able to read the ID, and further is a device that detects a stroke that is a user's stroke operation and converts it into stroke data that is two-dimensional data. Examples of the paper identification information reading device (not shown) include a CCD camera, a two-dimensional code reader, and a one-dimensional bar code reader.
(Explanation of stroke data)
FIG. 3 is an example of stroke data. Each item in the table 301 includes a sequence 302 that is an acquisition order of a plurality of coordinates constituting a handwritten character or symbol, a pen identification information 303 that indicates the type of pen used, and a time stamp that is the acquisition time of the plurality of coordinates. 304, an X component 305 of the plurality of coordinates, a Y component 306 of the plurality of coordinates, a pen down / pen up 307 indicating whether or not the pen and the paper surface are in contact with each other, and It is a stroke label 308 for distinguishing a stroke to which the plurality of coordinates belong. One stroke is a plurality of coordinate points from the operation in which the pen-down / pen-up 307 writes the coordinate point that becomes the pen-down to the operation that writes the coordinate point that becomes the pen-up in the order of the numbers shown in the items of the sequence 302 This is the writing action described. In addition, pen identification information 303 is used in a stroke instead of a color which is one of information representing symbols.

Next, regarding the operations of the print processing unit 113, the paper information acquisition unit 110, the paper identification information management unit 114, and the writing processing unit 111 that perform the acquisition of the paper information and the processing based on the paper information, which are features of the present invention, This will be described below.
(Operation of the print processing unit 113)
FIG. 4 is a flowchart for explaining the operation of the print processing unit 113 when a signal for requesting printing of a document is received from the user I / F control unit 102.

  In step 401, the print processing unit 113 acquires information on a document to be printed from the user I / F control unit. The document information includes, for example, “file name”, “page number”, “document type”, “path name or address indicating the position of the file on the disk”, and the like of the document stored in the disk device 107.

  In step 402, the paper identification information management unit 114 receives document information from the print processing unit 113, issues and registers an ID based on the document information, and transmits the ID to the print processing unit 113.

  In step 403, the print processing unit 113 acquires a document recorded on the disk device 107 via the disk I / F control unit 115 based on the document information. In step 404, the print processing unit 113 converts the ID into a code.

  In step 405, the print processing unit 113 generates an image of the acquired document, and generates a print image in which an ID and a timing mark necessary for performing skew correction by reading the printed paper surface are superimposed. To do.

In step 406, the print processing unit 113 transmits a print image to the printer 118 together with a command for requesting printing.
(Example of management data of the paper identification information management unit 114)
FIG. 5 is an example of an ID table managed by the paper identification information management unit 114.

  The ID table 501 is an ID table that manages a number that uniquely identifies a document as a content and a file name and / or path name of the document, and each column item is a number that uniquely identifies the document. The ID 502 includes a path name and file name 503 indicating an address when the document is read from the disk device, a page number 504 in the document, and a document type 505. Each row of the ID table 501 is information of one document, and a row 506 is information of a document whose ID is 0001. When processing a multi-page document, information having the configuration of the line 506 is required for the number of pages.

The document type 505 is an item for classifying the document type by numerical value. For example, a processing symbol input sheet described later is set to “0”, and other documents are set to “1”, and may be further classified. . Further, in addition to classification by numerical values, classification may be performed by any other data format such as characters.
(Description of print image)
FIG. 6 is an example of a document image, and FIG. 7 is an example of a print image generated by the print processing unit 113 from the document that is the image of FIG.

The image 601 in FIG. 6 includes characters and images. The print processing unit 113 encodes the ID received from the paper identification information management unit 114 and superimposes it on the image 601, thereby generating the image 701 in FIG. 7. In the image 701, marks 702 to 705 are timing marks used when the paper on which the image 701 is printed is read and skew correction is performed. The marks 702 to 705 are formed at predetermined four positions in the image. When distortion occurs during reading, the marks 702 to 705 are compared with a predetermined value of the image processing apparatus 101 to thereby compare the read image. Make corrections. Reference numeral 706 denotes an ID corresponding to the document of the image 601. An image 707 is a portion corresponding to the content of the image 601.
(Operation of the paper information acquisition unit 110)
FIG. 8 is a flowchart for explaining the operation of the paper information acquisition unit 110. FIG. 9 is a diagram for explaining an image in which a user adds a symbol or the like to the paper on which the image 701 is printed.

  In step 801, the paper information acquisition unit 110 acquires a signal specifying a paper input device to be used from the user I / F control unit. In this embodiment, the paper input devices are the scanner 116 and the digitizer 117, and the user designates and inputs one of these paper input devices from the user I / F device 119.

  In step 802, it is determined whether the content of the signal specifying the paper input device received by the paper information acquisition unit 110 is the scanner 116 or the digitizer 117. If the content of the signal is the scanner 116, the processing of steps 803 to 807 is performed.

  In step 803, when the scanner 116 is designated, the paper information acquisition unit 110 instructs the scanner I / F control unit 108 to read the paper surface, and receives image data. In step 804, the paper information acquisition unit 110 detects skew correction marks 702 to 705 from the received image and compares the marks with a specified value to perform skew correction on the image. In step 805, the paper information acquisition unit 116 detects the encoded ID 706 from the image subjected to skew correction, and obtains the document ID by decoding it.

  In step 806, the paper information acquisition unit 110 receives document information based on the acquired document ID from the paper identification information management unit 114. The document information includes, for example, “file name”, “page number”, “content type”, “path name or address indicating the position of the file on the disk”, and the like of the content stored in the storage device.

  In step 807, the paper information acquisition unit 110 transmits paper information including the acquired image data and document information to the writing processing unit 111, and the operation of the paper information acquisition unit 110, which is the processing of this flowchart, ends. .

  In step 802, if the content of the signal specifying the paper input device received by the paper information acquisition unit 110 is the digitizer 117, the processing of steps 808 to 810 is performed.

  In step 808, the paper information acquisition unit 110 instructs the digitizer I / F control unit 109 to stroke data when the user strokes on the printed paper, and the code formed on the printed paper. The acquisition of the image of the converted ID is instructed. The encoded ID image is, for example, an ID represented by a barcode or the like. The paper information acquisition unit 110 acquires the stroke data and the encoded ID image, and processes the ID image to obtain the document ID. In step 809, the paper information acquisition unit 110 receives document information based on the document ID from the paper identification information management unit 114. In step 810, the paper information acquisition unit 110 transmits paper information including the acquired stroke data and document information to the writing processing unit 111, and the operation of the paper information acquisition unit 110, which is the processing of this flowchart, ends.

When there are a plurality of sheets to be read, the processes from step 803 to step 807 or the processes from step 808 to step 810 are performed by the same number as the number of sheets.
(Processing for processing symbol input sheet)
The configuration of the processing symbol input sheet will be described later. Here, the operation of the paper information acquisition unit 110 when the processing symbol input sheet is read will be described.

When the processing symbol input sheet that does not have the marks 702 to 705 and the reference numeral 706 among the processing symbol input sheets is read by the scanner 116, the processing from step 804 to step 806 is not performed. If the processing symbol input sheet not having the reference numeral 706 is read by the digitizer 117, the paper information acquisition unit 110 receives only the stroke data from the digitizer control means 117 in step 808, and in step 809. No processing is performed. In step 810, the paper information acquisition unit 110 transmits only stroke data to the writing processing unit 111.
(Description of the page on which the user has handwritten the symbol)
In FIG. 9, a paper surface 901 is paper on which an image 701 is printed, and is overwritten by a user with a symbol or the like. A paper surface 901 is an element constituting the image 701, timing marks 702 to 705 for skew correction, an encoded ID 706 of the document, and a title character 902, a character region 903, which are contents of the document, and It has an illustration 904. The paper surface 901 further includes symbols 905 to 908 that the user has overwritten on the paper surface on which the image 701 is printed.
(Operation of the writing processing unit 111)
10, 11, and 15 are flowcharts for explaining the operation of the writing processing unit 111.

  FIG. 10 is a diagram for explaining the operation of the writing processing unit 111 when paper information is received from the paper information acquisition unit 110. Steps 1001 to 1005 repeat the process for the number of received paper information, and when there is no unprocessed paper information, the process ends.

  In step 1001, the writing processing unit 111 determines the presence or absence of unprocessed paper information. If there is one or more unprocessed paper information, the process proceeds to step 1002.

  In step 1002, the writing processing unit 111 determines whether or not the document corresponding to the paper information to be processed is a processing symbol-corresponding sheet. If the document is a processing symbol-corresponding sheet, the process proceeds to step 1003. If not, go to Step 1004.

In step 1002, the writing processing unit 111 acquires a symbol or the like corresponding to the processing content based on the paper information of the processing symbol corresponding sheet. In step 1004, the writing processing unit 111 determines whether or not acquisition of symbols or the like corresponding to the processing content has been completed. If acquisition of symbols or the like corresponding to the processing content has been completed, step 1005 is performed. Proceed to If acquisition of the symbol corresponding to the processing content has not been completed, the process returns to step 1001. In step 1005, the writing processing unit 111 compares the symbol corresponding to the processing content acquired in step 1003 with the symbol obtained from the paper information input in step 1005 to determine the coincidence. , Perform the corresponding process.
(Explanation of the flow to process the processing symbol correspondence sheet)
FIG. 11 shows details of the processing in step 1003 and is a flowchart for explaining the operation in which the writing processing unit 111 acquires data such as symbols corresponding to the processing content from the read image of the processing symbol corresponding sheet. The writing processing unit 111 already has paper information including the image or stroke data of the read processing symbol corresponding sheet.

  In step 1101, the writing processing unit 111 receives processing symbol corresponding sheet data corresponding to document information included in the paper information of the writing processing unit 111 from the disk device 107 via the disk I / F control unit 115. To get. When the document information is not included in the paper information, that is, when the encoded ID is not printed on the read paper, the processing symbol corresponding sheet data acquired by the writing processing unit 111 is stored in advance. This is data held by the information processing apparatus 101 or predetermined data stored in advance in the disk device 107 or the like.

  In step 1102, the writing processing unit 111 determines whether the paper information included in the writing processing unit 111 includes an image or stroke data. If yes, go to Step 1107.

  In step 1103, the writing processing unit 111 calculates a pixel-by-pixel difference between the read image included in the paper information and the image generated from the processing symbol corresponding sheet data corresponding to the document information. Then, an image is generated in which only symbols entered by the user by handwriting are drawn.

  In step 1104, the writing processing unit 111 obtains position information of a column to be filled in by the user on the paper from the data of the processing symbol corresponding sheet for the image in which only the symbol generated in step 1103 is drawn. The column is an entry part, and the difference image is extracted as a symbol for each entry part.

In step 1105, the writing processor 111 extracts the partial image extracted in step 1104, and
The data is stored in the disk device 107 or the like as a data pair in association with the processing content corresponding to the position on the paper surface on which the partial image is drawn.

  In step 1106, the writing processing unit 111 determines whether or not the processing in steps 1104 and 1105 has been performed for all entry parts. If there is an unprocessed entry part, the process proceeds to step 1104. When the processing of all the entry portions is completed, the process proceeds to step 1001.

  On the other hand, when the paper information included in the writing processing unit 111 includes stroke data, the processing from step 1107 to step 1109 is performed.

  In step 1107, the writing processing unit 111 obtains position information of fields to be filled in by the user on the paper from the data of the processing symbol corresponding sheet, and sets each field as a partial area, and the stroke included in each partial area. Generate data.

  In step 1108, the writing processing unit 111 stores the stroke data generated in step 1107 and a data pair in which the processing content corresponding to the partial area having the stroke data is associated with the disk device 107 or the like.

In step 1109, the writing processing unit 111 determines whether or not the processing has been completed for all the fields entered by the user included in the processing symbol correspondence sheet. Proceed to 1107, and if all the columns have been completed, proceed to step 1001.
(Explanation of processing symbol sheet)
FIG. 12 is an example of an image of a processing symbol corresponding sheet.

  The processing symbol correspondence sheet image 1201 includes a table 1210, a title character string 1211, columns 1202 to 1205 in which a user enters symbols, and character strings 1206 to 1209 representing processing contents corresponding to the entry columns, respectively. The table 1210 is a table that displays the processing contents in correspondence with the columns in which the user enters symbols, and the title 1211 is a character string that represents the title of the processing symbol corresponding sheet.

  In the entry fields 1202 to 1205, processing for each entered symbol or the like is defined. The entry column 1202 and the entry column 1203 determine whether or not to execute the corresponding processing depending on the presence or absence of a check symbol entered by the user. For example, when a check symbol is handwritten in the entry field 1202, a value indicating that the process is to be executed is associated with the process content for the corresponding process “symbol superimposed display”. The value indicating whether or not the corresponding process is executed may be a Boolean variable. Further, for example, the symbol entered in the entry field 1204 is recorded as image or stroke data in correspondence with the processing content 1208, and the image or stroke data based on the symbol entered in the entry field 1205 is recorded in the processing content 1209. Correspondingly recorded.

  FIG. 13 is an example of an image in which a user enters a symbol on the processing symbol correspondence sheet.

  The processing symbol corresponding sheet image 1301 includes symbols 1302 to 1304 handwritten by the user in addition to the elements included in the processing symbol corresponding sheet image 1201. The symbol 1302 entered in the entry column 1203 corresponds to the character string 1207 representing the processing content, and the symbol 1303 entered in the entry column 1204 corresponds to the character string 1208 representing the processing content and is entered in the entry column 1205. The symbol 1304 is recorded as data corresponding to the character string 1209 representing the processing content.

  Here, the processing symbol corresponding sheet image 1301 includes timing marks 702 to 705 for skew correction, and an encoded ID 706 of the processing symbol corresponding sheet. In such a case, a plurality of processing symbol correspondence sheets 1301 can be used.

  FIG. 14 is an example of a processing symbol management table that manages symbols and processing contents in association with each other based on the processing symbol corresponding sheet image 1301.

The processing contents corresponding to the entry fields 1202 and 1203, “superimpose symbol display” and “superimpose writing other than symbols”, correspond to Boolean variables. The processing contents respectively corresponding to the entry fields 1204 and 1205 are defined for each of “content extraction range”, “type of content to be processed”, and “processing content”. Symbols entered by the user in the processing contents 1303 and 1304 correspond respectively.
(Explanation of consistency judgment process)
FIG. 15 is a flowchart for explaining the process performed by the writing processing unit 111 to determine the coincidence of symbols and the like in step 1005.

  In step 1501, the writing processing unit 111 acquires document data corresponding to the paper information from the disk device 107 via the disk I / F control unit 115. In step 1502, it is determined whether the data included in the paper information is an image or stroke data. If the data is an image, the process proceeds to step 1503. If the data is stroke data, the process proceeds to step 1511.

  In the case of an image, the writing processing unit 111 generates an image from the document data acquired in step 1501 in step 1503, and the difference for each pixel between the image and the read image included in the paper information. To obtain a difference image, thereby obtaining an image composed of symbols handwritten by the user.

  In step 1504, the writing processing unit 111 performs clustering from the proximity and area for each pixel on the difference image generated in step 1503 to obtain a plurality of writing clusters, and for each writing cluster, the vertex of the circumscribed rectangular area Get the coordinates.

  On the other hand, in the case of stroke data, the writing processing unit 111 performs clustering on the stroke data included in the paper information by using the writing area and the time at which the coordinate points are drawn in step 1511, thereby providing a plurality of writing data. A cluster is obtained, and for each writing cluster, the vertex coordinates of the circumscribed rectangle are obtained.

  FIG. 16 is a diagram for explaining a writing cluster extracted from the image 901.

  The symbol written by the user is extracted from the image 901 as a differential image of the image or as stroke data, and is acquired as clusters 905 to 908 by clustering, and circumscribed rectangles 1601 to 1604 are obtained for each cluster. Each of the rectangles 1601 to 1604 is identified by four vertex coordinates.

  In step 1505, the writing processing unit 111 converts the document data corresponding to the ID included in the paper information into PDF (Portable Document Format). In addition, if it is a format that can hold the position on the image of the character area and the image area, which are the elements constituting the image, and can process each element separately, It is not limited to PDF.

  In step 1506, the writing processing unit 111 determines whether or not there is an unprocessed writing cluster. If there is an unprocessed writing cluster, the process proceeds to step 1507, and all the writing clusters have been processed. Then, go to Step 1510.

  In step 1507, the writing processing unit 111 compares the symbols included in the processing symbol management table, which manages the processing contents and the symbols in association with each other based on the processing symbol correspondence sheet, and the one writing cluster. to decide. If it is determined that there is a match, the process proceeds to step 1508, and if it is not determined that there is a match, the process proceeds to step 1506.

  The determination of coincidence uses a known method such as pattern matching. When the symbol is an image, for example, the coordinates of the points constituting the symbol are compared with the coordinates of the writing cluster, the similarity is judged by the ratio of the matching points, and the similarity exceeds a predetermined value. Then, it is determined that the symbols included in the processing symbol management table and the writing cluster have a match.

  Further, when the symbol image has color values in addition to the coordinate points, the symbol color values of the processing symbol management table are compared with the color values of the writing cluster, and the values of these values are compared. When the difference is equal to or smaller than a predetermined value, it is assumed that there is a match, and when both the coordinate and color values have a match, the symbol included in the processing symbol management table and the writing cluster have a match. to decide.

In the case where the symbol is stroke data, the consistency is determined based on , for example, the writing speed in addition to the coordinate and color values as values representing the stroke . The writing speed is calculated by calculating the average writing speed between these points from the time stamp of any one of the points constituting the stroke data and the time stamp of the point immediately before the writing order. Calculate and use this.

In step 1508, for the writing cluster that the writing processing unit 111 has determined to have consistency, the processing determined to have consistency with the writing cluster within a predetermined range of the position on the image 901 of the writing cluster. It is determined whether or not there is content satisfying the processing content corresponding to the symbol included in the symbol management table. If the content exists, the process proceeds to step 1509. If not, the process proceeds to step 1506.
For example, if it is determined that the symbol 1304 and the writing cluster 905 are consistent, the content of “image” exists in “in the symbol area”, which is the processing content corresponding to the symbol 1304 shown in FIG. Judge whether to do. The writing processing unit 111 determines from the coordinates of each element included in the PDF data of the image 901 that the image region 904 exists in the region of the writing cluster 905, and the process proceeds to step 1509.

  In step 1509, the processing contents corresponding to the symbol having the consistency with the writing cluster are executed for the content existing in the predetermined range of the position of the writing cluster in step 1508. For example, “acquisition” processing, which is processing content corresponding to the symbol 1304, is performed on the image content 904 existing in the writing cluster 905. The acquired image data is stored in accordance with an input from the user I / F device 119, for example.

Step 1510 includes, for all the writing clusters, when the processing from Step 1507 to Step 1509 is completed, the writing cluster that is not determined to be consistent with the symbols included in the processing symbol management table in Step 1507, and For each writing cluster that is determined to have a match, a process defined in advance in the process symbol management sheet or the like is performed. For example, when the writing clusters 907 and 908 are not determined to have the coincidence, the writing clusters 907 and 908 are overwritten on the image 601 based on the processing symbol management table of FIG.
(Configuration variation)
FIG. 2 shows an image processing apparatus 201 of the present invention in which input / output devices are connected via a network. The image processing apparatus 201 is connected to a user I / F device 119, and is connected to a network scanner 204, a network digitizer 205, a network printer 206, a network server 207, and a client PC 208 via a network 203.

  The image processing apparatus 201 replaces the scanner I / F control unit 108, the digitizer control unit 109, the printer I / F control unit 112, and the disk I / F control unit 115 included in the image processing apparatus 101 with a network I / F. The network I / F control unit 202 controls various devices connected to the network 203 and inputs / outputs data.

  The network server 207 includes a disk device 119, and performs data output from the disk device 119 and processing for storing data. The client PC 208 performs processing such as data transfer by FTP (File Transfer Protocol) and data storage.

In FIG. 2, a paper input device, a paper output device, and the like are connected to the image processing device 201 via a network. However, some of these devices are connected via a network, and other devices are connected to the image processing device 101. It may be configured to be directly connected and controlled by various I / F control units included in the.
(Modification of processing symbol management sheet)
FIG. 17 shows an example different from FIG. 12 of the processing symbol management sheet.

  An image 1701 is an image of a processing symbol management sheet, and includes selection columns 1702 to 1704 instead of the character string 1204 indicating the processing content, and selection columns 1705 to 1707 instead of the character string 1205 indicating the processing content. is doing.

The selection columns 1702 to 1707 correspond to “content extraction range”, “type of content to be processed”, and “processing content”, which are items representing the processing content in FIG. When the user enters a check symbol in each option included in the selection columns 1702 to 1707, the contents of each item representing the processing contents are determined.
(Modification of the shape of the writing cluster)
FIG. 18 is an explanatory diagram showing the shape of the writing area of the writing cluster. In the embodiment of the present invention, the writing area of the writing cluster 906 is expressed by using a circumscribed rectangle 1601, but is not necessarily limited to a rectangle, and for example, a shape surrounding the writing cluster 906, such as an area 1801. If it is good.
(Example of computer program)
FIG. 20 is an explanatory diagram for constructing the present invention by a computer program.

  A computer 2001 is a computer having an image processing program for processing content printed on paper based on symbols handwritten on the paper, and includes a user I / F device 119, a paper input device 2005, a printer. Are connected to the paper output device 118 and the disk device 108.

  A computer 2001 includes a CPU 2002, a ROM 2003, and a RAM 2004. The CPU 2002 executes a program recorded in the ROM 2003, so that various controls of parts constituting the computer and devices connected to the computer are performed. 116 to 119 and the disk device 107 are controlled.

The ROM 2003 holds an image processing program that is an embodiment of the present invention. A RAM 2004 is a memory that stores temporary data and the like when the CPU 2002 performs processing.
(Example of other processing)
In the above example, the processing for acquiring image data from content and changing the content is described. However, the present invention is not limited to the above example, and for example, the content is output by printing. In this case, it is possible to carry out predetermined processing such as when content is reproduced.
Although the best mode for carrying out the invention has been described above, the present invention is not limited to the embodiment described in the best mode. Modifications can be made without departing from the spirit of the present invention.

Example of image processing apparatus in this embodiment Example of connecting the image processing apparatus according to the present embodiment to a device via a network Example of stroke data Flow chart explaining the operation of the print processing unit Example of ID table Example document image Example of a document image printed on paper with an ID image added Flow chart explaining the operation of the paper information acquisition unit Example of a document image in which the user has handwritten symbols, etc. Flow chart explaining the operation of the writing processor Flow chart of operation to obtain information from processing symbol compatible sheet Example of processing symbol image Example of processing symbol-compatible sheet image in which the user has handwritten symbols Processing symbol management table example Flow chart of processing to determine coincidence of symbols etc. Illustration of the handwritten cluster extracted from the image Example of processing symbol compatible sheet with selection field Example of the shape of the writing area of a writing cluster Configuration principle diagram of the present invention Embodiment by the image processing program of the present invention

Explanation of symbols

111 Writing Processing Unit 110 Paper Information Acquisition Unit 114 Paper Identification Information Management Unit 113 Print Processing Unit 1941 Paper Identification Information Management Unit 1942 Processing Symbol Management Unit 1943 Processing Execution Unit

Claims (25)

Based on the content printed on the first paper, the paper identification information for identifying the first paper printed on the first paper, and the first symbol handwritten on the first paper, In an image processing apparatus for processing the content,
Paper identification information management means for managing the paper identification information and the content in association with each other;
Based on the result of reading a plurality of processing contents printed on the second paper and a second symbol handwritten for each processing content, the first symbol handwritten on the first paper and the first symbol Processing symbol management means for outputting processing contents corresponding to the second symbol for the content printed on the first paper when the two symbols have a match;
Based on the processing contents to be outputted by the processing symbol managing unit, it possesses a process executing means for processing the content corresponding to the paper identification information,
The image processing apparatus , wherein the processing execution means overwrites the content with the first symbol in accordance with a determination as to whether or not the second symbol has consistency. . The image processing apparatus according to claim 1, wherein the processing execution unit overwrites the content with the first symbol determined to have consistency with the second symbol. The image processing apparatus according to claim 1, wherein the processing execution unit overwrites the content with the first symbol that is not determined to be consistent with the second symbol. When printing the content on the first paper, printing means for printing the paper identification information for identifying the first paper on the first paper according to the association managed by the paper identification information management means the image processing apparatus to 3 according claims 1, characterized in that it has. The second paper, the image processing apparatus to 4 according claims 1 and having identification information printed the second sheet. The processing symbol management means determines that the similarity between the first symbol and the second symbol is within a predetermined range when determining whether or not there is the coincidence. It claims 1 to to 5 image processing apparatus according to any one claim. The image processing apparatus according to claim 5, wherein the processing symbol management unit holds a processing range, content to be processed, and processing content in association with the second symbol. The processing symbol management means, based on the content selected by handwriting on the second paper having a selection field for the range to be processed, the content to be processed, and the processing content, the image processing apparatus according to claim 5 or 6, characterized in that in matching the processing contents as the second symbol. In place of the first symbol handwritten on the first paper, the first stroke data when handwritten on the first paper, and the second symbol handwritten on the second paper 9. The image processing apparatus according to claim 1, wherein second stroke data when handwritten on the second paper is used instead. The processing symbol management means calculates the similarity between the first stroke data and the second stroke data by selecting one of a stroke coordinate value, a stroke writing speed value, and a stroke color value. The image processing apparatus according to claim 9, wherein the determination is made using a value representing the above stroke. The processing symbol management means includes a content of a plurality of the second stroke data, a stroke color value, a stroke position value, a stroke size value, and a stroke writing speed value. The image processing apparatus according to claim 9, wherein a priority level for performing processing corresponding to the second stroke data is held by a value representing one or more strokes. 12. The image processing apparatus according to claim 11, wherein the processing symbol management means determines a priority order for performing the processing for each processing corresponding to the plurality of second stroke data. Based on the content printed on the first paper, the paper identification information for identifying the first paper printed on the first paper, and the first symbol handwritten on the first paper, In an image processing method in an image processing apparatus having a paper identification information management unit that performs processing of content and manages the paper identification information and the content in association with each other,
A first reading step of reading a plurality of processing contents printed on the second paper and a second symbol handwritten for each processing content;
A second reading step of reading the content printed on the first paper, the paper identification information for identifying the first paper, and the first symbol handwritten on the first paper;
A processing symbol that outputs the processing content corresponding to the second symbol when the second symbol and the first symbol read in the first and second reading steps have a match. Administrative steps;
Based on the processing content that is output by the process symbol management step, it possesses a processing execution step of processing the content corresponding to the paper identification information,
The image processing method characterized in that, in the processing execution step, the first symbol is overwritten on the content in accordance with a determination as to whether or not the second symbol is consistent. . The image processing method according to claim 13, wherein the processing execution step overwrites the content with the first symbol determined to have consistency with the second symbol. The image processing method according to claim 13, wherein the processing execution step overwrites the content with the first symbol that is not determined to be consistent with the second symbol. When printing the content on the first paper, a printing step of printing the paper identification information for identifying the first paper on the first paper according to the association managed by the paper identification information management means 16. The image processing method according to claim 13, further comprising: It said second paper printed the second image processing method of claims 13 to 16, wherein the having the identification information of the paper. In the processing symbol management step, it is determined that the similarity between the first symbol and the second symbol is within a predetermined range when it is determined whether or not there is the coincidence. It claims 13 to 1 7 image processing method according to any one claim to. The image processing method according to claim 18, wherein the processing symbol management step retains the range to be processed, the content to be processed, and the processing content in association with the second symbol. . In the processing symbol management step, the range to be processed, the content to be processed, and the processing content, based on the content selected by the user by handwriting on the second paper having a selection field, claim 1 8 or 1 9 image processing method, wherein the in matching the processing contents as the second symbol. In place of the first symbol handwritten on the first paper, the first stroke data when handwritten on the first paper, and the second symbol handwritten on the second paper 21. The image processing method according to claim 13, wherein the second stroke data when handwritten on the second paper is used instead. In the processing symbol management step, the similarity between the first stroke data and the second stroke data is calculated based on one of a stroke coordinate value, a stroke writing speed value, and a stroke color value. The image processing method according to claim 21, wherein the determination is made using a value representing the above stroke. The processing symbol management step includes a content of a plurality of the second stroke data, a stroke color value, a stroke position value, a stroke size value, and a stroke writing speed value. 23. The image processing method according to claim 21, wherein a priority level for performing processing corresponding to the second stroke data is held by a value representing one or more strokes. 24. The image processing method according to claim 23, wherein the processing symbol management step determines a priority order for performing the processing for each processing corresponding to the plurality of second stroke data. An image processing program for causing a computer to execute the image processing method according to any one of claims 13 to 24.

Images